Search NASA⌕ Search

Engineering topics

Whitelam, Stephen

Publications and source records attributed to Whitelam, Stephen.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Using the Metropolis algorithm to explore the loss surface of a recurrent neural network

In the limit of small trial moves the Metropolis Monte Carlo algorithm is equivalent to gradient descent on the energy function in the presence of Gaussian white noise. This observation was originally used to demonstrate a correspondence between Metropolis Monte Carlo moves of model molecules and overdamped Langevin dynamics, but it also applies in the context of training a neural network: making small random changes to the weights of a neural network, accepted with the Metropolis probability, with the loss function playing the role of energy, has the same effect as training by explicit gradient descent in the presence of Gaussian white noise. We explore this correspondence in the context of a simple recurrent neural network. We also explore regimes in which this correspondence breaks down, where the gradient of the loss function becomes very large or small. In these regimes the Metropolis algorithm can still effect training, and so can be used as a probe of the loss function of a neural network in regimes in which gradient descent struggles. We also show that training can be accelerated by making purposely-designed Monte Carlo trial moves of neural-network weights.

Casert, Corneel↗
DOE OSTI2024

Learning protocols for the fast and efficient control of active matter

Exact analytic calculation shows that optimal control protocols for passive molecular systems often involve rapid variations and discontinuities. However, similar analytic baselines are not generally available for active-matter systems, because it is more difficult to treat active systems exactly. Here we use machine learning to derive efficient control protocols for active-matter systems, and find that they are characterized by sharp features similar to those seen in passive systems. We show that it is possible to learn protocols that effect fast and efficient state-to-state transformations in simulation models of active particles by encoding the protocol in the form of a neural network. We use evolutionary methods to identify protocols that take active particles from one steady state to another, as quickly as possible or with as little energy expended as possible. Our results show that protocols identified by a flexible neural-network ansatz, which allows the optimization of multiple control parameters and the emergence of sharp features, are more efficient than protocols derived recently by constrained analytical methods. Our learning scheme is straightforward to use in experiment, suggesting a way of designing protocols for the efficient manipulation of active matter in the laboratory.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2024

Free-energy estimates from nonequilibrium trajectories under varying-temperature protocols

The Jarzynski equality allows the calculation of free-energy differences using values of work measured from nonequilibrium trajectories. The number of trajectories required to accurately estimate free-energy differences in this way grows sharply with the size of work fluctuations, motivating the search for protocols that perform desired transformations with minimum work. However, protocols of this nature can involve varying temperature, to which the Jarzynski equality does not apply. Here, we derive a variant of the Jarzynski equality that applies to varying-temperature protocols, and show that it can have better convergence properties than the standard version of the equality. We derive this modified equality and the associated fluctuation relation within the framework of Markovian stochastic dynamics, complementing related derivations done within the framework of Hamiltonian dynamics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Nonequilibrium formulation of varying-temperature bit erasure

Landauer's principle states that erasing a bit of information at fixed temperature T costs at least $k$ B $T$ ln 2 units of work. Here we investigate erasure at varying temperature, to which Landauer's result does not apply. Here we formulate bit erasure as a stochastic nonequilibrium process involving a compression of configuration space, with physical and logical states associated in a symmetric way. Erasure starts and ends at temperature T, but temperature can otherwise vary with time in an arbitrary way. Defined in this way, erasure is governed by a set of nonequilibrium fluctuation relations that show that varying-temperature erasure can done with less work than $k$ B $T$ ln 2. As a result, erasure and the complementary process of bit randomization can be combined to form a work-producing engine cycle.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

On the origin of cooperativity effects in the formation of self-assembled molecular networks at the liquid/solid interface

In this work we investigate the behaviour of molecules at the nanoscale using scanning tunnelling microscopy in order to explore the origin of the cooperativity in the formation of self-assembled molecular networks (SAMNs) at the liquid/solid interface. By studying concentration dependence of alkoxylated dimethylbenzene, a molecular analogue to 5-alkoxylated isophthalic derivatives, but without hydrogen bonding moieties, we show that the cooperativity effect can be experimentally evaluated even for low-interacting systems and that the cooperativity in SAMN formation is its fundamental trait. We conclude that cooperativity must be a local effect and use the nearest-neighbor Ising model to reproduce the coverage vs. concentration curves. The Ising model offers a direct link between statistical thermodynamics and experimental parameters, making it a valuable tool for assessing the thermodynamics of SAMN formation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Learning stochastic dynamics and predicting emergent behavior using transformers

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Remembering the Work of Phillip L. Geissler: A Coda to His Scientific Trajectory

Phillip L. Geissler made important contributions to the statistical mechanics of biological polymers, heterogeneous materials, and chemical dynamics in aqueous environments. He devised analytical and computational methods that revealed the underlying organization of complex systems at the frontiers of biology, chemistry, and materials science. In this retrospective we celebrate his work at these frontiers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dynamics of Polymer Nanocapsule Buckling and Collapse Revealed by In Situ Liquid-Phase TEM

Nanocapsules are hollow nanoscale shells that have applications in drug delivery, batteries, self-healing materials, and as model systems for naturally occurring shell geometries. In many applications, nanocapsules are designed to release their cargo as they buckle and collapse, but the details of this transient buckling process have not been directly observed. In this work, we use in situ liquid-phase transmission electron microscopy to record the electron-irradiation-induced buckling in spherical 60-187 nm polymer capsules with ~3.5 nm walls. We observe in real time the release of aqueous cargo from these nanocapsules and their buckling into morphologies with single or multiple indentations. The in situ buckling of nanoscale capsules is compared to ex situ measurements of collapsed and micrometer-sized capsules and to Monte Carlo (MC) simulations. The shape and dynamics of the collapsing nanocapsules are consistent with MC simulations, which reveal that the excessive wrinkling of nanocapsules with ultrathin walls results from their large Föppl-von Kármán numbers around 10 5 . Our experiments suggest design rules for nanocapsules with the desired buckling response based on parameters such as capsule radius, wall thickness, and collapse rate.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Here using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Dynamical Large Deviations of Two-Dimensional Kinetically Constrained Models Using a Neural-Network State Ansatz

We use a neural-network ansatz originally designed for the variational optimization of quantum systems to study dynamical large deviations in classical ones. We use recurrent neural networks to describe the large deviations of the dynamical activity of model glasses, kinetically constrained models in two dimensions. We present the first finite size-scaling analysis of the large-deviation functions of the two-dimensional Fredrickson-Andersen model, and explore the spatial structure of the high-activity sector of the South-or-East model. These results provide a new route to the study of dynamical large-deviation functions, and highlight the broad applicability of the neural-network state ansatz across domains in physics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Decorated networks of native proteins: nanomaterials with tunable mesoscopic domain size

Natural and artificial proteins with designer properties and functionalities offer unparalleled opportunity for functional nanoarchitectures formed through self-assembly. However, to exploit this potential we need to design the system such that assembly results in desired architecture forms while avoiding denaturation and therefore retaining protein functionality. Here we address this challenge with a model system of fluorescent proteins. By manipulating self-assembly using techniques inspired by soft matter where interactions between the components are controlled to yield the desired structure, we have developed a methodology to assemble networks of proteins of one species which we can decorate with another, whose coverage we can tune. Consequently, the interfaces between domains of each component can also be tuned, with potential applications for example in energy – or electron – transfer. Our model system of eGFP and mCherry with tuneable interactions reveals control over domain sizes in the resulting networks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Neuroevolutionary Learning of Particles and Protocols for Self-Assembly

Within simulations of molecules deposited on a surface we show that neuroevolutionary learning can design particles and time-dependent protocols to promote self-assembly, without input from physical concepts such as thermal equilibrium or mechanical stability and without prior knowledge of candidate or competing structures. Furthermore, the learning algorithm is capable of both directed and exploratory design: it can assemble a material with a user-defined property, or search for novelty in the space of specified order parameters. In the latter mode it explores the space of what can be made, rather than the space of structures that are low in energy but not necessarily kinetically accessible.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Hysteresis curves reveal the microscopic origin of cooperative CO 2 adsorption in diamine-appended metal–organic frameworks

Diamine-appended metal-organic frameworks (MOFs) of the form Mg 2 (dobpdc)(diamine) 2 adsorb CO 2 in a cooperative fashion, exhibiting an abrupt change in CO 2 occupancy with pressure or temperature. This change is accompanied by hysteresis. While hysteresis is suggestive of a first-order phase transition, we show that hysteretic temperature-occupancy curves associated with this material are qualitatively unlike the curves seen in the presence of a phase transition; they are instead consistent with CO 2 chain polymerization, within one-dimensional channels in the MOF, in the absence of a phase transition. Here, our simulations of a microscopic model reproduce this dynamics, providing a physical understanding of cooperative adsorption in this industrially important class of materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Varied phenomenology of models displaying dynamical large-deviation singularities

Singularities of dynamical large-deviation functions are often interpreted as the signal of a dynamical phase transition and the coexistence of distinct dynamical phases, by analogy with the correspondence between singularities of free energies and equilibrium phase behavior. Here we study models of driven random walkers on a lattice. These models display large-deviation singularities in the limit of large lattice size, but the extent to which each model's phenomenology resembles a phase transition depends on the details of the driving. We also compare the behavior of ergodic and nonergodic models that present large-deviation singularities. Finally, we argue that dynamical large-deviation singularities indicate the divergence of a model timescale, but not necessarily one associated with cooperative behavior or the existence of distinct phases.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Watch and learn—a generalized approach for transferrable learning in deep neural networks via physical principles

Transfer learning refers to the use of knowledge gained while solving a machine learning task and applying it to the solution of a closely related problem. Such an approach has enabled scientific breakthroughs in computer vision and natural language processing where the weights learned in state-of-the-art models can be used to initialize models for other tasks which dramatically improve their performance and save computational time. Here we demonstrate an unsupervised learning approach augmented with basic physical principles that achieves fully transferrable learning for problems in statistical physics across different physical regimes. By coupling a sequence model based on a recurrent neural network to an extensive deep neural network, we are able to learn the equilibrium probability distributions and inter-particle interaction models of classical statistical mechanical systems. Our approach, distribution-consistent learning, DCL, is a general strategy that works for a variety of canonical statistical mechanical models (Ising and Potts) as well as disordered interaction potentials. Using data collected from a single set of observation conditions, DCL successfully extrapolates across all temperatures, thermodynamic phases, and can be applied to different length-scales. This constitutes a fully transferrable physics-based learning in a generalizable approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Insights into the Structure and Energy of DNA Nanoassemblies

Since the pioneering work of Ned Seeman in the early 1980s, the use of the DNA molecule as a construction material experienced a rapid growth and led to the establishment of a new field of science, nowadays called structural DNA nanotechnology. Here, the self-recognition properties of DNA are employed to build micrometer-large molecular objects with nanometer-sized features, thus bridging the nano- to the microscopic world in a programmable fashion. Distinct design strategies and experimental procedures have been developed over the years, enabling the realization of extremely sophisticated structures with a level of control that approaches that of natural macromolecular assemblies. Nevertheless, our understanding of the building process, i.e., what defines the route that goes from the initial mixture of DNA strands to the final intertwined superstructure, is, in some cases, still limited. In this review, we describe the main structural and energetic features of DNA nanoconstructs, from the simple Holliday junction to more complicated DNA architectures, and present the theoretical frameworks that have been formulated until now to explain their self-assembly. Deeper insights into the underlying principles of DNA self-assembly may certainly help us to overcome current experimental challenges and foster the development of original strategies inspired to dissipative and evolutive assembly processes occurring in nature.

59 BASIC BIOLOGICAL SCIENCES↗