Search NASA⌕ Search

SEARCH · Search NASA

Results for “stochastic control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Electrodeposition of ligand-free copper nanoparticles from aqueous nanodroplets

Copper nanoparticles have emerged as promising electrocatalysts for energy storage and conversion. Generally, homogenous nanoparticle synthesis requires a stabilizing ligand, which may influence the electrocatalysis. Ligands can be avoided by direct nanoparticle electrodeposition. Here, we extend nanodroplet-mediated electrodeposition to the electrodeposition of copper nanoparticles from aqueous nanodroplets suspended in 0.1 M tetrabutylammonium hexafluorophosphate ([TBA][PF 6 ]) and 1,2-dichloroethane. Stochastic electrochemistry on microelectrodes was used to elucidate nanoparticle growth kinetics for various solution conditions and substrate materials. A study on the effect of surfactants on nanoparticle morphology and growth kinetics demonstrated the surfactant's role as an avenue for morphological control. Nanoparticle morphology was studied by Scanning Electron Microscopy (SEM) and Transmission Electron Microscopy (TEM). Energy-dispersive X-ray Spectroscopy (EDX) confirmed the presence of copper nanoparticles. TEM and Selected Area Electron Diffraction (SAED) confirmed nanoparticle size, morphology, and polycrystallinity. We demonstrate tunable nanoparticle size using cyclic voltammetry on single aqueous nanodroplets by altering the voltammetric sweeping potentials of copper electroreduction. Finite element simulations validate the voltammetric response and ability to control the nanoparticle size with nanometer resolution during a voltammetric sweep. These results inform the electrodeposition of copper nanoparticles from aqueous nanodroplets for important applications, such as the conversion of CO 2 to hydrocarbon fuels.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Unsupervised anomaly clustering via offset alignment in multivariate grid sensing data

Modern industries increasingly rely on multi-sensor technologies to acquire complex, high-dimensional data streams, enabling advanced monitoring and control systems. One critical application is online anomaly detection in electrical smart grids, where multivariate and multimodal sensing technologies play a vital role. However, detecting anomalies in such time-series data is challenging due to their inherent temporal dependencies and stochastic behavior. Traditional approaches based on supervised and semi-supervised learning methods depend on labeled datasets, which are often unavailable in real-world scenarios. While unsupervised methods have emerged as promising alternatives, these methods are highly susceptible to noise and outliers commonly present in sensing applications. Furthermore, deep learning-based anomaly detection methods, despite their performance, are often criticized for their black-box nature, limiting their applicability in safety-critical and online environments where interpretability and explainability are paramount. In this work, we propose an unsupervised anomaly clustering method leveraging a cyclic alignment-based offset detection algorithm for multivariate time-series signals. The proposed method is applied to multivariate data collected from vibrational, voltage, and magnetic field sensors deployed in a local grid substation. Our results demonstrate the robustness of the algorithm in accurately clustering various anomalies/events across different sensing modalities. Additionally, we compare the effectiveness of the proposed approach against a simple pattern-based anomaly detection method, which performs well for univariate data but fails to generalize to multivariate and multimodal time-series data.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗
DOE OSTI2023

Risk-Constrained Reinforcement Learning for Inverter-Dominated Power System Controls

Here, this paper develops a risk-aware controller for grid-forming inverters (GFMs) to minimize large frequency oscillations in GFM inverter-dominated power systems. To tackle the high variability from loads/renewables, we incorporate a mean-variance risk constraint into the classical linear quadratic regulator (LQR) formulation for this problem. The risk constraint aims to bound the time-averaged cost of state variability and thus can improve the worst-case performance for large disturbances. The resulting risk-constrained LQR problem is solved through the dual reformulation to a minimax problem, by using a reinforcement learning (RL) method termed as stochastic gradient-descent with max-oracle (SGDmax). In particular, the zero-order policy gradient (ZOPG) approach is used to simplify the gradient estimation using simulated system trajectories. Numerical tests conducted on the IEEE 68-bus system have validated the convergence of our proposed SGDmax for GFM model and corroborate the effectiveness of the risk constraint in improving the worst-case performance while reducing the variability of the overall control cost.

Frequency control↗
DOE OSTI2021

The Phase-Locking of Tropical North Atlantic and the Contribution of ENSO

The Tropical North Atlantic (TNA) is characterized by significant interannual variability in sea surface temperature (SST), which is phase-locked to the boreal spring. In this study, the phase-locking of TNA is investigated by adopting a linear stochastic-dynamical model (SDM) using seasonally modulated TNA feedbacks together with the seasonal modulation of ENSO forcing. In the observations, the role of local TNA feedbacks and ENSO forcing in TNA phase-locking are equivalently important with both preferring the peak of TNA variability to appear in the boreal spring. Besides, the seasonal modulation of TNA feedbacks and ENSO forcing strength are both mainly controlled by thermodynamic processes. In most climate models, the contribution of ENSO on TNA phase-locking is weaker than that in observations. The strength of ENSO-related TNA phase-locking is highly correlated with the relationship between ENSO and TNA, which is mainly determined by the amplitude of ENSO and its teleconnection patterns.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Real-Time KMC Simulation of Vacancy-Mediated Intermixing in Au@Ag Octahedral Core–Cubic Shell Nanocrystals with Ab Initio-Guided Kinetics

Utilization of core–shell rather than monometallic nanocrystals (NCs) facilitates fine-tuning of NC properties for applications. However, compositional evolution via intermixing can degrade these properties prompting recent experimental studies. We develop an atomistic-level stochastic model for vacancy-mediated intermixing exploiting a formalism which allows incorporation at an ab initio density functional theory level of not just the thermodynamics of vacancy formation, but also relevant diffusion barriers for a vast number of possible local environments (in the core and in the shell, at the interface, and in the intermixed phase). This facilitates a predictive treatment and comprehensive understanding of intermixing on the relevant time scale (e.g., 10 1 –10 3 s). In contrast, previous modeling at the atomistic level utilized only unrealistic generic prescriptions of barriers or employed simplified continuum treatments. For Au@Ag octahedral core–cubic shell NCs, our modeling not only captures the experimentally observed rate or time scale for intermixing of ~100 s at 450 °C for 60 nm NCs, but also elucidates the underlying rate controlling processes and the effective intermixing barrier.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Training reinforcement learning models via an adversarial evolutionary algorithm

When training for control problems, more episodes used in training usually leads to better generalizability, but more episodes also requires significantly more training time. There are a variety of approaches for selecting the way that training episodes are chosen, including fixed episodes, uniform sampling, and stochastic sampling, but they can all leave gaps in the training landscape. In this work, we describe an approach that leverages an adversarial evolutionary algorithm to identify the worst performing states for a given model. We then use information about these states in the next cycle of training; this process can be repeated until the desired level of model performance is met. We demonstrate this approach with the OpenAI Gym cart-pole problem. With this problem, we show that the adversarial evolutionary algorithm did not reduce the number of episodes required in training needed to attain model generalizability when compared with stochastic sampling, and actually performed slightly worse.

Coletti, Mark↗
DOE OSTI2020

Toroidal modeling of runaway electron loss due to 3-D fields in DIII-D and COMPASS

The 3-D field induced relativistic runaway electron (RE) loss has been simulated for DIII-D, COMPASS and ITER plasmas, utilizing the MARS-F code incorporated with the recently developed and updated RE orbit module (REORBIT). Modeling shows effectively 100% loss of a post-disruption, high-current runaway beam in DIII-D, due to the 1 kG level of magnetic field perturbation produced by a fast growing n = 1 resistive kink instability. The RE loss is shown to be independent of the particle energy or the initial location of particles in the configuration space. Applied resonant magnetic perturbation (RMP) fields from in-vessel coils are not effective for RE beam mitigation in DIII-D, but do produce finite (>10%) RE loss in COMPASS, consistent with experimental observations in above two devices. The major reasons for this difference in RE control by RMP between these two devices are (i) the coil proximity to the RE beam and (ii) the effective coil current scaling versus the machine size and the toroidal magnetic field. About 10% RE loss fraction is also predicted for an ITER 15 MA scenario with pre-disruption plasma, highlighting the role of the plasma response. Up to 30% loss is computed, however, by artificially scaling the equilibrium pressure to zero. This is due to the more resistive plasma response and stronger resulting field line stochasticity. Distributions of the lost REs to the limiting surface show poloidally peaked profile near the high-field-side in both DIII-D and COMPASS, covering about 100o poloidal angle. Higher perturbation field level and/or higher particle energy also result in REs being lost to the low-field-side of the limiting surface of these two devices, increasing the effective wetted area. Finally, for the modeled ITER plasmas, the applied RMP field, with optimal poloidal spectrum that maximizes the edge localized mode control, leads to REs being lost to the lower divertor region of the limiting surface within a narrow poloidal band.

3-D field↗
DOE OSTI2022

Simulation study of fast ion losses associated with the rotating n = 1 resonant magnetic perturbations in KSTAR

In this work we describe a simulation framework for testing the fast ion loss mechanism associated with the experimentally observed beam ion losses when an externally applied toroidally rotating perturbed magnetic field is used to control edge localized modes in the KSTAR tokamak. The simulations reproduce the key qualitative features of neutral beam injection (NBI) ion detection by a fast ion loss detector. The NBI ion losses in the simulation mainly occur for passing particles due to orbit stochastization, which is caused by orbit resonance with the 3D field perturbations. The relative toroidal angle of NBI ion deposition to the orbit island determines the radial path of the particles between confinement or loss. The fast ion loss quantity depends on the toroidal angle rotation of the 3D field with respect to the beam deposition position. The averaged transport of toroidal canonical angular momentum reveals that linear and nonlinear resonance of the NBI ions within the 3D field are the main factors determining fast ion transport and loss.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Fundamental Behavior of ENSO Phase Locking

El Niño–Southern Oscillation (ENSO) events tend to peak at the end of the calendar year, a phenomenon called ENSO phase locking. This phase locking is a fundamental ENSO property that is determined by its basic dynamics. The conceptual ENSO recharge oscillator (RO) model is adopted to examine the ENSO phase-locking behavior in terms of its peak time, strength of phase locking, and asymmetry between El Niño and La Niña events. The RO model reproduces the main phase-locking characteristics found in observations, and the results show that the phase locking of ENSO is mainly dominated by the seasonal modulation of ENSO growth/decay rate. In addition, the linear/nonlinear mechanism of ENSO phase preference/phase locking is investigated using RO model. The difference between the nonlinear phase-locking mechanism and linear phase-preference mechanism is largely smoothed out in the presence of noise forcing. Further, the impact on ENSO phase locking from annual cycle modulation of the growth/decay rate, stochastic forcing, nonlinearity, and linear frequency are examined in the RO model. The preferred month of ENSO peak time depends critically on the phase and strength of the seasonal modulation of the ENSO growth/decay rate. Furthermore, the strength of phase locking is mainly controlled by the linear growth/decay rate, the amplitude of seasonal modulation of growth/decay rate, the amplitude of noise, the SST-dependent factor of multiplicative noise, and the linear frequency. The asymmetry of the sharpness of ENSO phase locking is induced by the asymmetric effect of state-dependent noise forcing in El Niño and La Niña events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

The Construction of Reduced Order Models for the HYBRID Repository

Reduced order models (ROMs) present an opportunity to reduce the computational burden of IES analysis using HYBRID models without significantly sacrificing data accuracy. Other applications of ROMs are digital twins, stochastic optimization studies, model exporting, and protection of proprietary information. In this paper the capability of creating ROMs of the models within the HYBRID repository will be demonstrated. Specifically, a linear regression ROM of a gas turbine and a dynamic mode decomposition with control (DMDc) ROM of a lithium-ion battery are constructed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Master equation emulation and coherence preservation with classical control of a superconducting qubit

Open quantum systems are a topic of intense theoretical research. The use of master equations to model a system's evolution subject to an interaction with an external environment is one of the most successful theoretical paradigms. General experimental tools to study different open system realizations have been limited, and so it is highly desirable to develop experimental tools which emulate diverse master equation dynamics and give a way to test open systems theories. In this paper we demonstrate a systematic method for engineering specific system-environment interactions and emulating master equations of a particular form using classical stochastic noise in a superconducting transmon qubit. We also demonstrate that non-Markovian noise can be used as a resource to extend the coherence of a quantum system and counteract the adversarial effects of Markovian environments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Monitoring and prediction of porosity in laser powder bed fusion using physics-informed meltpool signatures and machine learning

In this work we accomplished the monitoring and prediction of porosity in laser powder bed fusion (LPBF) additive manufacturing process. This objective was realized by extracting physics-informed meltpool signatures from an in-situ dual-wavelength imaging pyrometer, and subsequently, analyzing these signatures via computationally tractable machine learning approaches. Porosity in LPBF occurs despite extensive optimization of processing conditions due to stochastic causes. Hence, it is essential to continually monitor the process with in-situ sensors for detecting and mitigating incipient pore formation. In this work a tall cuboid-shaped part (10 mm × 10 mm × 137 mm, material ATI 718Plus) was built with controlled porosity by varying laser power and scanning speed. This test caused various types of porosity, such as lack-of-fusion and keyhole formation, with varying degrees of severity in the part. The meltpool was continuously monitored using a dual-wavelength imaging pyrometer installed in the machine. Physically intuitive process signatures, such as meltpool length, temperature distribution, and ejecta (spatter) characteristics, were extracted from the meltpool images. Subsequently, relatively simple machine learning models, e.g., K-Nearest Neighbors, were trained to predict both the severity and type of porosity as a function of these physics-informed meltpool signatures. These models resulted in a prediction accuracy exceeding 95% (statistical F1-score). The same analysis was carried out with a complex, black-box deep learning convolutional neural network which directly used the meltpool images instead of physics-informed features. The convolutional neural network produced a comparable F1-score in the range of 89–97%. Finally, these results demonstrate that using pragmatic, physics-informed meltpool signatures within a simple machine learning model is as effective for flaw prediction in LPBF as using a complex and computationally demanding black-box deep learning model.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Computational Aspects of Single-Molecule Kinetics for Coupled Catalytic Cycles: A Spectral Analysis

Catalysis from single active sites is analyzed using methods developed from single molecule kinetics. Using a stochastic Markov state description, the observable properties of general catalytic networks of reactions are expressed using an eigenvalue decomposition of the transition matrix for the Markov process. By the use of a sensitivity analysis, the necessary eigenvalues and eigenvectors are related to the energies of controlling barriers and wells located along the reaction routes. A generalization of the energetic span theory allows the eigenvalues to be computed from several activation energies corresponding to distinct barrier-well pairings. The formalism is demonstrated accurately for model problems for a physically realistic mechanism for an alkene hydrogenation reaction on a single atom catalyst.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Resistive Switching in SrFeO2.5/Nb:SrTiO3 Heterostructures with Growth-Controlled Film Orientation

Resistive switching, a behavior found in many oxide materials, has the potential to enable emerging computer hardware technologies and architectures. We present resistive switching devices fabricated from epitaxial brownmillerite SrFeO2.5 films with two distinct film orientations, wherein facile oxygen ion diffusion planes are aligned parallel (in-plane) and perpendicular (out-of-plane) with the electrodes. SrFeO2.5 films were grown on (001) oriented Nb:SrTiO3 to enable high-quality interfaces and future integration with Si CMOS technologies. Post-growth vacuum annealing and growth pressure were used to control film orientations, as confirmed by transmission electron microscopy and x-ray diffraction measurements. Films grown with diffusion planes oriented in-plane had oxygen-rich, perovskite-like nanodomains spread throughout the film, and fabricated devices exhibited worse switching consistency and more stochasticity. In contrast, films grown with diffusion planes oriented out-of-plane had a more uniform oxygen-rich perovskite interfacial layer above the bottom electrode, and devices built from this film orientation showed significant statistical improvements in switching voltages and cycling consistency.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

High-frequency and Vertical Variations of Soil Moisture and their Impact on Ecosystem Dynamics

The vertical distribution of subsurface moisture is a major control on ecosystem dynamics and climate. The Mediterranean climate of California, with wet winters and dry summers, provides a testbed for understanding how transpiration is sustained through extended dry periods. The project developed and tested a new subsurface hydrology model that includes a parameterization of stochastic hydraulic conductivity to capture fracture flow, a diagnostic model of rooting profile that depends on the subsurface hydrology, and a representation of hydraulic redistribution by plant roots. A sensitivity study shows that it is plants with rooting depths >8 m that could access the subsurface moisture. The insight is applied to understand differential tree mortality during the 2012-2016 drought in California. The analysis shows that greater mortality trees on plutonic rocks than on sedimentary rocks. The study demonstrates, for the first time, the importance of lithology for the distribution and health of ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗