SEARCH · Search NASA

Results for “Reinforcement Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Machine learning enabled discovery of superhard and ultrahard carbon polymorphs

The demand for multifunctional materials has motivated the move from near-equilibrium materials to metastable i.e. out-of-equilibrium phases that can meet several desired target properties. The search for such metastable phases with exotic properties is non-trivial and often serendipitous. Inverse design approaches based on evolutionary search have been powerful tools, but such traditional searches have focused on identifying primarily stable and metastable materials with the lowest enthalpy. The inverse design of materials, with a focus on a desired property such as, for example, hardness is a challenging task because of the expensive computational cost involved in sampling multiple structures. The recent advances in machine learning have brought new powerful AI techniques to the forefront which can potentially revolutionize the inverse design and discovery of materials, especially metastable phases capable of meeting multifunctionality. Here, in this work, we develop and apply an automated reinforcement learning workflow for inverse design that integrates first principles physics and atomistic simulations with machine learning (ML), and high-performance computing to allow rapid exploration of the superhard and ultrahard metastable phases of Carbon. We demonstrate an automatic machine learning based inverse design workflow to map new undiscovered metastable states ranging from near equilibrium to those far-from-equilibrium that satisfy multiple property objectives, specifically bulk moduli, shear moduli and hardness. We create a comprehensive library of carbon stable and metastable phases with varying hardness and subsequently shortlist 10 top performing candidate carbon structures, including two newly reported phases, based on their hardness and characterize their temperature dependent mechanical properties. A neural network model is built using featurization of allotropes of carbon to predict the quasi-harmonic Gibbs free energies. The Gibbs free energies of the top performing phases are analyzed to get an estimate of the experimental synthesizability of these superhard and ultrahard carbon phases. In general, we show using machine learning based inverse design approaches how hitherto inaccessible metastable states can be identified and potentially synthesized to meet the demand for multifunctional materials.

Balasubramanian, Karthik [Univ. of Illinois, Chica↗

DOE OSTI2024

Machine Learning a Simple Interpretable Short-Range Potential for Silica

A wide array of models, spanning from computationally expensive ab initio methods to a spectrum of force-field approaches, have been developed and employed to probe silica polymorphs and understand growth processes and atomic-level dynamical transitions in silica. However, the quest for a model capable of making accurate predictions with high computational efficiency for various silica polymorphs is still ongoing. Recent developments in short-range machine-learned models, such as GAP and NNPScan, have shown promise in providing reasonable descriptions of silica, but their computational cost remains high compared to force fields such as BKS which are based on simple interpretable functional forms. Here, in this study, we build on the recent success of our reinforcement learning (RL) workflow to derive a new set of optimal parameters for a promising short-range BKS-based model proposed by Soules. We use RL to navigate the eight-dimensional parameter space of the Soules potential using an experimental training data set that includes both local and global structural features from approximately 21 experimentally realized silica polymorphs, including high density phases and porous zeolites. We compare the performance of our machine-learned ML-Soules model with other high quality models including our recent machine-learned parametrization of BKS (ML-BKS), a machine-learned potential (GAP), as well as predictions of ab initio calculations with the highly fidelity SCAN functional. The ML-Soules accurately captures the relative energetic ordering of various polymorphs as well as their structural features at a significantly reduced computational expense. The ML-Soules model also reasonably captures the structure, density, and elastic constants of quartz, as well as metastable silica polymorphs. We further discuss the limitations of the Soules functional form and propose potential enhancements, including the incorporation of additional three-body terms and/or the utilization of different short-ranged functional forms to achieve greater accuracy for both global and local features in the modeling of silica while retaining low computational cost.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Replace Human Intelligence with Fast and Smart Geometric Reasoning and Graph Neural Network to Accelerate Next Gen ModSim Workflows

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Closing the Loop between In Situ Stress Complexity and EGS Fracture Complexity

42 ENGINEERING↗

DOE OSTI2024

Leverage modern artificial intelligence (AI) enabled systems for waste reduction

Manufacturing industries continue to face challenges in reducing waste, as upstream strategies such as source reduction and product redesign require a deeper understanding of processes compared to conventional recycling methods. Recent advancements in artificial intelligence (AI) and machine learning (ML) have opened new opportunities to integrate modern computational techniques with traditional waste minimization strategies. This paper explores AI-enabled approaches for product redesign, source reduction, and recycling that can significantly reduce waste generation while improving efficiency and sustainability. AI-driven material substitution and lightweighting in product design enable discovery of novel materials with optimized properties, reducing waste without compromising performance. Reinforcement learning models optimize process parameters, raw material specifications, and machine sequencing to minimize production losses, while Industrial Internet of Things (IIoT) systems paired with AI analytics enhance real-time waste tracking, predictive maintenance, and quality inspection. Furthermore, AI-based demand forecasting and production planning reduce overproduction and excess inventory, as demonstrated in industrial applications. In recycling, ML-powered pattern recognition and robotic sorting technologies achieve higher accuracy in waste segregation, directly improving recycling efficiency. Complementary solutions such as smart bins and AI-enabled waste pickup scheduling optimize collection logistics, reducing both costs and emissions. Although implementation requires upfront investment in infrastructure and training, the long-term benefits include higher material efficiency, reduced waste, improved product quality, and stronger sustainability outcomes across the supply chain. By leveraging AI-enabled systems, manufacturers can align waste minimization efforts with circular economy principles, creating scalable solutions for both industry and society.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Optimal Control of Differentially Private EV Charging: A Scalable Learning Approach Under Uncertainty

Internet of Things (IoT)-enabled electric vehicles (IoEVs) enable intelligent charging coordination that accounts for grid congestion. However, increased data exchange raises privacy concerns, as charging patterns can reveal sensitive driver behavior to grid operators. Here, we propose a differentially private (DP) EV charging framework that enables coordinated control while protecting driver data with theoretical privacy guarantees. Nevertheless, integrating DP inevitably introduces uncertainty into the control strategy for EVs, which can lead to infeasible solutions. To tackle this challenge, we develop a feasible and scalable control algorithm based on constrained reinforcement learning (CRL) and convex hulls. While our framework is designed to handle the uncertainty introduced by DP, it is general and also applicable to other sources of uncertainty in EV charging, such as the stochastic nature of driver behavior and renewable variability. This ensures feasible and privacy-preserving coordination of EV charging at scale. Our method constructs convex hulls within the action space to guarantee feasibility under stochastic constraints and incorporates constraint reduction techniques to improve scalability. Case studies based on IEEE benchmark systems demonstrate that the proposed approach effectively balances feasibility under uncertainty, scalability, and privacy in large-scale EV charging control.

Engineering - Power transmission and distribution↗

DOE OSTI2024

Third-integer Resonant Extraction Regulation System for Mu2e

A third-integer resonant slow extraction system is being developed for Fermilab's Delivery Ring to deliver protons to the upcoming Mu2e experiment. The timescale of the extraction (or spill) duration is 43 milliseconds, which is extremely short and unprecedented. Additionally, the experiment's strict and challenging requirements on the quality of the spill at this time scale has led to the development of a new Spill Regulation System (SRS) design. The SRS primarily consists of three components - slow regulation, fast regulation, and harmonic content suppressor. Contributions to the first two components of the SRS, i.e., Slow Regulation and Fast Regulation subsystems, will be presented in which new adaptive learning algorithm schemes for the slow regulation of the spill -- validated using particle tracking simulations -- shall be described. In addition to these novel methods for the enhancement of the spill regulation system, results of employing Machine Learning in enhancing the performance of the resonant extraction are also presented. At the forefront of applying ML techniques to solve non-linear accelerator control problems, this work includes optimizing the PID gains as well as the replacement of the traditional PID controller using Recurrent Neural Networks and Gated Recurrent Unit (GRU) ML models to achieve efficiencies greater than a PID controller. Cutting-edge on-going Reinforcement Learning efforts, including an actor-critic family of learning algorithms, to regulate the spill rate will be reviewed, as well as present analytical calculations pertaining the transit time of particles in a third-integer resonant extraction. Detailed numerical investigations and validations of such calculations, the model of which could be exported and reliably used in future analytical modeling of any resonant extraction, are discussed.

43 PARTICLE ACCELERATORS↗

DOE OSTI2026

Feedback, physics, and forecasts: The emerging paradigm of machine learning-driven battery research

Machine learning (ML) is reshaping how we understand, predict, and optimize electrochemical systems. In batteries, ML accelerates discovery across chemistry, design, and operation by transforming massive experimental and simulated datasets into predictive, interpretable models. This review consolidates a decade of progress in ML-driven battery innovation, from early-cycle feature extraction to operando image analysis and physics-informed modeling. We categorize approaches by data domain and physical fidelity, emphasizing interpretable ML for diagnostics, reinforcement learning for control, and multi-objective optimization for lifetime extension strategies. Additionally, we demonstrate how integrated models accelerate discovery, reduce testing time, and guide sustainable design. Economic analyses furthermore illustrate how these advances can lower cost per cycle and improve circularity. Together, these developments chart a path toward self-optimizing, sustainable battery technologies.

artificial intelligence↗

DOE OSTI2026

Machine learning at the Spallation Neutron Source accelerator and target

We describe the ongoing efforts to apply Machine Learning techniques to improve the performance of our accelerator and target. Specially, we are looking to minimize halo beam losses in the absence of a proper physics model, automatically detect and log anomalies in the target support systems such as cooling, and detect and prevent errant beam pulses in the linac. We also describe the infrastructure we use to acquire and stream data to the GPU cluster for training, our code development cycle, and edge computing for model inference. To minimize halo beam losses, we use a Reinforcement Learning technique tested on a virtual accelerator. The target anomaly detection is trained on archived data using incomplete physics models and is made part of the existing target reporting system. The errant beam prevention analyzes beam current and beam phase waveforms as well as accelerator configuration data to predict errant pulses. We also develop continual learning to adapt to changes in the accelerator.

Accelerator Physics↗

DOE OSTI2026

Enabling integrated AI control on DIII-D: a control system design with state-of-the-art experiments

We present the design and application of a general algorithm for Prediction And Control using MAchiNe learning (PACMAN) in DIII-D. Machine learning (ML)-based predictors and controllers have shown great promise in achieving regimes in which traditional controllers fail, such as tearing mode (TM) free scenarios, ELM-free scenarios and stable advanced tokamak conditions. The architecture presented here was deployed on DIII-D to facilitate the end-to-end implementation of advanced control experiments, from diagnostic processing to final actuation commands. This paper describes the detailed design of the algorithm and explains the motivation behind each design point. We also describe several successful ML control experiments in DIII-D using this algorithm, including a reinforcement learning controller targeting advanced non-inductive plasmas, a wide-pedestal quiescent H-mode ELM predictor, an Alfvén Eigenmode controller, a Model Predictive Control plasma profile controller and a state-machine TM predictor-controller. There is also discussion on guiding principles for real-time ML controller design and implementation.

machine learning↗

DOE OSTI2025

Operational resilience of additively manufactured parts to stealthy cyberphysical attacks using geometric and process digital twins

Cyberphysical attacks on the digital backbone of Additive Manufacturing (AM) can compromise the printed part’s functionality. They can alter features in the digital geometry to introduce geometric defects (e.g., missing fillets) or alter process parameters to create local defects (e.g., voids). Addressing the downtime, waste, and quality deterioration associated with existing solutions requires operational resilience, i.e., rapid elimination or disruption of defect formation (to retain part function) without production stoppage or part disposal (to retain yield). This need is unmet due to the inherently unpredictable nature of attack-induced alterations, lack of access to the original geometric model for identification of altered geometric features, and in-process imposition of unknown process dynamics via attack-driven alteration of real-time-uncontrolled (or exogenous) parameters. This work establishes the above-mentioned operational resilience for the first time by creating two Digital Twins (DT). The Geometric DT (Geo-DT) is based on a unique physical-field-driven soft sensor and topology optimization method. The Process Digital Twin (Pro-DT) combines local defect quantification with a novel Reinforcement Learning formulation and training method. The importance of these methodological advances and the scalability of our approach are examined on a real AM testbed. It is shown that Geo-DT can correct geometric defects without access to the original digital geometry or explicit knowledge of attack-altered geometric features. Further, Pro-DT can accelerate real-time disruption of local defects despite attack-driven imposition of unknown process dynamics. We discuss how our framework goes beyond the contemporary focus on pre-attack security and in-attack detection towards resilience for AM and beyond.

Additive Manufacturing↗

DOE OSTI2026

Cascading economic losses from port disruptions under capacity constrained multimodal freight networks

This study quantifies how throughput disruptions at major seaports cascade through capacity-constrained multimodal freight networks and interregional production systems. We couple an agent-based model (ABM) multimodal freight simulation that resolves rerouting, terminal queueing, and inventory drawdown under binding modal and facility capacities with a multiregional output loss input-output (MRIIM) model that propagates realized delivery shortfalls across regions and sectors. The framework is demonstrated for the Port of Los Angeles using Freight Analysis Framework flows and Bureau of Economic Analysis input-output accounts and is evaluated over a 52-week horizon under deterministic sector targeted shocks and stochastic disruption realizations with uncertain severity and duration. Results indicate nonlinear amplification: realized national losses concentrate in manufacturing and transportation/warehousing even when exogenous port shocks are dispersed, suggesting that congestion spillback and limited short-run substitution can dominate the initial shock allocation. We further evaluate a tabular reinforcement-learning (Q-learning) intervention layer that selects among a small set of implementable system level levers (truck-to-rail and truck-to-barge shift settings) without overriding shipper routing, finding that such interventions reduce total losses for moderate disruptions but yield diminishing returns once substitute modes approach capacity. By linking operational freight behavior to system wide impacts under uncertainty, the proposed ABM-MRIIM pipeline provides a reusable workflow for port disruption stress testing, identification of structurally critical sectors/corridors, and evaluation of resilience interventions under realistic capacity limits.

42 ENGINEERING↗

DOE OSTI2025

Impacts and emerging research opportunities in Vehicle-Grid Integration for transportation: A review

This review provides a comprehensive examination of Vehicle-Grid Integration (VGI) technologies and their impacts on transportation systems, with a particular emphasis on the transportation-energy nexus. It systematically explores how VGI affects key transportation applications such as charging infrastructure planning, electric vehicle (EV) routing, smart charging coordination, shared mobility, and dynamic pricing. By synthesizing recent literature from both transportation and energy systems perspectives, this study highlights how advanced methodologies, such as reinforcement learning, game theory, and optimization techniques, are used to model the complex interactions between EVs, mobility patterns, and distributed energy systems. Furthermore, the review also identifies critical challenges, including behavioral factors, data limitations, and system scalability. Drawing on these insights, the paper outlines emerging research opportunities to support the design of integrated, resilient, and user-centric VGI solutions that advance sustainable mobility and energy system efficiency.

Charging coordination↗

DOE OSTI2026

SmileyLlama: modifying large language models for directed chemical space exploration

Here we show that large language models (LLMs) can be transformed via supervised fine-tuning of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pretrained LLMs and chemical language models trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization to both improve SmileyLlama’s adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized three-dimensional conformations and high binding affinity to drug targets. By training an LLM to speak directly as a chemical language model, while retaining most of its natural language capabilities, we show that SmileyLlama can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the supervised fine-tuning/direct preference optimization/LLM framework can be extended to other chemical, biological and materials applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Generative AI for design of nanoporous materials: review and future prospects

Generative artificial intelligence (AI) is emerging as a powerful tool for advancing the design of nanoporous materials such as metal–organic frameworks, covalent–organic frameworks, and zeolites. These materials have potential application in important areas such as carbon capture, catalysis, gas storage, chemical separation, and drug delivery due to their modular, tunable structures, and their performance in these areas depends on precise control over their structure, chemical functionalities, and properties. Herein, we provide a review of generative AI algorithms that are emerging as powerful tools for the design of nanoporous materials, namely generative adversarial networks, variational autoencoders, diffusion models, genetic algorithms, reinforcement learning, and large language models. Some models are particularly good at generating diverse and high-quality designs, while others excel at exploring large design spaces or optimizing materials with desired properties. Certain algorithms also allow for efficient transitions between different designs, and some offer versatility in generating materials based on textual input. We discuss the advantages, limitations, and applications of these algorithms in porous material design and emphasize the future potential of integrating AI with experimental workflows to accelerate the development and validation of AI-generated materials.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Derivative-free stochastic optimization via adaptive sampling strategies

In this paper, we present a novel derivative-free framework for solving unconstrained stochastic optimization problems. Many problems in fields ranging from simulation optimization to reinforcement learning to quantum computing involve settings where only stochastic function values are obtained via a zeroth-order oracle, which has no available gradient information and necessitates the usage of derivative-free optimization methodologies. Our approach includes estimating gradients using stochastic function evaluations and integrating adaptive sampling techniques to control the accuracy in these stochastic approximations. Our framework encapsulates several gradient estimation techniques, including standard finite-difference, Gaussian smoothing, sphere smoothing, randomized coordinate finite-difference, and randomized subspace finite-difference methods. We provide theoretical convergence guarantees for our framework and analyze the worst-case iteration and sample complexities associated with each gradient estimation method. Finally, we demonstrate the empirical performance of the methods on logistic regression and nonlinear least squares problems.

Adaptive sampling↗

DOE OSTI2025

CityLearn v2: energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities

As more distributed energy resources become part of the demand-side infrastructure, quantifying their energy flexibility on a community scale is crucial. CityLearn v1 provided an environment for benchmarking control algorithms. However, there is no standardized environment utilizing realistic building-stock datasets for distributed energy resource control benchmarking without co-simulation or third-party frameworks. CityLearn v2 extends CityLearn v1 by providing a stand-alone simulation environment that leverages the End-Use Load Profiles for the U.S. Building Stock dataset to create grid-interactive communities for resilient, multi-agent, and objective control of distributed energy resources with dynamic occupant feedback. While the v1 environment used pre-simulated building thermal loads, the v2 environment uses data-driven thermal dynamics and eliminates the need for co-simulation with building energy performance software. This work details the v2 environment and provides application examples that use reinforcement learning control to manage battery energy storage system, vehicle-to-grid control, and thermal comfort during heat pump power modulation.

Nweye, Kingsley↗

DOE OSTI2024

Hamiltonian switching control of noisy bipartite qubit systems

Abstract We develop a Hamiltonian switching ansatz for bipartite control that is inspired by the quantum approximate optimization algorithm, to mitigate environmental noise on qubits. We demonstrate the control for a central spin coupled to bath spins via isotropic Heisenberg interactions, and then make physical applications to the protection of quantum gates performed on superconducting transmon qubits coupling to environmental two-level-systems (TLSs) through dipole-dipole interactions, as well as on such qubits coupled to both TLSs and a Lindblad bath. The control field is classical and acts only on the system qubits. We use reinforcement learning with policy gradient to optimize the Hamiltonian switching control protocols, using a fidelity objective for specific target quantum gates. We use this approach to demonstrate effective suppression of both coherent and dissipative noise, with numerical studies achieving target gate implementations with fidelities over 0.9999 (four nines) in the majority of our test cases and showing improvement beyond this to values of 0.999 999 999 (nine nines) upon a subsequent optimization by GRadient Ascent Pulse Engineering (GRAPE). We analyze how the control depth, total evolution time, number of environmental TLS, and choice of optimization method affect the fidelity achieved by the optimal protocols and reveal some critical behaviors of bipartite control of quantum gates.

Physics↗