Search NASA⌕ Search

SEARCH · Search NASA

Results for “Reinforcement learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Machine Learning an Ab-Initio Based Bond-Order Potential for Bismuthene

Bismuthene is a heavy 2D material whose strong spin–orbit coupling and recently observed single-element ferroelectricity have intensified interest in its structural, vibrational, and transport properties. Accurate modeling of these behaviors requires a short-range interatomic potential that can reproduce the underlying bonding physics at a fraction of the computational cost of first-principles methods. However, such a potential is currently unavailable. Here, in this work, we construct a Tersoff bond-order potential for β-bismuthene using a reinforcement-learning framework that integrates a continuous Monte Carlo Tree Search with a simplex-based local optimizer. The optimized parameter sets reproduce first-principles lattice constants, cohesive energy, the equation of state, elastic constants, and phonon dispersion. We validate the models by performing thermal-conductivity calculations and uniaxial fracture simulations our findings confirm the reliability of the resulting models across multiple thermomechanical regimes. Comparison of the three best solutions reveals how differences in pairwise interactions, angular terms, and bond-order behavior govern phonon features and mechanical responses. We demonstrate an interpretable and computationally efficient potential for bismuthene and demonstrate a general reinforcement-learning strategy for developing bond-order models in emerging 2D materials.

deformation↗
DOE OSTI2026

Towards AI-assisted neutrino flavor theory design

Particle physics theories, such as those which explain neutrino flavor mixing, arise from a vast landscape of model-building possibilities. A model’s construction typically relies on the intuition of theorists. It also requires considerable effort to identify appropriate symmetry groups, assign field representations, and extract predictions for comparison with experimental data. We develop Autonomous Model Builder (AMBer), a framework in which a reinforcement learning agent interacts with a streamlined physics software pipeline to search these spaces efficiently. AMBer selects symmetry groups, particle content, and group representation assignments to construct models while minimizing the number of free parameters introduced. We validate our approach in well-studied regions of theory space and extend the exploration to a previously unexamined symmetry group. While demonstrated in the context of neutrino flavor theories, this approach of reinforcement learning with physics software feedback may be extended to other theoretical model-building problems in the future.

Baretz, Jason Benjamin↗
NASA NTRS2011

Modeling Humans as Reinforcement Learners: How to Predict Human Behavior in Multi-Stage Games

This paper introduces a novel framework for modeling interacting humans in a multi-stage game environment by combining concepts from game theory and reinforcement learning. The proposed model has the following desirable characteristics: (1) Bounded rational players, (2) strategic (i.e., players account for one anothers reward functions), and (3) is computationally feasible even on moderately large real-world systems. To do this we extend level-K reasoning to policy space to, for the first time, be able to handle multiple time steps. This allows us to decompose the problem into a series of smaller ones where we can apply standard reinforcement learning algorithms. We investigate these ideas in a cyber-battle scenario over a smart power grid and discuss the relationship between the behavior predicted by our model and what one might expect of real human defenders and attackers.

Lee, Ritchie↗
DOE OSTI2023

Quantum logic gate synthesis as a Markov decision process

Reinforcement learning has witnessed recent applications to a variety of tasks in quantum programming. The underlying assumption is that those tasks could be modeled as Markov decision processes (MDPs). Here, we investigate the feasibility of this assumption by exploring its consequences for single-qubit quantum state preparation and gate compilation. By forming discrete MDPs, we solve for the optimal policy exactly through policy iteration. We find optimal paths that correspond to the shortest possible sequence of gates to prepare a state or compile a gate, up to some target accuracy. Our method works in both the absence and presence of noise and compares favorably to other quantum compilation methods, such as the Ross–Selinger algorithm. This work provides theoretical insight into why reinforcement learning may be successfully used to find optimally short gate sequences in quantum programming.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Surrogate model for third-integer resonance extraction at the Fermilab Delivery Ring

We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment’s production target, whereby the goal is to extract a uniform slice of the circulating 1e12 protons in the DR over 25,000 turns (43 ms). The DR contains 3 harmonic sextupoles that excite a third-integer resonance and three fast, tune-ramping quadrupole magnets that drive the horizontal tune towards the 29/3 resonance. In our initial work, the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the “environment” in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The controller will be hosted on a dedicated Arria 10 FPGA. In this work, we report the accuracy and fidelity of the surrogate model in comparison to the response dynamics of the physics simulator.

Narayanan, Aakaash [Fermilab]↗
DOE OSTI2025

Optimizing Traffic Signal Control to Enhance Transportation Efficiency and Maximize Pedestrian Benefits in the Road Network

Increasing urban mobility requirements demand efficient transportation system strategies for both vehicular and pedestrian movement. This study enhances the Decentralized Graph-based Multi-Agent Reinforcement Learning (DGMARL) approach, originally tailored for vehicular traffic signal timing, to incorporate pedestrian traffic dynamics. The improved algorithm considers crucial metrics such as Eco_PI, assesses vehicle fuel consumption by factoring in stops and delays, and addresses pedestrian waiting time, crucial for system efficiency while acknowledging driver waiting time impact. Utilizing Digital Twin simulation along the MLK Smart Corridor in Chattanooga, Tennessee, the algorithm's performance is compared for various pedestrian control scenarios. To evaluate the effectiveness of DGMARL, this study compared DGMARL-enabled signal management with automated pedestrian traffic detection and an actuated signal management system (real-word baseline) with pedestrian recall, which predetermingly enforces a pedestrian phase every cycle. Findings indicate substantial improvements with DGMARL, showing a 28.29% enhancement in vehicle Eco_PI, a 60.55 % reduction in pedestrian waiting time, and a 55.74% decrease in driver stop delay, on average, compared to the baseline actuated signal timing plan.

Kumarasamy, Vijayalakshmi K [The University of Ten↗
DOE OSTI2022

AI-Enabled Discovery and Physics-Based Optimization of Energy Efficient Processing Strategies for Advanced Turbine Alloys (Final Technical Report)

In this project, the multi-organizational team of academic and industrial researchers from the University of Kentucky an aerospace and energy generation OEM partner has leveraged novel Digital Process Twin (DPT) models of process/structure interactions (i.e., process-induced surface integrity) to advance a paradigm of fully integrated computational materials engineering (ICME). Using efficient process models as the core of a digital process simulator for a reinforcement learning algorithm, the team has integrated industrial data and metrics of structure/performance/energy relationships and manufacturing-related energy metrics to optimize dynamic processing parameters for significantly improved life-cycle energy efficiency of advanced γ-TiAl low-pressure turbine (LPT) alloys, as indicated by a set of design relevant parameters (e.g., residual stresses and scrap rate). The key objective and anticipated outcome of the project was at least a 10% reduction in life-cycle embodied energy for a recently developed, γ-TiAl low-pressure turbine (LPT) alloy and nickel-based superalloy Inconel 718, through the adoption of the proposed AI-enabled process optimization approach. The final project outcomes significantly exceeded this original target, realizing manufacturing-related energy efficiency improvements of more than 130% for TiAl and up to 80% for Inconel 718. Rather than following the prevailing and highly inefficient empirical paradigm, the proposed study demonstrated the feasibility of adopting a digital, physics-based process design and optimization paradigm. The recurring need for manual intervention, rework, reinspection causes significant production bottlenecks and unnecessary expense associated with delivering the requisite component quality. The OEM partner, and turbine industry in general, expect to reap significant cost and resource savings if an AI-optimized set of parameters can be applied to specific machining operations. The technical scope of the proposed project involved the paving of a realistic path towards model-based and AI-enabled Integrated Computational Materials Engineering (ICME), and away from inefficient empirical process optimization and legacy manufacturing practices, which are no longer able to efficiently process novel high-performance turbine alloy materials. The project team will address the fundamental knowledge gap that currently exists within the ICME paradigm with respect to the process/structure/performance/energy impacts of finishing processes. While significant resources have been devoted to the ‘early stages’ of manufacturing, such as alloy design, primary and secondary processing, finishing processes have not been adequately integrated within ICME. To provide an actionable path towards model-based finishing process design (e.g., machining, burnishing, grinding, polishing), we will employ a novel AI-enabled process optimization paradigm, based on a computationally efficient, physics-based process simulator. Through limited experimental work to calibrate and validate our process simulator model via an advanced in-situ characterization technique and process optimization via reinforcement learning, the project will seek to demonstrate a viable alternative to the inefficient ‘legacy’ processing strategies, empirical testing and broad scope machining learning approaches, all of which fail to adequately consider complex process physics. The project team has identified an intermetallic γ-TiAl LPT alloy, which is currently being used as part of the OEM partner’s advanced gas turbine designs. This particular alloy poses significant manufacturing challenges during finishing operations, which limit the degree to which the current turbine design can be manufactured in an energy- and cost-efficient manner. Empirical testing and numerical modeling efforts to optimize processing parameters for γ-TiAl have not been able to resolve these manufacturing challenges, so the proposed physics-based AI-enabled optimization technology would offer a truly novel and transformative capability. The multi-organizational team of academic and industry experts from the UKY and the OEM partner will work together closely to demonstrate the analytical and experimental critical function and characteristic proof of concept of this novel approach.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2022

A high-fidelity building performance simulation test bed for the development and evaluation of advanced controls

We present an open-source building performance simulation test bed, the Advanced Controls Test Bed (ACTB), that interfaces high-fidelity Spawn of EnergyPlus building models, with advanced controllers implemented in Python. Additionally, the ACTB leverages the Building Optimization Testing and Alfalfa platforms for managing simulations, providing an external clock, a representational state transfer (REST) application programming interface (API), and key performance indicators for evaluating the effectiveness of control strategies. The REST API allows the development of external controllers programmed in languages such as Python, which provides flexibility and a rich choice of scientific libraries for designing control sequences. We present three test cases based on the U.S. Department of Energy's Reference Small Office Building to demonstrate the ACTB's capabilities: (a) rule-based controls compliant with ASHRAE Guideline 36 control sequences; (b) an economic model predictive control implemented using do-mpc; and (c) a deep Q-network reinforcement learning agent implemented using OpenAI Gym.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Physics-Reinforced Machine Learning Algorithms for Multiscale Closure Model Discovery

The central objective of this project was to address the challenge of modeling and simulating complex multiscale turbulence phenomena by leveraging physics-guided machine learning (PGML) and hybrid modeling approaches. By integrating physics-based methods with data-driven models, the research focused on achieving robust and scalable solutions for geophysical turbulence, enhancing numerical weather prediction and climate research tools. The project resulted in significant advancements in computational modeling paradigms, predictive tools for reduced-order modeling, and innovative algorithms for fluid dynamics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Reinforcement Learning-based Output Structured Feedback for Distributed Multi-Area Power System Frequency Control

Load frequency control (LFC) is a key factor to maintain the stable frequency in multi-area power systems. As the modern power systems evolve from centralized to decentralized paradigm, LFC needs to consider the decentralized scheme that considers limited information from the information-exchange graph for the generator control of each interconnected area. This paper aims to solve a data-driven constrained LQR problem with mean-variance risk constraints and output structured feedback, and applies this framework to solve the LFC problem in multi-area power systems. By reformulating the constrained optimization problem into a minimax problem, the stochastic gradient descent max-oracle (SGDmax) algorithm with zero-order policy gradient (ZOPG) is adopted to find the optimal feedback gain from the learning, while guaranteeing the convergence. In addition, to improve the adaptation of the proposed learning method to new or varying models, we construct an emulator grid that approximates the dynamics of a physical grid and performs training based on this model. Once the feedback gain is obtained from the emulator grid, it is applied to the physical grid with a robustness test to check whether the controller from the approximated emulator applies to the actual system. Numerical tests show that the obtained feedback controller can successfully control the frequency of each area, while mitigating the uncertainty from the loads, with reliable robustness that ensures the adaptability of the obtained feedback gain to the actual physical grid.

Kwon, Kyung-bin↗
NASA NTRS2000

Adaptivity in Agent-Based Routing for Data Networks

Adaptivity, both of the individual agents and of the interaction structure among the agents, seems indispensable for scaling up multi-agent systems (MAS s) in noisy environments. One important consideration in designing adaptive agents is choosing their action spaces to be as amenable as possible to machine learning techniques, especially to reinforcement learning (RL) techniques. One important way to have the interaction structure connecting agents itself be adaptive is to have the intentions and/or actions of the agents be in the input spaces of the other agents, much as in Stackelberg games. We consider both kinds of adaptivity in the design of a MAS to control network packet routing. We demonstrate on the OPNET event-driven network simulator the perhaps surprising fact that simply changing the action space of the agents to be better suited to RL can result in very large improvements in their potential performance: at their best settings, our learning-amenable router agents achieve throughputs up to three and one half times better than that of the standard Bellman-Ford routing algorithm, even when the Bellman-Ford protocol traffic is maintained. We then demonstrate that much of that potential improvement can be realized by having the agents learn their settings when the agent interaction structure is itself adaptive.

Wolpert, David H.↗
DOE OSTI2023

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗