SEARCH · Search NASA

Results for “reward engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

42 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2022

graphenv: a Python library for reinforcement learning on graph search spaces

Many important and challenging problems in combinatorial optimization (CO) can be expressed as graph search problems, in which graph vertices represent full or partial solutions and edges represent decisions that connect them. Graph structure not only introduces strong relational inductive biases for learning (Battaglia et al., 2018) - in this context, by providing a way to explicitly model the value of transitioning (along edges) between one search state (vertex) and the next - but lends itself to problems both with and without clearly defined algebraic structure. For example, classic CO problems on graphs such as the Traveling Salesman Problem (TSP) can be expressed as either pure graph search or integer programs. Other problems, however, such as molecular optimization, do no have concise algebraic formulations and yet are readily implemented as a graph search (V. et al., 2022; Zhou et al., 2019). Such "model-free" problems constitute a large fraction of modern reinforcement learning (RL) research owing to the fact that it is often much easier to write a forward simulation that expresses all of the state transitions and rewards, than to write down the precise mathematical expression of the full optimization problem. In the case of molecular optimization, for example, one can use domain knowledge alongside existing software libraries to model the effect of adding a single bond or atom to an existing but incomplete molecule, and let the RL algorithm build a model of how good a given decision is by "experiencing" the simulated environment many times through. In contrast, a model-based mathematical formulation that fully expresses all the chemical and physical constraints is intractable. In recent years, RL has emerged as an effective paradigm for optimizing searches over graphs and led to state-of-the-art heuristics for games like Go and chess, as well as for classical CO problems such as the TSP. This combination of graph search and RL, while powerful, requires non-trivial software to execute, especially when combining advanced state representations such as Graph Neural Networks (GNN) with scalable RL algorithms.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Enterprise Artificial Intelligence Strategy for Los Alamos National Laboratory

In the 1984 martial arts drama film, The Karate Kid, a young Daniel LaRusso is unexpectedly placed in an adversarial environment unable to eYectively adapt to a series of new threats and limitations. Fortunately for the main character, once placed under the tutelage of a Mr. Miyagi, he finds resiliency not through the adoption of new tools, but a re-focused set of fundamentals. Much in the same way that Daniel learns waxing on and buYing oY car wax by hand has rewards for Karate, LANL is choosing the harder path of self-hosting Large Language Models (LLMs) for enterprise use instead of only relying on buying access to a hosted AI service like Azure’s OpenAI Application Programming Interface (API). We also are not willing to wait for software-as-a-service (SAAS) AI services to meet us where we need to be from a FedRAMP accreditation standpoint. Our operations regularly depend on access at CUI, UCNI, ITAR and other FIPS-199 moderate-impact data levels and hosting our own services gives us the right security and compliance posture to be useful across the broad range of our work at LANL. With the rise in threats to critical infrastructure, cloud service providers (CSPs), and supply chain attacks from both state and non-state actors, we are not placing the bet that SAAS hosted AI services will be available when we need them. Should a major event occur, we do not want our staY and operations left without a pathway for us to fix the problem and resume the use of AI tools.

42 ENGINEERING↗

DOE OSTI2020

Low Temperature Activation of Methane on Metal-Oxides and Complex Interfaces: Insights from Surface Science

The abundance of inexpensive, natural gas has transformed the energy landscape, whereby revealing new possibilities for sustainable chemical technologies or impacting those that have relied on traditional fossil fuels. The primary component, methane, is underutilized and wastefully exhausted, leading to anthropogenic global warming. Historically, the manipulation of methane remained “ clavis aurea ,” an insurmountable yet rewarding challenge and thus the focus of intense research. This is primarily due to an inability to dissociate C–H bonds in methane selectively, which requires a high energy penalty and is an essential prerequisite for the direct conversion of methane into a large set of value-added products. The discovery of such processes would promise an energy gainful use of natural gas benefiting several essential chemical processes associated with C1 chemistry. This first C–H bond dissociation step of the methane molecule appears in numerous catalytic mechanisms as the rate-determining step or most essential barrier sequence for all subsequent steps that follow in the production of C–C, C–O, or C x –H y –O z bonds found in value added products. A main goal is to catalytically reduce the energy barrier for the first C–H bond dissociation to be able to achieve the activation of methane at low or moderate temperatures. As such there is great value in understanding the fundamental nature of the active sites responsible for bond breaking or formation and thus be able to facilitate better control of this chemistry, leading to the development of new technologies for fuel production and chemical conversion. Surface science studies offer enhanced perspectives for a careful manipulation of bonds over the last layer atoms of catalyst surfaces, an essential factor for the design of atomically precise catalysts and unravelling of the reaction mechanism. With the advent of new surface imaging, spectroscopy, and in situ tools, it has been possible to decipher the surface chemistry of complex materials systems and further our understanding of atomic active sites on the surfaces of metals, oxides, and carbides or metal–oxide and metal–carbide interfaces. The once considered near impossible step of C–H bond activation is now observed at low temperatures with high propensity over a collection of oxide, metal–oxide, and metal–carbide systems in a conventional or inverse configuration (oxide or carbide on metal). The enabling of C–H activation at low temperature has opened interesting possibilities for the specific production of chemicals such as methanol directly from methane, a step toward facile synthesis of liquid fuels. We highlight the most recent of these results and present the key aspects of active site configurations engineered from surface science studies which enable such a simple reactive event through careful manipulation of the last surface layer of atoms found in the catalyst structure. New concepts which help in the activation and conversion of methane are discussed.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Synergistic double-atom catalysts of metal-boron anchored on g-C 2 N for electrochemical nitrogen reduction: Mechanistic insight and catalyst screening

The rational design of a novel catalytic center with a sound basis remains both challenging and rewarding for the electrochemical reduction of N 2 (eNRR), which has provided a feasible route for achieving clean and sustainable NH 3 production under ambient conditions. Herein, using density functional theory calculations, we demonstrate that hybrid metal (M)-boron (B) double-atom catalysts (DACs) embedded in g-C 2 N substrate (M-B@C 2 N, M = 3d, 4d and 5d transition metals) can achieve both high catalytic activity and high selectivity in eNRR. The proposed M-B@C 2 N DACs have exhibited impressive feasibility and stability thanks to the resilient and robust C 2 N substrate with abundant pyridinic N atoms distributed among right-sized pore structures. Our results reveal that like the metal center, the embedded B atom can actively involve in Ntriple bondN bond activation via π*-backdonation mechanism concomitant with the substantial charge transfer to adsorbed *N 2 , leading to sizable Nsingle bondN bond elongation. Accordingly, both adsorption energy and Nsingle bondN bond length of *N 2 can be employed as catalytic descriptors for predicting eNRR activity in terms of the limiting potentials (UL). Using high-throughput screening method, we found that six M-B@C 2 N candidates have stood out as the outstanding electrocatalysts for driving eNRR, namely, M = Ti (U L = 0 V), Mo (U L = 0 V), Nb (U L = -0.04 V), W (U L = -0.23 V), Zr (U L = -0.26 V), V (U L = -0.28 V). The underlying origin is attributed to the balanced and constrained N-affinity of M-B dual site working in synergy, which can thus be used as one important guide of catalyst design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa.

97 MATHEMATICS AND COMPUTING↗