SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ProteinTuneRL is a framework designed to harness the power of reinforcement learning for advanced protein design. The project enables fine-tuning of generative models to explore and optimize protein sequences with tailored structural and functional properties.
The Workflow Orchestration Language Framework (WOLF) is an agentic framework grounded in natural language with an architecture inspired by reinforcement learning (RL)—designed to orchestrate, scale, and accelerate complex workflows. The concept of WOLF was born out of the very successful ASC Tri-lab Multi-Agent Design Assistant (MADA) project, but extends beyond its domain-specific design agents to provide a more general and extensible architecture. WOLF capitalizes on the lessons learned from MADA and is fully aligned with Sutton’s The Bitter Lesson—that the most enduring progress in AI comes from general-purpose methods that scale with computation, rather than narrow techniques built on domain-specific human knowledge. In this spirit, WOLF enables agents to autonomously learn workflows, capture strategies as reusable playbooks, and build a growing corpus of interpretable, auditable “wisdom artifacts.” These artifacts, expressed in natural language, bridge human and machine understanding while preserving adaptability and scalability as computational power continues to expand.
In computer science, Document Summarization is the task of condensing some quantity of text and related content through automated means. In this document, we review recent literature in text summarization. “Hybrid” extractive-abstractive approaches continue to be explored. Some of the latest efforts have also sought to enable users to adjust summaries with queries or other structure and begun to test reinforcement-learning style agentic LLM-based solutions.
Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.
Rare-earth elements (REEs) are essential for electronics, renewable energy, and defense technologies. However, the current supply of REEs relies on mining concentrated in a few countries and energy-intensive separations. DOE’s Basic Energy Sciences (BES) program has launched a grand challenge which aims to ensure a sustainable supply of critical REEs by developing innovative and environmentally friendly separation methods. As an alternative to costly and harmful traditional methods, the Non-Equilibrium Transport Driven Separations (NETS) initiative has created a microfluidic Y-channel co-flow method that applies external fields to exploit magneto- and electrohydrodynamic effects for separating dilute REE ions from complex feedstocks. Computational fluid dynamics (CFD) studies have identified a few operating conditions with promising ion selectivity and separation efficiency. However, challenges remain regarding Y-channel versatility across feedstocks and accurate incorporation of physical phenomena into CFD models. In this work, we develop a multi-fidelity modelling approach which integrates experimental results with CFD simulation to build a surrogate model for the dependence of separation efficiency to variation of design parameters. The surrogate model enables a reinforcement learning (RL) method to adaptively launch CFD and experimental runs, improving model fidelity around optimal Y-channel parameters.
The PIP-II linac will enable >1.2 MW beam power for DUNE, requiring unprecedented operational reliability across its warm front-end (RFQ, MEBT) and five distinct SRF sections operating at 162.5/325/650 MHz. We present a comprehensive digital twin framework uniquely combining a fully differentiable fast beam transport code with neural network surrogates trained on high-fidelity PIC simulations, capturing space charge and nonlinear dynamics beyond traditional envelope codes while achieving 10⁴ speedup at <1% accuracy. End-to-end differentiability enables gradient-based optimization across 500+ parameters simultaneously previously impossible with conventional tools while the model incorporates static/dynamic errors and serves as a virtual commissioning platform for diverse hardware integration. The framework facilitates reinforcement learning for pulsed/CW mode transitions, predictive maintenance through anomaly detection, and autonomous tuning algorithm development with real-time execution capability. Validation against physics simulations shows excellent agreement for the front-end, with initial results demonstrating potential for 30% commissioning time reduction and proactive fault mitigation, providing a scalable blueprint for operating next-generation high-intensity accelerators.
We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This effort is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment s production target, wherein the goal is to extract a uniform slice of the circulating $1e12$ protons in the DR over 25,000 turns (43~ms). The DR contains 3 harmonic sextupoles which excite a third-integer resonance as well as three fast, tune-ramping quadrupole magnets which drive the horizontal tune towards the $29/3$ resonance. In our initial work the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the environment in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The control agent will be hosted on a dedicated Arria 10 FPGA, introducing its own requirements on control agent architecture. In this work we report the accuracy and fidelity of surrogate models in comparison to the response dynamics of the physics simulator.
This project represents a continuation of the collaboration between iProTech and NLR to simulate, optimize and design the iProTech Pitching Inertial Pump (PIP) device. The objectives of this TEAMER project are twofold: 1. Refining the physical characteristics of the existing iProTech PIP WEC-Sim model to enhance the model’s fidelity and include controllable components. Key model enhancements target the inclusion of Coulomb friction, the introduction of a controllable bypass valve, and the replacement of traditional check valves with advanced motorized ones. 2. Exploring traditional and advanced control algorithms. From traditional methods like latching control to cutting-edge reinforcement learning (RL) algorithms, the goal is to ensure the PIP device's adaptability and optimal performance across a range of ocean conditions. NLR is tasked with augmenting the WEC-Sim model and implementing the control algorithms, culminating in performance comparison analyses. iProTech will update their existing 3D models, advise on model improvements, and determine crucial system metrics. WEC-Sim, developed in MATLAB/SIMULINK with Simscape Multibody, is the main piece of software that will be used in this project. Coupled with the MATLAB RL Toolbox, it offers a robust platform for in-depth simulation and optimization of the iProTech PIP device. Building on previous work to explore the PIP design space and optimize its geometry, mass distribution, center of gravity and other key parameters, this project aims to refine iProTech’s existing numerical models and develop effective control algorithms that can seamlessly integrate into their future hardware testing campaigns.
Advanced nuclear reactors are a key part of the future of nuclear energy both in the United States and globally. They offer unique benefits for various energy-demanding applications, including use in remote locations, compact size, modular manufacturing, remote monitoring, low and/or variable power rating operation, and reliance on novel technologies to enhance operational safety. To achieve economic feasibility, advanced reactors must significantly reduce their workforces in comparison with the current fleet. Achieving this reduction will occur through reducing staff workloads using technology to achieve autonomous or semi-autonomous operations, demonstrated by comprehensive testing and validation activities. These operations will require both software and hardware platforms during the design and testing phases. While simulations are useful during the design phase, their performance can significantly deviate during actual deployment on hardware. This report presents the outcomes of a collaborative technical initiative between the U.S. Department of Energy (DOE) Microreactor Program (MRP) and Advanced Sensors and Instrumentation (ASI) Program. The collaboration utilized the Microreactor Automated Control System (MACS) hardware platform to bridge the gap between theoretical reactor design and actual startup and control operations. Two key use cases were investigated: facilitating the startup testing period and demonstrating supervisory control. The first use case details the key Microreactor Applications Research Validation and Evaluation (MARVEL) reactor startup physics testing activities conducted using the MACS platform. These activities included drum worth measurements, shutdown margin assessment, temperature feedback analysis, and scram time evaluation, as well as unique testing that would apply to the MARVEL reactor to demonstrate the testing methodologies in a low-risk environment. The MACS platform, serving as a surrogate representation of the MARVEL reactor, proved instrumental in performing these tests. The exercise revealed aspects that led to optimized processes, refined hardware design, and enhanced base software capabilities. By maturing methods and technologies in this manner, the initiative promises to reduce wasted time in the actual on-site reactor deployment effort, thereby saving significant time and resources. The second use case focuses on the development and implementation of supervisory control methods aimed at managing core tilt, which can result from asymmetrical operations or manufacturing imperfections in fuel rods or reactivity control devices. A key objective was to assess and compare the use of artificial intelligence (AI) for supervisory control. The effort aimed to define the role of supervisory control to enhance performance without risking control instability. This effort explored three distinct approaches: rules-based (RB) methods, optimization techniques, and reinforcement learning (RL) algorithms. Each approach was evaluated for its ease of implementation, its usability, and its effectiveness in responding to asymmetries in neutron flux. Comparative analysis of these approaches provided valuable insights into their applicability and effectiveness, offering a robust framework for advanced reactor operations. Together, these two use cases highlight the potential of hardware test beds to help streamline the design, operation, and control of advanced nuclear reactors. This collaborative effort underscores the importance of continued innovation and experimentation in achieving the next generation of safe, reliable, and economically viable nuclear energy solutions.
PIPII project will raise Fermilab Booster intensity and ramp rate. Beam losses will limit average power and are hard to simulate. Presently, Booster uses operator-guided empirical tuning. This task is challenging due to high dimensionality, multiple objectives, critical safety constraints, and drifts. We developed a synergistic suite of Bayesian optimization (BO) and reinforcement learning (RL) tools to optimize and stabilize beam losses. First, active learning was used to build a rough model. Data was collected parasitically using two novel safety constraint types – nonlinear input space restrictions (based on optics model), and uncertainty constraints (to stop bad steps/beam aborts). We then applied online multi-objective BO with scalarized objectives and fitting to improve/rebalance losses, increasing safety margins by 25%. Using BO model as a safety veto, we tried several on/off-policy RL agents for long term stabilization; SAC had best performance. We found that adding contextual (state) information further improved performance, eventually integrating key knobs like linac phase and temperature into the parameter space. Long term testing is ongoing to enable operational use.
The accumulation of municipal solid waste (MSW) continues to rise due to burgeoning population, rapid global urbanization and economic growth, intensifying ecological concerns associated with landfills and greenhouse gas (GHG) emissions. Over the past 2 decades, global waste generation has surged by 50%, with one-third remaining uncollected and about 70% sent to landfills. This review examines the critical role of integrating emerging technologies, such as advanced sensors and artificial intelligence (AI), into end-to-end MSW management to alleviate landfill burdens. The suitability of various AI tools for different stages of MSW management is assessed, alongside the deployment of advanced sensors including hyperspectral cameras, computer vision systems, and internet of things (IoT) devices for material identification. Applications of genetic algorithms and reinforcement learning for optimizing collection routes, reducing costs, and lowering emissions are highlighted. Life cycle assessment (LCA) across all stages of MSW management is also reviewed, along with future trends in leveraging generative AI, natural language processing (NLP), and agent-based AI systems to analyze waste generation patterns and public sentiment. Efficient collection and handling can be enhanced through route optimization with geographic information systems and real-time bin-level monitoring. Furthermore, sensor-embedded, real-time object detection systems paired with robotics enable material characterization and automated sorting, thereby lowering costs and diverting waste from landfills into value-added products for diverse industrial sectors including packaging, chemicals, textiles, metals and glass, transportation, and electronics industries. Without intervention, global waste is projected to reach 4.54 billion tons by 2050, contributing direct economic costs of $\$$400 billion and roughly 2.38 billion tons of CO 2 -equivalent emissions annually. This review demonstrates how AI-driven, end-to-end solutions for MSW management can mitigate economic and environmental challenges, while directly supporting the United Nations Sustainable Development (UNDP) goals related to innovation and infrastructure (SDG 9), sustainable cities (SDG 11), responsible consumption and production (SDG 12), and climate action (SDG 13).
This paper investigates the challenges and solutions associated with integrating a hydrogen-generating nuclear-renewable integrated energy system (NR-IES) under a transactive energy framework. The proposed system directs excess nuclear power to hydrogen production during periods of low grid demand while utilizing renewables to maintain grid stability. Using digital real-time simulation (DRTS) in the Typhoon HIL 404 model, the dynamic interactions between nuclear power plants, electrolyzers, and power grids are analyzed to mitigate issues such as harmonic distortion, power quality degradation, and low power factor caused by large non-linear loads. A three-phase power conversion system is modeled using the Typhoon HIL 404 model and includes a generator, a variable load, an electrolyzer, and power filters. Active harmonic filters (AHFs) and hybrid active power filters (HAPFs) are implemented to address harmonic mitigation and reactive power compensation. The results reveal that the HAPF topology effectively balances cost efficiency and performance and significantly reduces active filter current requirements compared to AHF-only systems. During maximum electrolyzer operation at 4 MW, the grid frequency dropped below 59.3 Hz without filtering; however, the implementation of power filters successfully restored the frequency to 59.9 Hz, demonstrating its effectiveness in maintaining grid stability. Future work will focus on integrating a deep reinforcement learning (DRL) framework with real-time simulation and optimizing real-time power dispatch, thus enabling a scalable, efficient NR-IES for sustainable energy markets.
This paper presents a reinforcement learning (RL)-driven strategy to improve the transient stability of power systems via tuning parameters of multiple virtual synchronous generators (VSGs). We proposed a scalable method to support RL training convergence probability and speed, even when a large number of contingencies are considered. The proposed scalable RL framework first decomposes the large number of contingencies into multiple groups and then conducts parallel training for each group, decreasing the state space and complexity of each training. Additionally, we propose a contingency grouping algorithm to streamline the RL action space and facilitate the training. The proposed method is validated across various standard test systems.
The PIP-II linac will enable >1.2 MW beam power for DUNE, requiring unprecedented operational reliability across its warm front-end (RFQ, MEBT) and five distinct SRF sections operating at 162.5/325/650 MHz. We present a comprehensive digital twin framework uniquely combining a fully differentiable fast beam transport code with neural network surrogates trained on high-fidelity PIC simulations, capturing space charge and nonlinear dynamics beyond traditional envelope codes while achieving 10⁴× speedup at <1% accuracy. End-to-end differentiability enables gradient-based optimization across 500+ parameters simultaneously—previously impossible with conventional tools—while the model incorporates static/dynamic errors and serves as a virtual commissioning platform for diverse hardware integration. The framework facilitates reinforcement learning for pulsed/CW mode transitions, predictive maintenance through anomaly detection, and autonomous tuning algorithm development with real-time execution capability. Validation against physics simulations shows excellent agreement for the front-end, with initial results demonstrating potential for 30% commissioning time reduction and proactive fault mitigation, providing a scalable blueprint for operating next-generation high-intensity accelerators.
The nuclear industry is experiencing renewed interest in autonomous robotics, yet most deployed systems remain teleoperated with limited autonomy. This work presents two contributions toward fully autonomous nuclear robotic systems: autonomous waste inspection at the Hanford Site and an autonomous hot cell laboratory framework. Inspections of Hanford's underground waste storage tanks are performed manually at significant cost and personnel exposure. We developed a reinforcement-learning (RL) training pipeline for a custom-built inspection arm. In parallel, we are designing an autonomous laboratory framework for post-irradiation examination in hot cells at the Specimen Preparation Laboratory (SPL) that integrates computer vision, task and motion planning, hardware execution, and operator-in-the-loop control. These systems demonstrate a path toward safer, more efficient nuclear operations by reducing human exposure while maintaining rigorous human oversight at critical decision points.
Vision-Language-Action (VLA) models enable test-time behavioral steering via neuron-level interventions, but existing methods use fixed strengths and operate in open loop. This static modulation fails under evolving task dynamics, leading to overcorrection, oscillations, and reduced task success—especially for temporal attributes like speed. We propose CTRL-STEER, a control-theoretic framework that casts activation steering as closed-loop feedback with adaptive, time-varying interventions. Instead of assuming neurons encode temporal concepts, we steer along motion-aligned residual directions and regulate intervention magnitude via feedback. We instantiate this with both PID and reinforcement learning controllers that jointly optimize concept adherence and task success. Experiments on fine-tuned OpenVLA policies across four LIBERO suites show improved stability and a better steering–success trade-off over fixed-coefficient baselines, without retraining the base model.
Not Available
With ubiquitous digitization, sensing, and computational intelligence deployed in increasingly more and broader domains, including critical infrastructure, potentially misleading and destabilizing effects of multimodal anomalies and adversarial behavior are growing in importance. Here, we develop randomized and reinforcement learning-based strategies for strategically recruiting and utilizing deployed (and, thus, vulnerable and potentially faulty and/or compromised) nodes from information and inference networks, while defending against adversaries that attempt to misguide assessments of inferred variables. Recognizing that, besides communication and other costs, sampling from any observable node can either provide true data or dangerously expose our inference to misinformation (without being easily distinguishable what actually happens), the proposed strategies proceed by progressively recruiting nodes and cautiously scaling their information contribution based on assumed, or, in our reinforcement learning approach, intelligently weighed trustworthiness, with the learning approach also considering network-wide, threat-inclusive risk/value tradeoffs. While avoiding the hardware, communication, analytical and computational burden of explicit redundancy, the proposed defensive schemes enable on-the-fly assessments of underlying processes, and system-wide situational awareness with demonstrable resilience against adversarial activities.