Search NASA⌕ Search

SEARCH · Search NASA

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

Deep reinforcement learning to discover multi-fuel injection strategies for compression ignition engines

Over the past several decades, regulation of compression ignition engine emissions has become increasingly stringent as concern about the environmental and health implications of these emissions has grown. These changing constraints have led to a series of new, alternative fuel injection strategies that aim to maintain power output while reducing in-cylinder generated emissions by operating in the low-temperature combustion (LTC) regime. These advanced injection strategies are created and retuned for individual combinations of engine geometry, fuel, and emissions constraints. Deep reinforcement learning has been shown to be an effective alternative to traditional optimization approaches for highly combinatorial control problems, such as discovering the optimal injection schedules for compression ignition engines. In this study, we deploy a previously presented deep reinforcement learning framework to iteratively optimize a series of engine geometries over a range of increasingly strict NO x emissions regulations. We then examine the resulting injection schedules. We discuss the potential for using this deep reinforcement learning framework for fuel selection screening and for discovering unique injection strategies for different engine geometries and future emissions standards.

33 ADVANCED PROPULSION SYSTEMS↗

High-Temperature Aquifer Thermal Energy Storage (HT-ATES) Projects in Germany and the Netherlands—Review and Lessons Learned

Aquifer thermal energy storage (ATES) is a concept that can help to address heating and cooling needs through the use of the subsurface as a seasonal thermal energy storage (STES) system. Over 2800 ATES systems have been deployed with storage temperatures typically below 25 °C and only a few with higher temperatures (>40 °C), which would increase the energy density and utility of the stored thermal fluids. Until now, only a few high-temperature aquifer thermal energy storage (HT-ATES) projects have been initiated and are still in operation. These HT-ATES projects have encountered a range of technical and non-technical challenges. This study reviews ten such projects: four in Germany and six in the Netherlands. The non-technical issues include public acceptance, a lack of regulatory framework for these systems, managing overlapping uses of the subsurface, managing changes with the providers and off-takers of thermal energy, and obtaining financing to implement these projects. Common technical issues include geological factors such as incomplete characterization of the subsurface and reservoir heterogeneity; geochemical issues such as mineral scaling, corrosion, and biofouling; lower than expected thermal recovery; and issues with system design and reliability. This review highlights benefits and challenges faced by HT-ATES projects with the goal to use the lessons learned to improve the siting, design, development, and operation of such systems. Recommendations include improved initial subsurface site characterization, use of coupled process models to optimize system design and predict system performance, cascaded uses of stored thermal energy to better utilize the stored heat, monitoring networks to provide feedback on system performance, and expanded system scale to allow for continued operation even when maintenance of some system components is required. Techno-economic modeling and risk analysis could be used to optimize such HT-ATES project design and identify key factors that will affect sustained economic viability. In addition, design flexibility is important for these systems to allow for changing conditions regarding the supply and demand of thermal energy. Adopting these findings should improve the performance and reduce the risks for future HT-ATES projects worldwide.

15 - GEOTHERMAL ENERGY↗

Encoding Frequency Constraints in Preventive Unit Commitment Using Deep Learning With Region-of-Interest Active Sampling

With the increasing penetration of renewable energy, frequency response and its security are of significant concerns for reliable power system operations. Frequency-constrained unit commitment (FCUC) is proposed to address this challenge. Despite existing efforts in modeling frequency characteristics in unit commitment (UC), current strategies can only handle oversimplified low-order frequency response models and do not consider wide-range operating conditions. This paper presents a generic data-driven framework for FCUC under high renewable penetration. Here, deep neural networks (DNNs) are trained to predict the frequency response using real data or high-fidelity simulation data. Next, the DNN is reformulated as a set of mixed-integer linear constraints to be incorporated into the ordinary UC formulation. In the data generation phase, all possible power injections are considered, and a region-of-interest active sampling is proposed to include power injection samples with frequency nadirs closer to the UFLC threshold, which enhances the accuracy of frequency constraints in FCUC. The proposed FCUC is investigated on the IEEE 39-bus system. Then, a full-order dynamic model simulation using PSS/E verifies the effectiveness of FCUC in frequency-secure generator commitments.

42 ENGINEERING↗

Development of a Machine-Learned Cruise Guide Indicator for Rotorcraft

This paper presents a machine-learned virtual cruise guide indicator (vCGI) for Chinook helicopters. Two temporal neural networks were trained and evaluated on measured data from 55 flight tests, one for the fore rotor and another for the aft rotor, to predict a vCGI value, which protects 23 components from fatigue damage during steady-state conditions. Three different classes of machine learning architectures were evaluated for prediction of the vCGI from time sequences: a temporal convolutional neural network with 1D dilated causal convolutions, a long short-term memory recurrent neural network, and an attention-based transformer architecture. The final average model accuracy on unseen flight data is currently greater than 93% for CGI values which could result in fatigue damage and 90% for normal operation CGI values. Model accuracy was improved through a series of advancements in:(1) selection of optimal training data using temporal collective variables and unsupervised learning, (2) dataset augmentation with maximum-entropy temporal collective variables, and (3) implementation of a mixture-of-experts classification- regression approach using an adversarial classification approach to assign maneuver labels. The results are presented for each advancement in model development along with lessons learned in training machine learning models on real- world, time-dependent rotorcraft data.

Boyer, Mathew↗

Active Learning of Microgrid Frequency Dynamics Using Neural Ordinary Differential Equations

Accurate frequency modelling of inverter‐based resource (IBR)‐dominated power systems is crucial for ensuring stable, reliable and resilient operations, particularly given their inherent low‐inertia characteristics and fast dynamics that traditional swing equation‐based models inadequately capture. This paper explores neural ordinary differential equations (Neural ODEs) as a computationally efficient, data‐driven framework for modelling power system frequency dynamics, specifically within microgrids integrating high penetrations of distributed energy resources (DERs). The developed neural ODEs framework incorporates a neural network architecture designed to capture input dynamics. By actively perturbing the system with a known signal, the Python‐based neural ODEs framework was trained using measured system states and inputs, without the need for detailed system information. The framework, tested on a model of the Cordova, AK, microgrid, achieved a goodness of fit ranging from 60% to 99% across different state variables and maintained a mean square error in the 10 -6 p.u. range under square and step excitation signals. The proposed approach demonstrated robustness to measurement noise and initial condition variations while maintaining low computational complexity suitable for real‐time power system control applications. Furthermore, transfer learning enabled the neural ODEs model to adapt to the following changes in system topology or generator dispatch, highlighting its effectiveness for dynamic microgrids with frequently evolving configurations and diverse DERs.

Aryal, Tara [South Dakota State Univ., Brookings, ↗

A novel multi-model neuro-fuzzy-based MPPT for three-phase grid-connected photovoltaic system

This paper presents a novel methodology for Maximum Power Point Tracking (MPPT) of a grid-connected 20 kW photovoltaic (PV) system using neuro-fuzzy network. The proposed method predicts the reference PV voltage guarantying optimal power transfer between the PV generator and the main utility grid. The neuro-fuzzy network is composed of a fuzzy rule-based classifier and three multi-layered feed forwarded Artificial Neural Networks (ANN). Inputs of the network (irradiance and temperature) are classified before they are fed into the appropriated ANN for either training or estimation process while the output is the reference voltage. The main advantage of the proposed methodology, comparing to a conventional single neural network-based approach, is the distinct generalization ability regarding to the nonlinear and dynamic behavior of a PV generator. In fact, the neuro-fuzzy network is a neural network based multi-model machine learning that defines a set of local models emulating the complex and nonlinear behavior of a PV generator under a wide range of operating conditions. Simulation results under several rapid irradiance variations proved that the proposed MPPT method fulfilled the highest efficiency comparing to a conventional single neural network and the Perturb and Observe (P and O) algorithm dispositive. (author)

14 SOLAR ENERGY↗

Optimal Control of Differentially Private EV Charging: A Scalable Learning Approach Under Uncertainty

Internet of Things (IoT)-enabled electric vehicles (IoEVs) enable intelligent charging coordination that accounts for grid congestion. However, increased data exchange raises privacy concerns, as charging patterns can reveal sensitive driver behavior to grid operators. Here, we propose a differentially private (DP) EV charging framework that enables coordinated control while protecting driver data with theoretical privacy guarantees. Nevertheless, integrating DP inevitably introduces uncertainty into the control strategy for EVs, which can lead to infeasible solutions. To tackle this challenge, we develop a feasible and scalable control algorithm based on constrained reinforcement learning (CRL) and convex hulls. While our framework is designed to handle the uncertainty introduced by DP, it is general and also applicable to other sources of uncertainty in EV charging, such as the stochastic nature of driver behavior and renewable variability. This ensures feasible and privacy-preserving coordination of EV charging at scale. Our method constructs convex hulls within the action space to guarantee feasibility under stochastic constraints and incorporates constraint reduction techniques to improve scalability. Case studies based on IEEE benchmark systems demonstrate that the proposed approach effectively balances feasibility under uncertainty, scalability, and privacy in large-scale EV charging control.

Engineering - Power transmission and distribution↗

Reinforcement learning-based design of shape-changing metamaterials

During the last decade, artificially architected materials have been designed to obtain properties unreachable by naturally occurring materials, whose properties are determined by their atomic structure and chemical composition. In this work, we implement a new reinforcement learning (RL) method able to rationally design unique metamaterial structures at the nano-, micro-, and macroscale, which change shape during operational conditions. As an example, we apply this method to design nanostructured silicon anodes for Li-ion batteries (LIBs). The RL model is designed to apply different actions and predict change during operational conditions. The multi-component reward function comprises an increase in the total storage capacity of the resulting battery electrode and structural parameters, such as the minimum distance between the individual components of the nanostructure. Upon experimental validation using a polymer-based 3D printing technique, we expect that the newly discovered structures improve the current Si-based LIB anodes state-of-the-art by almost three times and almost ten times the current commercial LIB based on a graphitic anode. Furthermore, this RL-based optimization method opens up vast design space for other responsive metamaterials with tailored properties and pre-programmed structural transformation.

25 ENERGY STORAGE↗

Training and certification of human organizational factors designers in nuclear plant engineering - 344

Performing an increasing number of small and large-scope nuclear Human Factors Programs, AREVA NP is facing the challenging situation that (1) well-trained experts in Human Factors or Usability Engineering are currently hard to find and (2) newly hired ones need at least two years 'on the job' to become familiar with the nuclear field and the complexity of engineering Human Factors in this field. To innovatively meet this challenge we focus instead on company-internal engineers. Demonstrated nuclear engineering knowledge and skills and pronounced interest to extend both renders them optimal applicants to the AREVA NP two years Human Organizational Factors Training and Certification program. The AREVA NP training shall convey Human Factors knowledge and skills within the limited time of two years. We also stress the need to change participants' mindset from a technically-biased to a human-centered approach, committed to involve end-users throughout design. Next to psychology principles, user testing, user-centered design, plant operation principles, and the acquisition and documentation of users' requirements, design constraints, and inherent tradeoffs, participants will be trained to apply learned methods in a real environment and meet the challenges of real projects. The following sections provide insights in the training conception and preparation phase, the identification of appropriate candidates, the development of course syllabuses, and the schedule of lectures and tests. The promising experiences made thus far are reported as well. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Applications Of Machine Learning to Gas Plume Analysis In Longwave Infrared Hyperspectral Images

Longwave infrared hyperspectral images can be used for gas plume analysis, as many gases exhibit distinct absorption features in this portion of the electromagnetic spectrum. In practice, accurately identifying weak gas signatures is difficult because the observed radiance is dominated by background radiance, which varies with material, temperature, and viewing conditions. Many gas plume analysis pipelines operate on single images, limiting the ability to leverage spatial and multi-view information that could enhance the analysis. The goal of this dissertation is to explore how machine learning and deep learning methods can complement classical approaches to improve gas plume identification in longwave infrared hyperspectral imagery, and to investigate the use of neural radiance fields for hyperspectral scene reconstruction.

3D Scene Reconstruction↗

Searching for Subsecond Stellar Variability with Wide-field Star Trails and Deep Learning

We present here a method that enables wide-field ground-based telescopes to scan the sky for subsecond stellar variability. The method has operational and image processing components. The operational component takes star trail images. Each trail serves as a light curve for its corresponding source and facilitates subexposure photometry. We train a deep neural network to identify stellar variability in wide-field star trail images. We use the Large Synoptic Survey Telescope Photon Simulator to generate simulated star trail images and include transient bursts as a proxy for variability. The network identifies transient bursts on timescales down to 10 ms. We argue that there are multiple fields of astrophysics that can be advanced by the unique combination of time resolution and observing throughput that our method offers.

79 ASTRONOMY AND ASTROPHYSICS↗

Evaluation of Physical Security Risk for Potential Implementation of FLEX using Dynamic Simulation Methods

The requirements for United States nuclear power plants to maintain a large onsite physical security force contribute to their large operational costs. The cost of maintaining the current physical security posture is approximately 10% of the overall operation and maintenance budget for commercial nuclear power plants. The goal of the Light Water Reactor Sustainability Program Physical Security Pathway is to develop tools, methods, and technologies and provide the technical basis for an optimized physical security posture. This pathway will analyze and minimize the conservatisms built into current security postures in order to reduce security costs while still ensuring adequate security and operational safety. The research performed at Idaho National Laboratory within this pathway has successfully developed a dynamic force-on-force (FOF) modeling framework using various computer simulation tools and integrated them with the dynamic assessment Event Modeling Risk Assessment using Linked Diagrams (EMRALD) tool. This document provides an overview of lessons learned in applying a dynamic computational framework that links results from a commercially available FOF simulation tool, a commercially available thermal-hydraulic tool, and EMRALD to an operating commercial nuclear power plant. This process of including plant procedures and multiple analysis results is being called Modeling and Analysis for Safety Security using Dynamic EMRALD Framework. Previous reports described how a user could integrate their plant-specific FOF models with the dynamic simulation tool EMRALD, model operator actions, integrate with probabilistic risk assessment tools, such as Computer Aided Fault Tree Analysis System or Systems Analysis Programs for Hands-on Integrated Reliability Evaluations, and with thermal-hydraulic tools, such as RELAP-5. Previous reports applied various combinations of available simulations codes with EMRALD using generic plant models to demonstrate how to perform the analysis. This report documents the results of applying the dynamic computational framework to an actual nuclear facility using their security scenarios and timelines. The purpose of this study was to verify that results achieved using generic models are similar to actual plant results and to refine our guidance of the use of the framework. Such an assessment enables further analysis, such as what-if scenarios and staff-reduction evaluation, thereby optimizing physical security at plants. NOTE: The work performed in this report is based on a generic EMRALD model with actual plant data used for the analysis. However, only the generic model and general results of the analysis are in the report. No plant’s sensitive information is discussed in this report. The discussion shows examples of insights that can be obtained from the MASS-DEF methodology.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Explainable machine learning for incipient anomaly detection in compact molten salt heat exchanger with overlapping feature distributions

High-temperature molten salt-cooled reactors (MSCRs) are a promising next-generation nuclear technology option, offering efficient power conversion and inherent safety features. However, the reliability of these systems depends on the robust operation of heat exchangers (HXs), which are susceptible to failure due to temperature gradients and channel plugging caused by fluid freezing. Conventional monitoring methods, relying on inlet and outlet measurements, lack the spatial resolution needed to detect early-stage faults. We propose a novel design of a compact salt-to-salt matrix-type HX design consisting of interleaved arrays of parallel tubes, with integrated synthetic fiber optic distributed temperature sensing (DTS) to enable localized detection of incipient faults. To evaluate performance of this design, we generate high-fidelity synthetic data using heat transfer computational modeling to simulate channel plugging, and introduce sensor noise for realistic modeling of measurements. The dataset comprises of 97% normal operation and 3% anomaly cases, with each anomaly class representing 1% of the data. These early anomalies result in overlapping temperature profiles between normal and faulty channels, producing a non-separable dataset that challenges traditional classification techniques. We benchmark eight supervised machine learning (ML) models and demonstrate that XGBoost achieves the highest performance. To improve transparency, we develop an explainability framework combining Shapley values and partially ordered sets (POSETs) to quantify and structurally analyze feature importance. This approach identifies both dominant predictors and ambiguous feature relationships, enhancing trust and interpretability. Our results highlight the potential of combining DTS and explainable ML with intelligent feature selection to improve predictive maintenance and ensure operational resilience in advanced nuclear systems.

Prantikos, Konstantinos [Argonne National Laborato↗

Multi-system analysis of offshore geologic carbon storage: a review of open-source data science solutions

Geologic carbon storage projects are maturing worldwide and the footprint of deployment in the offshore is expanding. At present, there are ten projects in operation or that have been completed, more than 50 in construction and development, and dozens of characterization studies completed or underway. Offshore geologic carbon storage offers potential benefits over onshore geologic carbon storage. These offshore projects are generally remote in location, distant from population centers, and avoid complicated pore space rights while having abundant prospective storage potential. Some offshore fields targeted for carbon storage have comparatively fewer prior borehole penetrations except for areas that have been explored for petroleum production, minimizing potential issues such as pressure interference and infrastructure impacts. Yet offshore geologic carbon storage projects face distinctive technical and economic challenges, such as seafloor geohazards (e.g., seabed instability), expensive maritime transport, and meteorological-oceanographic conditions that can damage infrastructure and impact operations. Analytical capabilities and improved computational speeds have advanced engineering, earth and energy sciences in the wake of the arrival of modern data science over the last decade. These advancements have created an opportunity for integrated, multi-systems modeling approaches utilizing artificial intelligence and machine learning that are no longer limited by computational issues. Analytical tools developed alongside this advancement in data science can be leveraged to calibrate the potential advantages and challenges of carbon storage operations in the offshore. New methods and approaches that incorporate data science to analyze multiple aspects of engineered and natural systems can provide insights that complement the characterization and onsite engineering that traditional commercial and operational software addresses. These new methods and approaches can potentially improve the outcome of energy operations and carbon storage. Providing multi-system, science-driven data analytics enhances the knowledge base that offshore developers, operators, and regulatory bodies may draw from to improve offshore site selection and operational efficiency. Here, we provide a brief synopsis of geologic carbon storage efforts to date, an overview of the engineered and natural systems involved in offshore geologic carbon storage, and a review of publicly available, open-source, offshore and/or carbon storage related data- and science-driven tools developed by 2010 or later that are suitable for screening and assessing regions for offshore geologic carbon storage.

artificial intelligence↗

Q-Learning-Based Impact Assessment of Propagating Extreme Weather on Distribution Grids

The increasing number of power outage events due to extreme weather conditions is hampering us socioeconomically. Preparing in advance for the extreme weather event is critical. It can help utility operators to reduce grid damages, restore grid service quickly, allocate energy resources and repair crews strategically, and hence dramatically increase grid resilience. In this paper, we propose a method to identify the sequence of worst impact zones in the power grid caused by extreme weather events based on Q-learning (a reinforcement learning algorithm). To quantity the weather severity and its effect on the grid, we model the impact of extreme weather on the grid as a function of intensity, vulnerability, and exposure. A modified IEEE 123-node distribution feeder is presented in a mesh grid and experimented for sequences of zones identification. Finally, simulation results present the identified sequences and their associated impacts on the grid caused by extreme weather events.

distribution system↗

Q-Learning Based Impact Assessment of Propagating Extreme Weather on Distribution Grids: Preprint

Increasing number of power outage events due to extreme weather condition is hampering us socioeconomically. Preparing in advance for the extreme weather event is critical and can help utility operators to reduce grid damages, restore grid service quickly, allocate energy resources and repair crews strategically, and hence dramatically increase grid resilience. In this paper, we propose a method to identify the sequence of worst impact zones in the power grid caused by extreme weather events based on Q-learning (a reinforcement learning algorithm). To quantify weather severity and it’s effect on the grid, we model the impact of extreme weather on the grid as a function of intensity, vulnerability and exposure. A modified IEEE 123-node distribution feeder is presented in a mesh grid and experimented for sequences of zones identification. Finally, simulation results present the identified sequences and their associated impacts on the grid caused by the extreme weather events.

distribution system↗

Operational Planning Strategy for a Sequim Bay Underwater UXO Test Site (SBU2)

The Department of Defense (DoD) has identified more than 400 underwater sites that are potentially contaminated with munitions. Most areas are in shallow water (0-35 m) where the munitions pose a threat to human health and the environment. The Strategic Environmental Research and Development Program (SERDP) and Environmental Security Technology Certification Program (ESTCP) held a workshop in 2018 to establish the requirements, framework, protocols, responsibilities, and time lines for development of a series of underwater UXO Standardized Demonstration sites (“test beds”) for the purpose of testing the efficacy of sensors, equipment, and technologies to detect UXO. The workshop recommended a phased approach to building test beds to capitalize on: 1) incorporation of lessons learned from existing DoD funded projects, 2) leveraging existing underwater test bed environments funded by other programs, and 3) supporting recursive learning from early phases of the test bed development (i.e. efficiently optimizing through a scaling-up approach). This strategy document provides the framework to establish a working operational plan for the Sequim Bay Underwater UXO (SBU2) Test Bed (Phase II). Details of the plan are dependent on earlier phases of the effort as shown in Figure 1. Results from Phase I are being evaluated, and when complete, will inform the next iteration of this plan. As such, it should be considered a living document that will be informed by the lessons learned and updated as new information becomes known and requirements are refined. This approach will allow SBU2 to evolve to meet the needs of the remediation community.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗