SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles

Plug-in hybrid electric vehicles (PHEVs) show great promise in reducing transportation-related fossil fuel consumption and greenhouse gas emissions. Designing an efficient energy management system (EMS) for PHEVs to achieve better fuel economy has been an active research topic for decades. Most of the advanced systems rely either on a priori knowledge of future driving conditions to achieve the optimal but not real-time solution (e.g., using a dynamic programming strategy) or on only current driving situations to achieve a real-time but nonoptimal solution (e.g., rule-based strategy). This paper proposes a reinforcement learning–based real-time EMS for PHEVs to address the trade-off between real-time performance and optimal energy savings. The proposed model can optimize the power-split control in real time while learning the optimal decisions from historical driving cycles. Here, a case study on a real-world commute trip shows that about a 12% fuel saving can be achieved without considering charging opportunities; further, an 8% fuel saving can be achieved when charging opportunities are considered, compared with the standard binary mode control strategy.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2025

BOPTest As a Platform for Building Controls and Grid-Interactive Buildings Workforce Training

Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.

Paul, Lazlo↗

DOE OSTI2025

BOPTEST as a Platform for Building Controls and Grid-Interactive Buildings Workforce Training

Paul, Lazlo↗

DOE OSTI2018

Optimizing Voltage Control for Large-Scale Solar

Learn how NREL researchers, working with GE’s Grid Solution business and Duke Energy, used an advanced distribution management system to better manage voltage swings from utility-scale photovoltaic systems. The research brought NREL’s extensive power-hardware-in-the-loop and modeling capabilities together with real data from Duke Energy and GE’s advanced distribution management system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. Here, in this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Size-Resolved Shape Evolution in Inorganic Nanocrystals Captured via High-Throughput Deep Learning-Driven Statistical Characterization

Precise size and shape control in nanocrystal synthesis is essential for utilizing nanocrystals in various industrial applications, such as catalysis, sensing, and energy conversion. However, traditional ensemble measurements often overlook the subtle size and shape distributions of individual nanocrystals, hindering the establishment of robust structure–property relationships. In this study, we uncover intricate shape evolutions and growth mechanisms in Co 3 O 4 nanocrystal synthesis at a subnanometer scale, enabled by deep-learning-assisted statistical characterization. By first controlling synthetic parameters such as cobalt precursor concentration and water amount then using high resolution electron microscopy imaging to identify the geometric features of individual nanocrystals, this study provides insights into the interplay between synthesis conditions and the sizedependent shape evolution in colloidal nanocrystals. Utilizing population-wide imaging data encompassing over 441,067 nanocrystals, we analyze their characteristics and elucidate previously unobserved size-resolved shape evolution. This high-throughput statistical analysis is essential for representing the entire population accurately and enables the study of the size dependency of growth regimes in shaping nanocrystals. Our findings provide experimental quantification of the growth regime transition based on the size of the crystals, specifically (i) for faceting and (ii) from thermodynamic to kinetic, as evidenced by transitions from convex to concave polyhedral crystals. Additionally, we introduce the concept of an “onset radius,” which describes the critical size thresholds at which these transitions occur. This discovery has implications beyond achieving nanocrystals with desired morphology; it enables finely tuned correlation between geometry and material properties, advancing the field of colloidal nanocrystal synthesis and its applications.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2025

Ansatz-Free Hamiltonian Learning with Heisenberg-Limited Scaling

Learning the unknown interactions that govern a quantum system is crucial for quantum information processing, device benchmarking, and quantum sensing. The problem, known as Hamiltonian learning, is well understood under the assumption that interactions are local, but this assumption may not hold for arbitrary Hamiltonians. Previous methods all require high-order inverse polynomial dependency with precision, unable to surpass the standard quantum limit and reach the gold-standard Heisenberg-limited scaling. Whether Heisenberg-limited Hamiltonian learning is possible without prior assumptions about the interaction structures, a challenge we term ansatz-free Hamiltonian learning , remains an open question. In this work, we present a quantum algorithm to learn arbitrary sparse Hamiltonians without any structure constraints using only black-box queries of the system’s real-time evolution and minimal digital controls to attain Heisenberg-limited scaling in estimation error. Our method is also resilient to state-preparation-and-measurement errors, enhancing its practical feasibility. We numerically demonstrate our ansatz-free protocol for learning physical Hamiltonians and validating analog quantum simulations, benchmarking our performance against the state-of-the-art Heisenberg-limited learning approach. Moreover, we establish a fundamental trade-off between total evolution time and quantum control on learning arbitrary interactions, revealing the intrinsic interplay between controllability and total evolution-time complexity for any learning algorithm. These results pave the way for further exploration into Heisenberg-limited Hamiltonian learning in complex quantum systems under minimal assumptions, potentially enabling new benchmarking and verification protocols.

machine learning↗

DOE OSTI2021

Chapter 18: Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality

We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.

energy distribution↗

DOE OSTI2023

Measurement of Beam Energy in the Fermilab's Linac taken at the transfer Line

Linac is the first machine in the Accelerator chain at Fermilab where 𝐻− particles are accelerated from 35 keV to 400 MeV and travel to the Booster where they are stripped. of the extra electrons to become protons. The Booster requires stable beam energy at injection. The Linac extraction energy tuning has been performed using two Beam Velocity Monitors (a.k.a Griffin detectors) to maintain stable relative energy between the Booster and the Linac. Our goal is to control the Linac energy parameters using Machine Learning techniques to increase reliability and quality of beam delivered from Linac. The first step is to understand the data from the Linac extraction diagnostics; and develop reliable, accurate methods for energy measurement and control before implementing machine learning. There were two methods of energy measurement implemented, and the results of these are compared. The first method was time of flight measurements using the Phase probes inside B eam Position Monitors (BPM) that provide the beam’s phase of arrival data. The second method used the relation between beam position data from the BPMs and dispersion values from the 400MeV MAD-X model to calculate momentum variation. The results of these two energy measurement methods compare well as the Linac energy is varied.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Reinforcement Learning for Intentional Islanding in Resilient Power Transmission Systems

Intentional islanding is the process of identifying and deliberately decomposing the transmission network to form self-sustained islands from an endangered network during disruptions to improve resilience and security. Most existing intentional islanding models are offline resilience decision tools and hence do not provide outage responses in a timely manner. In this paper, a reinforcement learning (RL) based model for intentional islanding is developed, which offers real-time switching control, online deployability, and adaptability to varying system conditions. The intentional islanding process is formulated as a Markov decision process, where the optimal transmission switching policy is learned using the RL approach. The control policy is learned over an environment that encompasses a Power System Simulator for Engineering (PSS/E) model of the transmission network, facilitated by an interface to the standard openAI Gym framework. The proposed RL-based methodology aims to form stable and self-sustainable islands by ensuring voltage stability while reducing the power mismatch in the formed islands. A proximal policy optimization algorithm is designed, which is suitable for controlling the on/off status of the switches with multi-layer perceptron as value and actor networks. The effectiveness of the proposed framework in the self-recovery of the grid by island formation is applied on the modified IEEE 39-bus test network and validated by dynamic simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

A high-fidelity building performance simulation test bed for the development and evaluation of advanced controls

We present an open-source building performance simulation test bed, the Advanced Controls Test Bed (ACTB), that interfaces high-fidelity Spawn of EnergyPlus building models, with advanced controllers implemented in Python. Additionally, the ACTB leverages the Building Optimization Testing and Alfalfa platforms for managing simulations, providing an external clock, a representational state transfer (REST) application programming interface (API), and key performance indicators for evaluating the effectiveness of control strategies. The REST API allows the development of external controllers programmed in languages such as Python, which provides flexibility and a rich choice of scientific libraries for designing control sequences. We present three test cases based on the U.S. Department of Energy's Reference Small Office Building to demonstrate the ACTB's capabilities: (a) rule-based controls compliant with ASHRAE Guideline 36 control sequences; (b) an economic model predictive control implemented using do-mpc; and (c) a deep Q-network reinforcement learning agent implemented using OpenAI Gym.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗

DOE OSTI2023

A Better Method to Calculate Fuel Burnup in Pebble Bed Reactors Using Machine Learning

Burnup measurement is an important step in material control and accountancy (MC&A) at nuclear reactors, and may be done by examining gamma spectra of fuel samples. Traditional approaches rely on known correlations to specific photopeaks (e.g. 137 Cs) and operate via a standard linear regression method. However, the quality of these regression methods is limited even in the best case, and is significantly poorer at short fuel cool-down times, due to the elevated radiation background by short life-time isotopes, and self-shielding effect of the fuel. For practical operation of pebble bed reactors (PBRs), quick measurements (in minutes) and short cooling times (in hours) are required from a safety and security perspective. We investigated the efficacy and performance of machine learning (ML) methods to predict the burnup of the pebble fuel from full gamma spectra (rather than specific discrete photopeaks) and found a full-spectrum ML approach to far outperform baseline regression predictions in all measurement and cooling conditions - including in operational-like measurement conditions. We also performed model and data ablation experiments to determine the relative performance impact of our ML methods' capacity to model data nonlinearities and the inherent additional information in full spectra. Applying our ML methods, we found a number of surprising results, including improved accuracy at shorter fuel cooling times (the opposite of the norm), remarkable robustness to spectrum compression (via rebinning), and competitive burnup predictions even when using background signal only (i.e. explicitly omitting known isotope photopeaks).

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2024

Optimal control of the electron temperature profile in DIII-D using machine learning surrogate models

The viability of the tokamak as a potential fusion reactor depends on the ability to keep the plasma in a stable regime while achieving temperatures, densities, and confinement times that are as high as possible. Tokamak scenario development attempts to find plasma regimes that achieve all of these conditions and are accessible with a given set of hardware constraints. This requires the ability to control plasma properties such as the normalized beta, the internal inductance, safety factor, rotation, etc. One property that has received less attention than some of the others, but is no less critical to achieving high performance, is the electron temperature (T e ) profile. In this work, Linear Quadratic Integral (LQI) control is used to develop a controller for the electron temperature profile in DIII-D. The controller is based on a linearized model derived from the transport equation that describes the evolution of the electron temperature, and includes contributions from the neural network surrogate models NubeamNet and MMMnet. Furthermore, the controller is tested in simulation using COTSIM, and is proven capable of tracking a target T e profile.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Exploration of Domain Aware Machine Learning for Grid Analytics: Transfer-Learnt Energy Models to Assist Buildings Control with Sparse Field Data

Buildings are a primary consumer of energy in the United States and are also increasingly being perceived as providers of grid services such as load shifting, shedding and modulation. High fidelity models of building energy consumption are needed to set appropriate baselines for measurement and verification (M&V) of controllers designed for energy efficient operation of buildings and to enable buildings to provide grid services via. participation in demand response programs. State-of-the-art building energy modeling techniques either rely on Physics based models, or extensive instrumentation of the building envelope to gather “big” data to train machine learning based models such as deep neural networks. While Physics based models are often limited by their accuracy, it is not always feasible to gather a significant amount of field data required to train machine learning based models with sufficient accuracy. In this paper, we explore the use of transfer learning-based strategies to address unsatisfactory accuracy of models for estimating building energy consumption when available field data for training is sparse or of unacceptable quality. In particular, we transfer knowledge in the form of data and parameters, from Physics based simulation frameworks to the field to improve the model accuracy, thus resulting in a Physics-informed Machine Learning framework. We evaluated the efficacy of our approach on field data collected from six commercial buildings and our results indicate that the proposed transfer learning based models provide comparative (and in some cases better) accuracy than state-of-the-art machine learning and deep learning solutions, with just one month of field data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2020

Advancing Fusion with Machine Learning Research Needs Workshop Report

Abstract Machine learning and artificial intelligence (ML/AI) methods have been used successfully in recent years to solve problems in many areas, including image recognition, unsupervised and supervised classification, game-playing, system identification and prediction, and autonomous vehicle control. Data-driven machine learning methods have also been applied to fusion energy research for over 2 decades, including significant advances in the areas of disruption prediction, surrogate model generation, and experimental planning. The advent of powerful and dedicated computers specialized for large-scale parallel computation, as well as advances in statistical inference algorithms, have greatly enhanced the capabilities of these computational approaches to extract scientific knowledge and bridge gaps between theoretical models and practical implementations. Large-scale commercial success of various ML/AI applications in recent years, including robotics, industrial processes, online image recognition, financial system prediction, and autonomous vehicles, have further demonstrated the potential for data-driven methods to produce dramatic transformations in many fields. These advances, along with the urgency of need to bridge key gaps in knowledge for design and operation of reactors such as ITER, have driven planned expansion of efforts in ML/AI within the US government and around the world. The Department of Energy (DOE) Office of Science programs in Fusion Energy Sciences (FES) and Advanced Scientific Computing Research (ASCR) have organized several activities to identify best strategies and approaches for applying ML/AI methods to fusion energy research. This paper describes the results of a joint FES/ASCR DOE-sponsored Research Needs Workshop on Advancing Fusion with Machine Learning, held April 30–May 2, 2019, in Gaithersburg, MD (full report available at https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf ). The workshop drew on broad representation from both FES and ASCR scientific communities, and identified seven Priority Research Opportunities (PRO’s) with high potential for advancing fusion energy. In addition to the PRO topics themselves, the workshop identified research guidelines to maximize the effectiveness of ML/AI methods in fusion energy science, which include focusing on uncertainty quantification, methods for quantifying regions of validity of models and algorithms, and applying highly integrated teams of ML/AI mathematicians, computer scientists, and fusion energy scientists with domain expertise in the relevant areas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2017

Human Performance Measures for the Evaluation of Nuclear Power Plant Control Room Interfaces: A Systematic Review - 265

The purpose of this study was to review the literature related to measuring human performance in the design and evaluation of the human-system interface (HSI) of nuclear power plant (NPP) control rooms and synthesize lessons learned and future directions for research and practice in this field. The design and evaluation of digital or hybrid HSI requires comprehensive measurement of human performance to ensure quality and safety of the nuclear power process control. Human factors evaluation frameworks have been proposed to guide the measurement of human performance constructs. However, there are few guides for selecting appropriate measures for those constructs. A systematic review of empirical research literature was conducted. Studies were identified through academic publication databases and grey literature searches. Eligible studies were coded and information regarding human performance measures was extracted and analyzed. A total of 80 studies met the criteria. Human performance measures extracted from the reviewed studies included measures of: behavioral performance; workload; situation awareness; human-human or human-machine cooperation; other cognitive performance indicators; and system performance. Behavioral performance (speed and accuracy), workload, and situation awareness were the most frequently measured constructs with validated and domain-specific instruments. The results of the literature review provided an overview of the human performance constructs and their corresponding measures for NPP control room HSI evaluation. These results can be used as an initial guide to establish appropriate and comprehensive measurements of human performance for such evaluations. Future directions in measure development are also discussed. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗