Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Machine Learning-Based PV Reserve Determination Strategy for Frequency Control on the WECC System

This paper proposes a machine learning based strategy, that is suitable for real-time operation, to determine the optimal photovoltaic (PV) power plants reserve for frequency control. The proposed machine learning algorithm is trained and tested on 1,987 offline simulations of a 60% renewable penetration Western Electricity Coordinating Council (WECC) system. On a realistic 1-day operation profile of the WECC system, the ML model demonstrates a savings of more than 40% PV headroom compared to a conservative approach.

14 SOLAR ENERGY↗
DOE OSTI2025

Quantum Reinforcement Learning for Volt-VAR Control in Power Distribution Systems

Volt-VAR control (VVC) is crucial in active distribution networks for optimizing voltage profiles and minimizing network losses. While traditional deep reinforcement learning (DRL) algorithms exhibit promise for VVC, they often require extensive computational resources to handle such a high-dimensional problem. As a potential solution, quantum reinforcement learning (QRL) algorithms integrate the computational capabilities of quantum computing into the DRL framework. However, existing QRL algorithms struggle with complex VVC problems due to the limitations of current quantum hardware. To bridge this gap, this paper proposes an innovative QRL algorithm featuring an end-to-end architecture that integrates a classical autoencoder, variational quantum circuits (VQCs), and classical post-processing layers. This design efficiently compresses high-dimensional grid states, enabling VQCs to leverage quantum advantages while producing multiple control device outputs tailored for VVC tasks. Numerical studies on three representative distribution systems verify the effectiveness and scalability of the proposed QRL algorithm, and demonstrate its enhanced performance over classical approaches with only approximately 1% of the parameters. Additionally, the robustness of our developed algorithm is validated through noisy quantum environments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Machine learning modeling and model predictive control of a closed-circuit reverse osmosis system

Closed-circuit reverse osmosis (CCRO) offers a flexible and energy-efficient alternative to conventional reverse osmosis by operating in a semi-batch mode that recycles brine, enabling higher recovery rates and reduced specific energy consumption (SEC). However, developing accurate, system-level dynamic models for CCRO remains challenging due to its nonlinear, multi-phase operation and sensitivity to variable feed water conditions. Traditional modeling approaches, such as NARMAX (nonlinear autoregressive moving average with exogenous inputs), often struggle to generalize across varying inlet feed concentrations, necessitating frequent parameter re-estimation and limiting their utility for real-time control applications. To address these limitations, we developed a long short-term memory (LSTM) neural network model trained on an extensive experimental data set from a CCRO pilot plant. The model accepts three inputs, feed flow rate, recirculation flow rate, and initial feed conductivity, and predicts three key outputs: reject conductivity, feed pump power draw, and recirculation pump power draw. We validated the LSTM model against experimental data, demonstrating its ability to distinguish between different feed conductivities and adapt to variable flow rates. Subsequently, we incorporated the LSTM model within a nonlinear model predictive control (MPC) scheme and conducted closed-loop simulations to optimize the integrated SEC (iSEC). In conclusion, the results project up to a 6% reduction in iSEC by using MPC to optimize performance over the entire experiment duration, without requiring any random excitation for data collection or parameter re-estimation.

Desalination↗
DOE OSTI2025

Artificial Intelligence/Deep Learning FRNN Software for Prediction & Real-Time Control of DIII-D Plasma Control System (PCS)

This collaborative project integrated an improved version of the Artificial Intelligence/Deep Learning FRNN prediction and control software into the real-time DIII-D PCS (plasma control system). A key AI/DL software challenge is to build a modern high-performance computing (HPC) enabled “synthetic plasma simulator” capable of carrying out HPC-driven real-time plasma control applications. This involves development of a deep learning framework to train the surrogate model for a first-principles-based instability analysis simulator (“SGTC”) derived from the global gyrokinetic code GTC. The role of SGTC is to provide accurate and detailed plasma instability information from a real-time AI-based simulator capability to complement the deep learning prediction and control from experimentally-measured signals, such as ECE Imaging, supplemented by synthetic SGTC-ECEI.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2019

Automated control of microfluidic devices based on machine learning

A system is provided to automatically monitor and control the operation of a microfluidic device using machine learning technology. The system receives images of a channel of a microfluidic device collected by a camera during operation of the microfluidic device. Upon receiving an image, the system applies a classifier to the image to classify the operation of the microfluidic device as normal, in which no adjustment to the operation is needed, or as abnormal, in which an adjustment to the operation is needed. When an image is classified as normal, the system may make no adjustment to the microfluidic device. If, however, an image is classified as abnormal, the system may output an indication that the operation is abnormal, output an indication of a needed adjustment, or control the microfluidic device to make the needed adjustment.

42 ENGINEERING↗
DOE OSTI2021

Automated control of microfluidic devices based on machine learning

A system is provided to automatically monitor and control the operation of a microfluidic device using machine learning technology. The system receives images of a channel of a microfluidic device collected by a camera during operation of the microfluidic device. Upon receiving an image, the system applies a classifier to the image to classify the operation of the microfluidic device as normal, in which no adjustment to the operation is needed, or as abnormal, in which an adjustment to the operation is needed. When an image is classified as normal, the system may make no adjustment to the microfluidic device. If, however, an image is classified as abnormal, the system may output an indication that the operation is abnormal, output an indication of a needed adjustment, or control the microfluidic device to make the needed adjustment.

Giera, Brian↗
DOE OSTI2025

Deploying advanced supervisory control strategies to small- and medium-sized commercial buildings: Case study and lessons learned

Deploying advanced supervisory control strategies (ASCSs) in small- and medium-sized commercial buildings (SMBs) is vital but faces two issues: (1) a lack of building control and communication infrastructure (BCCI) in SMBs and (2) the significant engineering efforts required to implement and configurate ASCSs. Despite hindering the large-scale adoption of ASCSs in SMBs, these issues have not been adequately explored in the literature, which tends to focus more on feasibility than scalability. This paper provides a comprehensive evaluation of these two issues through a case study of an occupied SMB in eastern Tennessee of the United States. Specifically, we design and implement a BCCI for the studied building with commercial off-the-shelf products to accommodate the needs for deploying ASCSs. Here, we then deploy two ASCSs—a rule-based setback control and a model predictive control (MPC)—with the BCCI and evaluate their performance throughout the summer of 2024. This study reveals that the main bottlenecks in establishing BCCIs for SMBs are the high initial cost (∼$56/m 2 ) and communication delays (up to 9 min). Additionally, the assessment of the two ASCSs indicates that the majority (∼75 %) of the engineering effort required for implementation and configuration is spent on model identification, debugging, and tuning.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗
DOE OSTI2021

A synthetic building operation dataset

Abstract This paper presents a synthetic building operation dataset which includes HVAC, lighting, miscellaneous electric loads (MELs) system operating conditions, occupant counts, environmental parameters, end-use and whole-building energy consumptions at 10-minute intervals. The data is created with 1395 annual simulations using the U.S. DOE detailed medium-sized reference office building, and 30 years’ historical weather data in three typical climates including Miami, San Francisco, and Chicago. Three energy efficiency levels of the building and systems are considered. Assumptions regarding occupant movements, occupants’ diverse temperature preferences, lighting, and MELs are adopted to reflect realistic building operations. A semantic building metadata schema - BRICK, is used to store the building metadata. The dataset is saved in a 1.2 TB of compressed HDF5 file. This dataset can be used in various applications, including building energy and load shape benchmarking, energy model calibration, evaluation of occupant and weather variability and their influences on building performance, algorithm development and testing for thermal and energy load prediction, model predictive control, policy development for reinforcement learning based building controls.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

AlphaBuilding - Synthetic Buildings Operation Dataset

This is a synthetic building operation dataset which includes HVAC, lighting, miscellaneous electric loads (MELs) system operating conditions, occupant counts, environmental parameters, end-use and whole-building energy consumptions at 10-minute intervals. The data is created with 1395 annual simulations using the U.S. DOE detailed medium-sized reference office building, and 30 years' historical weather data in three typical climates including Miami, San Francisco, and Chicago. Three energy efficiency levels of the building and systems are considered. Assumptions regarding occupant movements, occupants' diverse temperature preferences, lighting, and MELs are adopted to reflect realistic building operations. A semantic building metadata schema - BRICK, is used to store the building metadata. The dataset is saved in a 1.2 TB of compressed HDF5 file. This dataset can be used in various applications, including building energy and load shape benchmarking, energy model calibration, evaluation of occupant and weather variability and their influences on building performance, algorithm development and testing for thermal and energy load prediction, model predictive control, policy development for reinforcement learning based building controls.

AlphaBuilding↗
DOE OSTI2019

A Real-Time Big Data Control-Theoretical Framework for Cyber-Physical-Human Systems [Book Chapter]

It is known that cyber-physical-human systems naturally arise from interdependent infrastructure systems and smart connected communities. Such applications require ubiquitous information sensing and processing, intelligent machine-to-machine communication for a seamless coordination, as well as intelligent interactions between humans and machines. This chapter presents a control-theoretical framework to model heterogeneous physical dynamic systems, information and communication, as well as cooperative controls and/or distributed optimization of such interconnected systems. It is shown that efficient analytical and computational algorithms can be modularly designed and hierarchically implemented to operate and optimize cyber-physical-human systems, first to quantify individually the input–output relationship of nonlinear dynamic behaviors of every physical subsystem, then to coordinate locally both cyber-physical interactions of neighboring agents as well as physical-human interactions, and finally to dynamically model and optimize the overall networked system. The hierarchical structure makes the overall optimization and control problem scalable and solvable. Moreover, the three levels integrate individual designs and optimization, distributed cooperative optimization, and decision-making through real-time, data-driven, model-based learning and control. Specifically, one of the contributions of the chapter is to demonstrate how the combination of dissipativity theory and cooperative control serves as a natural framework and promising tools to analyze, optimize, and control such large-scale system. Application to digital grid is investigated as an illustrative example.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Predicting disruptive instabilities in controlled fusion plasmas through deep learning

Nuclear fusion power delivered by magnetic-confinement tokamak reactors holds the promise of sustainable and clean energy1. The avoidance of large-scale plasma instabilities called disruptions within these reactors is one of the most pressing challenges, because disruptions can halt power production and damage key components. Disruptions are particularly harmful for large burning-plasma systems such as the multibillion-dollar International Thermonuclear Experimental Reactor (ITER) project currently under construction, which aims to be the first reactor that produces more power from fusion than is injected to heat the plasma. Here we present a method based on deep learning for forecasting disruptions. Our method extends considerably the capabilities of previous strategies such as first-principles-based and classical machine-learning approaches. In particular, it delivers reliable predictions for machines other than the one on which it was trained—a crucial requirement for future large reactors that cannot afford training disruptions. Our approach takes advantage of high-dimensional training data to boost predictive performance while also engaging supercomputing resources at the largest scale to improve accuracy and speed. Trained on experimental data from the largest tokamaks in the United States (DIII-D) and the world (Joint European Torus, JET), our method can also be applied to specific tasks such as prediction with long warning times: this opens up the possibility of moving from passive disruption prediction to active reactor control and optimization. These initial results illustrate the potential for deep learning to accelerate progress in fusion-energy science and, more generally, in the understanding and prediction of complex physical systems.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2014

Final Project Report "Advanced Concept Exploration For Fast Ignition Science Program"

The Fast Ignition (FI) Concept for Inertial Confinement Fusion has the potential to provide a significant advance in the technical attractiveness of Inertial Fusion Energy (IFE) reactors. FI differs from conventional “central hot spot” (CHS) target ignition by decoupling compression from heating: using the laser (or heavy ion beam or Z pinch) drive pulse (10’s of ns) to create a dense fuel and a second, much shorter (~10 ps) high intensity pulse to ignite a small region of it. There are two major physics issues concerning this concept; controlling the laser-induced generation of large electron currents and their propagation through high density plasmas. This project has addressed these two significant scientific issues in Relativistic High Energy Density (RHED) physics. Learning to control relativistic laser matter interaction (and the limits and potential thereof) will enable a wide range of applications. While these physics issues are of specific interest to inertial fusion energy science, they are also important for a wide range of other HED phenomena, including high energy ion beam generation, isochoric heating of materials, and the development of high brightness x-ray sources. Generating, controlling, and understanding the extreme conditions needed to advance this science has proved to be challenging: Our studies have pushed the boundaries of physics understanding and are at the very limits of experimental, diagnostic, and simulation capabilities in high energy density laboratory physics (HEDLP). Our research strategy has been based on pursuing the fundamental physics underlying the Fast Ignition (FI) concept. We have performed comprehensive study of electron generation and transport in fast-ignition targets with experiments, theory, and numerical modeling. A major issue is that the electrons produced in these experiments cannot be measured directly—only effects due to their transport. We focused mainly on x-ray continuum photons from bremsstrahlung and x-ray line radiation from K-shell fluorescence. Integrated experiments, which combine target compression with short-pulse laser heating, yield additional information on target heating efficiency. This indirect way of studying the underlying behavior of the electrons must be validated with computational modeling to understand the physics and improve the design. This program execution required a large, well-organized team and it was managed by a joint Collaboration between General Atomics (GA), Lawrence Livermore National Laboratory (LLNL), and the Laboratory for Laser Energetics (LLE). The Collaboration was formed 8 years ago to understand the physics issues of the Fast Ignition concept, building on the strengths of each partner. GA fulfills its responsibilities jointly with the University of California, San Diego (UCSD), The Ohio State University (OSU) and the University of Nevada at Reno (UNR). Since RHED physics is pursued vigorously in many countries, international researchers have been an important part of our efforts to make progress. The division of responsibility was as follows: (1) LLE had primary leadership for channeling studies and the integrated energy transfer, (2) LLNL led the development of measurement methods, analysis, and deployment of diagnostics, and (3) GA together with UCSD, OSU and UNR studied the detailed energy-transfer physics. The experimental program was carried out using the Titan laser at the Jupiter Laser Facility at LLNL, the OMEGA and OMEGA EP lasers at LLE and the Texas Petawatt laser (TPW) at UT Austin. Modeling has been pursued on large computing facilities at LLNL, OSU, and UCSD using codes developed (by us and others) within the HEDLP program, commercial codes, and by leveraging existing supercomputer codes developed by the NNSA ICF program. This Consortium brought together all the components—resources, facilities, and personnel—necessary to accomplish its aggressive goals. The ACE Program has been strongly collaborative, taking advantage of the expertise of the participating institutions to provide a research effort that is far greater than the sum of its parts. The results of this work have firmly strengthened the scientific foundation from which the viability of FI and other applications can be evaluated. Program execution has also led to improved diagnostics for probing dense, hot plasmas, detailed understanding of high-current, relativistic electron energy generation and transport across boundaries and into dense plasmas, and greatly enhanced predictive modeling capabilities. One important aspect of this program was the involvement and training of young scientists including postdoctoral fellows and graduate students. During the entire 8 years of FI and ACE project period since 2005, more than fifteen graduate students completed their doctoral dissertations including three from OSU and two from UCSD in last three years. This project generated an impressive forty articles in high quality journals including nine (including two under review) in Physical Review Letters during the last funding period since 2011.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Integrating Overhead Camera Imagery with Reinforcement Learning to Improve Fuel Economy Through Adaptive Traffic Control

Overhead cameras have been deployed for some time as a sensor for traffic control, with many advantages over conventional inductive loop sensors in terms of robustness, ease of installation, and simplicity with respect to maintenance. We explore the utility of such imagery (in particular, fisheye lens cameras) for intelligent traffic control in real-time using reinforcement learning, with a goal of improving fuel economy for the overall intersection and grid of intersections. We summarize our initial work in data collection and approach to ascertain the ability to visually sense vehicular fuel consumption using real-world data. We then describe our deep reinforcement learning topology to test adaptive control informed by the camera capabilities. Next, we tested the proof-of-concept through a variety of simulation scenarios using the Simulation of Urban Mobility (SUMO) traffic simulator, which included different estimates on vehicle classifications, different traffic distributions, and different traffic densities. Six different control policies were tested ranging from simple timing, to heuristic visual-based methods, to actual policies learned via the reinforcement learning approach. Our results demonstrate the feasibility of the approach for controlling traffic lights for better fuel efficiency based on visual vehicle estimates from commercial, fisheye lens cameras.

Karnowski, Thomas↗
DOE OSTI2022

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters

Over the last few decades, the deployment of distributed solar photovoltaic (PV) systems has increased consistently. High PV penetration could cause adverse effects on the grid, such as voltage violations. This paper proposes a new distributed soft actor-critic based multi-agent deep reinforcement learning (SAC-MADRL) control solution to minimize the PV real power curtailment while keeping the grid voltage in an acceptable range. New reward functions have been designed to coordinate different agents during the learning process, yielding improved convergence. Comparison results with other control methods on a real feeder in western Colorado U.S. with 80% penetration of PVs demonstrate that the proposed method has better capability of effectively regulating voltage while minimizing the PV real power curtailment.

distribution system↗
DOE OSTI2021

Deep Reinforcement Learning for Autonomous Water Heater Control

Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

A Data Science and Machine Learning Platform Supporting Large Particle Accelerator Control and Diagnostics Applications Final Report: SBIR Initial Phase II DE-SC0022583

The Machine Learning Data Platform (MLDP) is a product providing full-stack support for data science, Machine Learning, and Artificial Intelligence (ML/AI) applications at particle accelerator and large experimental physics facilities. It supports ML/AI applications from front-end, high-speed acquisition of heterogeneous, time-series data, through data archiving and management, to back-end analysis. The MLDP embodies a “data-science ready” platform for data analysis and ML/AI applications in diagnosis, modelling, control, and optimization of these facilities. It provides data scientists and applications a consistent, datacentric interface to archive data standardizing implementation and deployment of ML/AI algorithms to different operations configurations within the same facility, or between facilities. Being an open-source, public-domain project, the MLDP is intended for broadest possible impact by increasing accessibility and minimizing the required expertise for installation and operation. The MLDP can also be deployed at user facilities for experimental data collection, archiving, and analysis. It is capable of acquisition and archiving of heterogeneous data from experimental equipment (e.g., images, arrays, structures, etc.) along with system hardware configurations (e.g., scalars, tables), control system process variables, and any metadata required for provenance. Thus, the MLDP can manage experimental data through its entire lifecycle, from acquisition and archiving, through analysis and investigation, to release and final publication.

43 PARTICLE ACCELERATORS↗