Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Initial testing of Alfvén eigenmode feedback control with machine-learning observers on DIII-D

A first of its kind fully data-driven system has been developed and implemented into the DIII-D plasma control system to detect and control Alfvén eigenmodes (AE) in real-time. Susceptibility to fast ion-induced AE is a challenge in fully non-inductive tokamak operation, which significantly reduces fast-particle confinement and results in degraded fusion gain. Controlling AEs in real-time to improve fast-ion confinement is, hence, important for future advanced tokamak fusion reactors. The models were implemented and tested in experiments which showed that neural networks (NN) are highly effective in detecting 5 types of AE (BAE, EAE, LFM, RSAE, TAE) using high resolution ECE. To estimate the neutron deficit, a NN has been trained that outputs the classical neutron rate using similar inputs to NUBEAM. Also a preliminary ML-based proportional control has been designed and gone through initial testing in experiment to use feedback-control on the neutral beam power to achieve desired amplitude of AE modes and neutron deficits. The effect of AEs on fast-ion confinement is measured by analysing the gap in classical neutron rate from the proposed NN-based NUBEAM and the measured neutron rate.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Machine learning reveals factors that control ion mobility in anti-perovskite solid electrolytes

Solid-state batteries are projected to exhibit improved energy densities and safety compared to liquid-electrolyte-based systems. Consequently, the development of solid electrolytes that can conduct cations at rates comparable to that of liquid electrolytes is an active area of research. To facilitate this search it is desirable to understand the chemical and structural features that control ion transport through a crystalline lattice. Here, machine learning (ML) is used to identify features that influence ion mobility in solids, quantify the relative importance of those features, and demonstrate how the variation of individual features alters ion mobility. Adopting the anti-perovskite lattice as a model system, ML algorithms were trained to predict migration barriers from a data set containing hundreds of barriers calculated using density functional theory. In this study, more than 10 6 feature sets of non-redundant descriptors were evaluated for their ability to predict migration barriers. The present analysis correctly reiterates the importance of several features that are known to influence mobility – for example, anion polarizability – but also identifies others whose importance is not widely recognized. Lattice properties such as hopping distance and channel width exhibit the greatest influence on cation mobility. Individual conditional expectation analyses performed on these features shows that barriers decrease as hopping distance decreases and as channel width increases. The defect formation energy – which is commonly associated with the concentration of defects, but not with their mobility – is identified as an important feature for interstitial migration. In sum, this analysis aids in the design of optimal solid electrolytes by simplifying the multi-dimensional design space to a sub-set of properties that are the most important.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Deep Reinforcement Learning for Automatic Generation Control of Wind Farms

This paper provides a model-free framework for real-time control of wind farms to accurately track a power reference signal. This problem requires tractable dynamical models for capturing the aerodynamic interaction between wind turbines and controllers that can make decisions in realtime given varying atmospheric conditions. In this paper, we propose a deep reinforcement learning framework to provide real-time yaw control of a wind farm. Modifications have been made to FLOw Redirection and Induction in Steady State (FLORIS), a modeling tool that incorporates transient wake behavior. The control problem is formulated to track a synthetic power reference signal based on historical atmospheric (wind speed and direction) information, price signals, and regulation deployment data from U.S. regional transmission operators. Results indicate that a wind farm, with this control paradigm, can achieve good tracking performance when tested with real atmospheric data.

49 EE - Wind and Water Power Program - Wind (EE-4W↗
DOE OSTI2025

Digital twin framework for PIP-II linac: AI-driven multi-scale modeling from ion source to 800 MeV

The PIP-II superconducting linac at Fermilab is designed to deliver multi-megawatt proton beams for neutrino physics and other high-intensity applications. To expedite commissioning and enhance operational reliability, we have developed an EPICS-based data flow framework that seamlessly integrates digital twins (DT) with physical twins (PT). These digital twins comprise high-fidelity beam dynamics models or data-driven surrogate models connected to their physical counterparts through real-time diagnostics and advanced machine-learning algorithms.Central to this framework is Linac_Gen, an accelerated simulation tool that incorporates convolutional neural networks, random forests, and genetic algorithms to provide up to a tenfold speedup in optimizing the accelerator geometry model. An EPICS translator layer ensures interoperability by efficiently mapping lattice parameters across diverse simulation platforms.Our EPICS-based framework supports multiple operational modes—monitoring, passive learning, closed-loop control, and online learning—covering the entire machine lifecycle. By leveraging HPC resources and multi-objective optimization techniques, the digital twin enables adaptive trajectory correction, real-time fault detection, and predictive modeling of beam stability. This comprehensive approach paves the way for robust, high-intensity operation and data-driven accelerator R&D at Fermilab.

Pathak, Abhishek [Fermilab]↗
DOE OSTI2021

Data-driven building energy modeling with feature selection and active learning for data predictive control

Three gaps impede the development of cost-effective and accurate data-driven building energy modeling/models (DBEM) for energy forecasting and predictive control strategies. Gap 1: data bias is common in building operation data, but this topic is hardly studied in DBEM; Gap 2: high data dimensionality is common in DBEM, but a systematic and scalable methodology is lacking to solve the problem; Gap 3: the interactions between data bias and high dimensionality have not been systematically studied for DBEM and predictive control in buildings. In this work, to address the three gaps mentioned above, we develop a framework that integrates active learning and feature selection for DBEM used for whole building data predictive control (or DPC, which is a branch of model predictive control). The framework provides a systematic methodology and automatic workflow that starts with raw data from building automation systems to the establishment of data-driven energy models for DPC controllers. The developed strategies and framework are evaluated in a virtual testbed based on EnergyPlus and BCVTB. Improved performance and reduced computational complexity are observed from the DBEM built with the developed framework, as well as the DPC controller based on that DBEM, indicating the effectiveness of the developed framework.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

MLOps for Beam Controls

Machine learning operations (MLOps) is the standardization and streamlining of the ML development lifecycle to address the challenges associated with large-scale machine learning applications. The full MLOps pipeline consists of open-source tools: DataHub, MinIO and MLflow. It is being used for dataset management and model development to handle changing data dependencies, varying business needs, reproducibility, and diverse teams working with differing tools and skills. To demonstrate the completion of an MLOps pipeline for particle accelerator operations, we are deploying a simple script that computes settings for the Booster’s gradient magnet power supply. Once the demonstration is complete, we will develop and deploy ML-based optimization algorithms to improve Booster’s overall efficiency. This MLOps pipeline opens the gate to systematically develop and deploy ML applications for accelerator controls and diagnostics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

MLOps for Beam Controls

Machine learning operations (MLOps) is the standardization and streamlining of the ML development lifecycle to address the challenges associated with large-scale machine learning applications. The full MLOps pipeline consists of open-source tools: DataHub, MinIO and MLflow. It is being used for dataset management and model development to handle changing data dependencies, varying business needs, reproducibility, and diverse teams working with differing tools and skills. To demonstrate the completion of an MLOps pipeline for particle accelerator operations, we are deploying a simple script that computes settings for the Booster’s gradient magnet power supply. Once the demonstration is complete, we will develop and deploy ML-based optimization algorithms to improve Booster’s overall efficiency. This MLOps pipeline opens the gate to systematically develop and deploy ML applications for accelerator controls and diagnostics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

MLOps for Beam Controls

Machine learning operations (MLOps) is the standardization and streamlining of the ML development lifecycle to address the challenges associated with large-scale machine learning applications. The full MLOps pipeline consists of open-source tools: DataHub, MinIO and MLflow. It is being used for dataset management and model development to handle changing data dependencies, varying business needs, reproducibility, and diverse teams working with differing tools and skills. To demonstrate the completion of an MLOps pipeline for particle accelerator operations, we are deploying a simple script that computes settings for the Booster’s gradient magnet power supply. Once the demonstration is complete, we will develop and deploy ML-based optimization algorithms to improve Booster’s overall efficiency. This MLOps pipeline opens the gate to systematically develop and deploy ML applications for accelerator controls and diagnostics.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Xopt and Badger: a machine learning ecosystem for real-time accelerator control and optimization

Machine learning (ML)-based black-box optimization algorithms have demonstrated significant improvements in accelerator optimization speed, often by orders of magnitude. However, deploying these algorithms in real-time facility control remains challenging due to the specialized expertise and infrastructure required. To bridge this gap, we introduce the Xopt ecosystem, a versatile suite of tools designed to make advanced ML-based optimization accessible to the broader accelerator community. This ecosystem includes Xopt, a modular Python framework that facilitates the integration of ML-based optimization algorithms with arbitrary control problems, and Badger, a graphical user interface built on top of Xopt, which enables seamless deployment of ML algorithms in real-time control systems. The Xopt ecosystem has been successfully applied towards solving challenging real-time control problems at leading international accelerator facilities, including SLAC, LBNL, Argonne, Fermilab, BNL, DESY, and ESRF, demonstrating its effectiveness in real-world optimization tasks. In this presentation, we provide an overview of Xopt’s capabilities and illustrate its impact through case studies from SLAC accelerator facilities including LCLS, LCLS-II, and FACET-II.

Roussel, Ryan [SLAC]↗
DOE OSTI2022

Anomaly Detection and Mitigation in FACTS-based Wide-Area Voltage Control Systems using Machine Learning

With the increasing deployment of Flexible AC Transmission System (FACTS) devices in wide-area voltage control systems (WAVCS) for achieving improved voltage stability of bulk power systems, the possibility for cyber attacks on these systems is also increasing. Successful stealthy cyber attacks that are difficult to detect by traditional informational technology (IT)-based cybersecurity solutions or threshold-based bad data detectors can lead to a voltage collapse in power grid. This paper presents the testbed-based attacks implementation and real-time evaluation of machine learning (ML) algorithm for detecting and mitigating stealthy cyber attacks on FACTS-based WAVCS on a hardware-in-the-loop (HIL) testbed. Initially, we discuss the implementation of a fuzzy logic controller (FLC) that controls a Static VAR Compensator (SVC) device deployed in a two-area four-machine Kundur power system for improving transient voltage stability. Later, the ML-based Anomaly Detection and Mitigation (ADM) system is implemented on the cyber-physical HIL testbed to detect and mitigate various stealthy cyber attacks, which are injected in real-time over the wide-area network (WAN). The experimental results show accurate and effective performance of ADM system in detecting and mitigating anomalies while keeping the grid stable and within the system operating limits, as defined by the North America Electric Reliability Corporation (NERC).

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Chapter 7: Learning Stable Local Volt/Var Controllers in Distribution Grids

This chapter describes a framework to synthesize provably stable local Volt/Var controllers for distributed energy resources (DERs) in power distribution grids (DGs). The goal is to control the reactive power injections of DERs to improve the system performance as quantified by a generic optimal reactive power flow (ORPF) problem. To achieve this, we jointly design for each DER the control function, which prescribes the reactive power update rule, and the equilibrium function, which approximates the ORPF solutions from local measurements of voltages and powers. We provide conditions on the equilibrium functions and the control parameters ensuring the stability of the closed-loop system. In particular, we discuss the trade-offs between each set of conditions accounting for practical considerations, like fully exploiting the DERs' generation capabilities and reducing the optimality gap. These conditions are then translated into learning constraints on the neural networks' parameters that are enforced in the training phase. We validate our framework with numerical simulations on the IEEE 37-bus network and through a comparison with an optimized version of standard piece wise linear control rules.

closed-loop asymptotic stability↗
DOE OSTI2024

Developing an intrinsically secure information barrier for arms control verification through machine learning

Near-term solutions are needed to allow for flexible engagement in future nuclear arms control discussions. This project developed a method for implementing an information barrier (IB) on commercial systems, shortening the research and development lifecycle for warhead verification technologies while offering improved and inherently flexible capabilities. The crux of the verification challenge remains the difficulty in developing an authenticatable IB which prevents sensitive host country information from inadvertent transmission to an inspector. Many concepts for IB’s rely on dedicated “trusted” processor modules developed with dedicated custom radiation detection systems and associated algorithms. Without a priori knowledge of the treaty item, the parameter space for measurements can be nearly infinite and robustness against spoofing without the ability to view sensitive data is key. This project has produced an unclassified framework capable of ingesting data from common gamma detectors and identifying the presence of weapons grade nuclear material at over 90% accuracy.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

Data-Driven Distribution System Coordinated PV Inverter Control Using Deep Reinforcement Learning

The deployment of distributed solar photovoltaic (PV) systems has increased consistently over the past decades. High penetrations of PVs could cause a series of adverse grid impacts, such as voltage violations. The recent development of smart inverter technologies rises the incentives of developing PV control solutions that regulate the inverter output power and seeking the optimization on system operational objectives. This paper proposes a data-driven control solution based on deep reinforcement learning (DRL) to optimize PV inverters for voltage regulation. The proposed solution can minimize PV real power curtailment while maintaining network voltage at an acceptable range. Comparison results between the proposed DRL control algorithms with deep deterministic policy gradient (DDPG) and volt-var control on a real feeder in west Colorado highlight the advantage of the proposed framework in controlling the system voltage while minimizing the PV real power curtailment.

deep reinforcement learning↗
DOE OSTI2022

A Machine Learning-Assisted Framework to Control Thermally Anisotropic Building Envelopes in Residential Buildings

To curb the energy consumption of buildings and their related CO2 emissions, Oak Ridge National Laboratory (ORNL) has developed the thermally anisotropic building envelope (TABE) —a multi-layer design comprising insulation materials and metal foils connected to thermal loops. In this study, a machine learning-assisted framework was developed to control the TABE in residential buildings to reduce the computation load for future optimal rule-based control and application. First, a 2D finite element model was established in COMSOL to calculate the hourly heat flux through exterior walls installed with the TABE. Then, TABE wall heat fluxes were simulated for various given indoor and outdoor boundary conditions, as well as thermal loops fluid temperatures and flow rates. Since the finite element simulations are computationally expensive, an artificial neural network (ANN) was then trained to use as a proxy of the finite element (COMSOL) modeling. Finally, the trained ANN model was coupled with the EnergyPlus model to predict the energy consumption of a US Department of Energy prototype single-family house installed with the TABE. An optimal simple rule-based control was determined from predefined rules for a case study. The results demonstrate that the developed machine learning–assisted framework can reduce 99.9% of the computation time while efficiently managing residential building energy for installed TABE walls.

Shen, Zhenglai↗
DOE OSTI2020

Deep Reinforcement Learning for Residential HVAC Control with Consideration of Human Occupancy

The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

Mckee, Evan↗
DOE OSTI2021

A Cyber-Physical System for Freeway Ramp Meter Signal Control Using Deep Reinforcement Learning in a Connected Environment

Freeway bottlenecks such as on-ramp merging areas account for about 40% of recurring freeway congestion. It is generally agreed that building more roads and adding more lanes to existing infrastructure does not solve the congestion problem, and so dynamic traffic control measures offer a more cost-effective alternative. Ramp meters, traffic signal devices that regulate traffic flow entering freeways, are among the most effective measures to mitigate congestion at on-ramp merging areas on freeways. The confluence of deep reinforcement learning (RL) and connectivity provides a possible solution to advance ramp meter signal control. Deep RL is a group of machine-learning methods that enables an agent learning from the environment to improve its performance. In this study, three deep RL methods-proximal policy optimization (PPO), Ape-X deep Q-network (DQN), and asynchronous advantage actor-critic agents (A3C)-are explored for ramp meter signal control to maximize vehicle speed and traffic throughput, as well as to minimize energy consumption and emissions at freeway on-ramp merging areas in a connected environment. The low computational requirement and scalability of deep RL for deployment make it a powerful optimization tool for time-sensitive applications such as ramp meter signal control. The results of this study show that deep RL methods yield superior performance to both a fixed-time controller and ALINE A, a state-of-the-art feedback controller.

ADVANCED PROPULSION SYSTEMS,MATHEMATICS AND COMPUT↗
DOE OSTI2022

Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control

Deep Reinforcement Learning (DRL) has started showing success in real-world applications such as building energy optimization. Much of the research in this space utilized simulated environments to train RL-agent in an offline mode. Very few research have used DRL-based control in real-world systems due to two main reasons: 1) sample efficiency challenge---DRL approaches need to perform a lot of interactions with the environment to collect sufficient experiences to learn from, which is difficult in real systems, and 2) comfort or safety related constraints---user's comfort must never or at least rarely be violated. In this work, we propose a novel deep Reinforcement Learning framework with online Data Augmentation (RLDA) to address the sample efficiency challenge of real-world RL. We used a time series Generative Adversarial Network (TimeGAN) architecture as a data generator. We further evaluated the proposed RLDA framework using a case study of an intelligent HVAC control. With a ≈28% improvement in the sample efficiency, RLDA framework lays the way towards increased adoption of DRL-based intelligent control in real-world building energy management systems.

Kurte, Kuldeep↗