Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Machine Learning Pattern Recognition Algorithm With Applications to Coherent Laser Combination

Herein we analyze a new kind of machine learning algorithm designed to feedback stabilize coherently combined lasers. This algorithm learns differential, rather than absolute, values of action in phase space, in order to facilitate learning on initially unstable systems. Experiments have shown that this approach can control small-scale spatial beam combination with high stability. In this paper we analyze the algorithm's performance and limitations in depth, showing that it can continuously learn during operation in order to track changes. Using simulation, we extend the application to temporal combination, and show that it scales to more complex instances by combining 81 beams.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey

Here, the rapid evolution of modern electric power distribution systems into complex networks of interconnected active devices, distributed generation (DG), and storage poses increasing difficulties for system operators. The large-scale integration of distributed energy resources (DERs) and the rapid exchange of measurement data via communication networks present major opportunities for advancing grid operations but also introduce greater uncertainty, higher data dimensionality, more complex network and device models, and challenging control and optimization problems. Deep reinforcement learning (DRL) algorithms are promising in addressing these challenges. However, they have not been effectively adapted for power systems applications, requiring extensive customization for implementation and evaluation. This has resulted in reproducibility challenges and a steep learning curve for researchers new to applying DRL algorithms to the power systems domain. To bridge these gaps, this tutorial aims to serve as a valuable resource for researchers interested in exploring learning-based algorithms to operate active power distribution networks. Specifically, this work presents a generalized process for translating sequential decision-making problems in power distribution systems into Markov decision process (MDP) formulations, illustrated through concrete grid service examples. Additionally, we introduce a simple environment design strategy to develop and evaluate example DRL algorithms for distribution system applications, complete with an included code repository to guide users through environment construction.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Toward particle accelerator machine state embeddings as a modality for large language models

Understanding and diagnosing the state of a particle accelerator requires navigating high-dimensional control system data, often involving hundreds of interdependent parameters. We propose a novel multimodal embedding framework that jointly learns representations of machine states from both numerical control system readouts and natural language descriptions. This enables the translation of complex machine conditions into human-readable summaries while maintaining fidelity to the underlying physical system. The obtained embeddings are subsequently adapted to an open-weights large language model via cross-attention conditioning. We demonstrate a first implementation trained on European XFEL machine state data. This work covers the embedding model architecture, training methodology, and presents initial examples demonstrating the model's capabilities in action. Due to the general concept of machine state, the model can be easily adapted to other facilities and control system environments.

Accelerator Physics↗
DOE OSTI2021

Identifying Key Drivers of Wildfires in the Contiguous US Using Machine Learning and Game Theory Interpretation

Abstract Understanding the complex interrelationships between wildfire and its environmental and anthropogenic controls is crucial for wildfire modeling and management. Although machine learning (ML) models have yielded significant improvements in wildfire predictions, their limited interpretability has been an obstacle for their use in advancing understanding of wildfires. This study builds an ML model incorporating predictors of local meteorology, land‐surface characteristics, and socioeconomic variables to predict monthly burned area at grid cells of 0.25° × 0.25° resolution over the contiguous United States. Besides these predictors, we construct and include predictors representing the large‐scale circulation patterns conducive to wildfires, which largely improves the temporal correlations in several regions by 14%–44%. The Shapley additive explanation is introduced to quantify the contributions of the predictors to burned area. Results show a key role of longitude and latitude in delineating fire regimes with different temporal patterns of burned area. The model captures the physical relationship between burned area and vapor pressure deficit, relative humidity (RH), and energy release component (ERC), in agreement with the prior findings. Aggregating the contribution of predictor variables of all the grids by region, analyses show that ERC is the major contributor accounting for 14%–27% to large burned areas in the western US. In contrast, there is no leading factor contributing to large burned areas in the eastern US, although large‐scale circulation patterns featuring less active upper‐level ridge‐trough and low RH two months earlier in winter contribute relatively more to large burned areas in spring in the southeastern US.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2018

Stirling Convertor Controller Development at NASA Glenn Research Center

Over the past decade, NASA Glenn Research Center (GRC) has been supporting the development of Radioisotope Power Systems (RPS). NASA desires higher conversion efficiency RPS options that are reliable and robust with long life design. Dynamic conversion, such as Stirling and Brayton, offer the potential for higher conversion efficiencies but have yet to be demonstrated in a flight application. The RPS program sent out a solicitation to investigate options for dynamic conversion technologies. As a result of this solicitation, four dynamic power convertor (DPC) technologies were selected for design and fabrication of a prototype dynamic convertor. One lesson learned from the Advanced Stirling Radioisotope Generator (ASRG) project is that controller development should start early in the development of a dynamic convertor. As a result of this, NASA GRC has been utilizing hardware from past Stirling convertor projects including that of the ASRG to support controller development for the DPC's. NASA GRC has developed a strong knowledge base on both analog and digital Stirling dynamic power convertor controllers and will continue to expand and apply that knowledge to the four DPC's. Over the past 15 years, controllers were developed in-house at GRC, at Lockheed Martin Coherent Technologies (LMCT) and by the Johns Hopkins University/Applied Physics Laboratory (JHU/APL). Various generations of the controllers, have been developed as lessons were learned through various component and system level tests. Some of the tests performed were fault tolerance, qualification vibration level, electromagnetic interference, Radioisotope Power System Systems Integration (RSIL) tests, and extended operation. The fault tolerance test characterized the controller's ability to handle various fault conditions, including high or low bus power consumptions, total open load or short circuit, and replacing a failed controller card while the backup maintains control of the ASC. The vibration test confirms the controller's ability to control an ASC during launch. The EMI test characterized the AC and DC magnetic and electric fields emitted by the single ASC and if the controller has an impact on the radiated EMI. RSIL testing provided insight into the electrical interactions between the representative RPS, its associated control schemes, and realistic electric system loads. The extended operation test allows data to be collected over a period of thousands of hours to obtain long term performance data of the system. This paper describes the history of controller development at NASA GRC, tests performed on these controllers, and lessons learned.

Dugala, Gina M.↗
NASA NTRS2018

Stirling Convertor Controller Development at NASA Glenn Research Center

For nearly two decades, NASA Glenn Research Center (GRC) has been supporting the development of Radioisotope Power Systems (RPS). NASA desires higher conversion efficiency RPS options that are reliable and robust with long life design. Dynamic conversion, such as Stirling and Brayton, offer the potential for higher conversion efficiencies than current RPS but have yet to be demonstrated in a flight application. The RPS program sent out a solicitation to investigate options for dynamic conversion technologies. As a result of this solicitation, four dynamic power convertor (DPC) technologies were selected for design and three are proceeding to the fabrication phase of prototype dynamic convertors. One lesson learned from the Advanced Stirling Radioisotope Generator (ASRG) project is that controller development should be coordinated with the development of a dynamic convertor. As a result of this, NASA GRC has been utilizing hardware from past Stirling convertor projects including that of the ASRG to support controller development for the DPC's. NASA GRC has developed a strong knowledge base on both analog and digital Stirling dynamic power convertor controllers and will continue to expand and apply that knowledge to the DPC's. Over the past 15 years, controllers were developed at GRC, at Lockheed Martin (LM) and by the Johns Hopkins University/Applied Physics Laboratory (JHU/APL). Various generations of the controllers have been developed as lessons were learned through various component and system level tests. Some of the tests performed were fault tolerance, flight acceptance vibration, electromagnetic interference (EMI), spacecraft integration, and extended operation. The fault tolerance test characterized the controller's ability to handle various fault conditions, including high or low bus power consumptions, total open load or short circuit, and replacing a failed controller card while the backup maintains control of the Stirling convertor. The vibration test confirms the controller's ability to control an ASC during launch. The EMI test characterized the AC and DC magnetic and electric fields emitted by the single ASC and if the controller has an impact on the radiated EMI. Spacecraft integration testing in the Radioisotope Power Systems System Integration Laboratory (RSIL) provided insight into the electrical interactions between the representative RPS, its associated control schemes, and realistic electric system loads. The extended operation test allows data to be collected over a period of thousands of hours to obtain long term performance data of the system. This paper describes the history of controller development at NASA GRC, tests performed on these controllers, and lessons learned.

Dugala, Gina M.↗
NASA NTRS2018

Stirling Convertor Controller Development at NASA Glenn Research Center

For nearly two decades, NASA Glenn Research Center has been supporting the development of radioisotope power systems (RPS). NASA desires higher conversion efficiency RPS options that are reliable and robust with long-life design. Dynamic conversion, such as Stirling and Brayton, offer the potential for higher conversion efficiencies than current RPS but have yet to be demonstrated in a flight application. The RPS program sent out a solicitation to investigate options for dynamic conversion technologies. As a result of this solicitation, four dynamic power convertor (DPC) technologies were selected for design and three are proceeding to the fabrication phase of prototype dynamic convertors. One lesson learned from the Advanced Stirling Radioisotope Generator (ASRG) project is that controller development should be coordinated with the development of a dynamic convertor. As a result of this, Glenn has been utilizing hardware from past Stirling convertor projects, including that of the ASRG, to support controller development for the DPCs. Glenn has developed a strong knowledge base on both analog and digital Stirling DPC controllers and will continue to expand and apply that knowledge to the DPCs. Over the past 15 years, controllers were developed at Glenn, at Lockheed Martin (LM), and by the Johns Hopkins University Applied Physics Laboratory (APL). Various generations of the controllers have been developed as lessons were learned through various component- and system-level tests. Some of the tests performed were fault tolerance, flight acceptance vibration, electromagnetic interference (EMI), spacecraft integration, and extended operation. The fault tolerance test characterized the controller’s ability to handle various fault conditions, including high or low bus power consumption, total open load or short circuit, and replacing a failed controller card while the backup maintains control of the Stirling convertor. The vibration test confirms the controller’s ability to control an Advanced Stirling Convertor (ASC) during launch. The EMI test characterized the alternating-current (AC) and direct-current (DC) magnetic and electric fields emitted by the single ASC and if the controller has an impact on the radiated EMI. Spacecraft integration testing in the Radioisotope Power Systems (RPS), System Integration Laboratory (RSIL) provided insight into the electrical interactions between the representative RPS, its associated control schemes, and realistic electric system loads. The extended operation test allows data to be collected over a period of thousands of hours to obtain long-term performance data of the system. This paper describes the history of controller development at Glenn, tests performed on these controllers, and lessons learned.

Dugala, Gina M.↗
DOE OSTI2019

Detecting control system misbehavior by fingerprinting programmable logic controller functionality

In recent years, attacks such as the Stuxnet malware have demonstrated that cyberattacks against control systems cause extensive damage. These attacks can result in physical damage to the networked systems under their control. In this paper, we discuss our approach for detecting such attacks by distinguishing between programs running on a programmable logic controller (PLC) without having to monitor communications. Using power signatures generated by an attached, high-frequency power measurement device, we can identify what a PLC is doing and when an attack may have altered what the PLC should be doing. To accomplish this, we generated labeled data for testing our methods and applied feature engineering techniques and machine learning models. The results of this research demonstrate that Random Forests and Convolutional Neural Networks classify programs with up to 98% accuracy for major program differences and 84% accuracy for minor differences. Our results can be used for both online and offline applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Hamiltonian switching control of noisy bipartite qubit systems

Abstract We develop a Hamiltonian switching ansatz for bipartite control that is inspired by the quantum approximate optimization algorithm, to mitigate environmental noise on qubits. We demonstrate the control for a central spin coupled to bath spins via isotropic Heisenberg interactions, and then make physical applications to the protection of quantum gates performed on superconducting transmon qubits coupling to environmental two-level-systems (TLSs) through dipole-dipole interactions, as well as on such qubits coupled to both TLSs and a Lindblad bath. The control field is classical and acts only on the system qubits. We use reinforcement learning with policy gradient to optimize the Hamiltonian switching control protocols, using a fidelity objective for specific target quantum gates. We use this approach to demonstrate effective suppression of both coherent and dissipative noise, with numerical studies achieving target gate implementations with fidelities over 0.9999 (four nines) in the majority of our test cases and showing improvement beyond this to values of 0.999 999 999 (nine nines) upon a subsequent optimization by GRadient Ascent Pulse Engineering (GRAPE). We analyze how the control depth, total evolution time, number of environmental TLS, and choice of optimization method affect the fidelity achieved by the optimal protocols and reveal some critical behaviors of bipartite control of quantum gates.

Physics↗
DOE OSTI2020

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Fully Convolutional Spatio-Temporal Models for Representation Learning in Plasma Science

We have trained a fully convolutional spatio-temporal model for fast and accurate representation learning in the challenging exemplar application area of fusion energy plasma science. The onset of major disruptions is a critically important fusion energy science issue that must be resolved for advanced tokamak plasmas such as the $25B burning plasma international thermonuclear experimental reactor (ITER) experiment. While a variety of statistical methods have been used to address the problem of tokamak disruption prediction and control, recent approaches based on deep learning have proven particularly compelling. In the present paper, we introduce further improvements to the fusion recurrent neural network (FRNN) software suite, which delivered cross-machine disruption predictions with unprecedented accuracy using a large database of experimental signals from two major tokamaks. Up to now, FRNN was based on the long short-term memory (LSTM) variant of recurrent neural networks to leverage the temporal information in the data. Here, we implement and apply the "temporal convolutional neural network (TCN)" architecture to the time-dependent input signals. Furthermore, this allows highly optimized convolution operations to carry the majority of the computational load of training, thus enabling a reduction in training time, and the effective use of high-performance computing resources for hyperparameter tuning. At the same time, the TCN-based architecture achieves better predictive performance when compared with the LSTM architecture for various tasks for a representative fusion database.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1987

Automated Subsystem Control for Life Support System (ASCLSS)

The Automated Subsystem Control for Life Support Systems (ASCLSS) program has successfully developed and demonstrated a generic approach to the automation and control of space station subsystems. The automation system features a hierarchical and distributed real-time control architecture which places maximum controls authority at the lowest or process control level which enhances system autonomy. The ASCLSS demonstration system pioneered many automation and control concepts currently being considered in the space station data management system (DMS). Heavy emphasis is placed on controls hardware and software commonality implemented in accepted standards. The approach demonstrates successfully the application of real-time process and accountability with the subsystem or process developer. The ASCLSS system completely automates a space station subsystem (air revitalization group of the ASCLSS) which moves the crew/operator into a role of supervisory control authority. The ASCLSS program developed over 50 lessons learned which will aide future space station developers in the area of automation and controls..

Block, Roger F.↗
NASA NTRS2021

Predicting Arrival and Departure Runway Assignments with Machine Learning

Runway assignments at major airports are made by air traffic controllers subject to various constraints, and to achieve various objectives. In this research, we describe our efforts training machine learning (ML) models to predict both departure and arrival runway assignments using an entirely data-driven approach. This approach is compared to existing rule-based approaches developed in previous research using input from Subject Matter Experts. The models have features derived from various FAA data feeds, and leverage multiple machine learning algorithms. Results for models trained for nine major U.S. airports are described and compared to one another across various important dimensions. Particular attention was paid to developing a repeatable framework for training these models so the approach could be scaled to other airports, and to developing models that are useful in a real-time environment. In addition, the models were designed to be functional in a real-time environment to support NASA’s ATD-2 project, as part of an ML-powered shadow system to compare against the performance of the fielded system.

machine learning↗
NASA NTRS2021

Predicting Arrival and Departure Runway Assignments with Machine Learning

Runway assignments at major airports are made by air traffic controllers subject to various constraints, and to achieve various objectives. In this research, we describe our efforts training machine learning (ML) models to predict both departure and arrival runway assignments using an entirely data-driven approach. This approach is compared to existing rule-based approaches developed in previous research using input from Subject Matter Experts. The models have features derived from various FAA data feeds, and leverage multiple machine learning algorithms. Results for models trained for nine major U.S. airports are described and compared to one another across various important dimensions. Particular attention was paid to developing a repeatable framework for training these models so the approach could be scaled to other airports, and to developing models that are useful in a real-time environment. In addition, the models were designed to be functional in a real-time environment to support NASA’s ATD-2 project, as part of an ML-powered shadow system to compare against the performance of the fielded system.

machine learning↗
DOE OSTI2021

Restoring Distribution System Under Renewable Uncertainty Using Reinforcement Learning

Distributed energy resources (DER) in distribution systems, including renewable generation, micro-turbine, and energy storage, can be used to restore critical loads following extreme events to increase grid resiliency. However, properly coordinating multiple DERs in the system for multi-step restoration process under renewable uncertainty and fuel availability is a complicated sequential optimal control problem. Due to its capability to handle system non-linearity and uncertainty, reinforcement learning (RL) stands out as a potentially powerful candidate in solving complex sequential control problems. Moreover, the offline training of RL provides excellent action readiness during online operation, making it suitable to problems such as load restoration, where in-time, correct and coordinated actions are needed. In this study, a distribution system prioritized load restoration based on a simplified single-bus system is studied: with imperfect renewable generation forecast, the performance of an RL controller is compared with that of a deterministic model predictive control (MPC). Our experiment results show that the RL controller is able to learn from experience, adapt to the imperfect forecast information and provide a more reliable restoration process when compared with the baseline MPC controller.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2020

Restoring Distribution System Under Renewable Uncertainty Using Reinforcement Learning

Distributed energy resources (DERs) in distribution systems, including renewable generation, micro-turbine, and energy storage, can be used to restore critical loads following extreme events to increase grid resiliency. However, properly coordinating multiple DERs in the system for multi-step restoration process under renewable uncertainty and fuel availability is a complicated sequential optimal control problem. Due to its capability to handle system non-linearity and uncertainty, reinforcement learning (RL) stands out as a potentially powerful candidate in solving complex sequential control problems. Moreover, the offline training of RL provides excellent action readiness during online operation, making it suitable to problems such as load restoration, where in-time, correct and coordinated actions are needed. In this study, a distribution system prioritized load restoration based on a simplified single-bus system is studied: with imperfect renewable generation forecast, the performance of an RL controller is compared with that of a deterministic model predictive control (MPC). Our experiment results show that the RL controller is able to learn from experience, adapt to the imperfect forecast information and provide a more reliable restoration process when compared with the baseline controller.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2020

Restoring Distribution System Under Renewable Uncertainty Using Reinforcement Learning: Preprint

Distributed energy resources (DER) in distribution systems, including renewable generation, micro-turbine, and energy storage, can be used to restore critical loads following extreme events to increase grid resiliency. However, properly coordinating multiple DERs in the system for multi-step restoration process under renewable uncertainty and fuel availability is a complicated sequential optimal control problem. Due to its capability to handle system non-linearity and uncertainty, reinforcement learning (RL) stands out as a potentially powerful candidate in solving complex sequential control problems. Moreover, the offline training of RL provides excellent action readiness during online operation, making it suitable to problems such as load restoration, where in-time, correct and coordinated actions are needed. In this study, a distribution system prioritized load restoration based on a simplified single-bus system is studied: with imperfect renewable generation forecast, the performance of an RL controller is compared with that of a deterministic model predictive control (MPC). Our experiment results show that the RL controller is able to learn from experience, adapt to the imperfect forecast information and provide a more reliable restoration process when compared with the baseline MPC controller.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2024

Adaptive control of wave energy converters

A wave energy capture system deployed in water converts mechanical motion induced by waves in the water to electrical energy. A controller of the wave energy capture system receives input regarding real-time wave conditions in a vicinity of the wave energy capture system. The controller applies a control model to the received input to select a value of a control parameter for the wave energy capture system, where the control model includes a model that has been trained using machine learning to take wave condition data as input and to output control parameter values selected based on the wave condition data in order to increase an amount of energy captured by the wave energy capture system. The controller implements the selected value of the control parameter on the wave energy capture system.

Orona, Alexander J.↗