Search NASA⌕ Search

SEARCH · Search NASA

Results for “learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Deep-Learning-Based Koopman Modeling for Online Control Synthesis of Nonlinear Power System Transient Dynamics

Power system stability and control have become more challenging due to the increasing uncertainty associated with renewable generation. Here, the performance of conventional control is highly driven by the physics-based offline-developed dynamic models that can deviate from the actual system characteristics under different operating conditions and/or configurations. Data-driven approaches based on online measurements can be a better solution to addressing these issues by capturing real-time operation conditions. This article describes a novel fully data-driven probabilistic framework to derive a linear representation of postcontingency grid dynamics and online prescribe control based on the derived model to enhance transient stability. The complex nonlinear power system dynamics is approximated by a linear model by using multiple neural network modules that infer distributions of the observations and introducing a Koopman layer to sample possible Koopman linear models from the inferred distributions. The trained model features linearity that can be easily incorporated into the existing linear control design paradigm and ease the controller design process. The effectiveness of Koopman-based control designs is validated through comparative case studies, which demonstrate increased prediction accuracy and control performance when applied to a power system with heterogeneous generator dynamics.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Machine Learning for Slow Spill Regulation in the Fermilab Delivery Ring for Mu2e

A third-integer resonant slow extraction system is being developed for the Fermilab’s Delivery Ring to deliver protons to the Mu2e experiment. During a slow extraction process, the beam on target is liable to experience small intensity variations due to many factors. Owing to the experiment’s strict requirements in the quality of the spill, a Spill Regulation System (SRS) is currently under design. The SRS primarily consists of three components - slow regulation, fast regulation, and harmonic content tracker. In this presentation, we shall present the investigations of using Machine Learning (ML) in the fast regulation system, including further optimizations of PID controller gains for the fast regulation, prospects of an ML agent completely replacing the PID controller using supervised learning schemes such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) ML models, the simulated impact and limitation of machine response characteristics on the effectiveness of both PID and ML regulation of the spill. We also present here nascent results of Reinforcement Learning efforts, including continuous-action soft actor-critic methods, to regulate the spill rate.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Lessons Learned from Industry Engagement in Export Controls

There are several U.S. government-sponsored programs with significant experience engaging with foreign government and industry partners to support capacity-building in export controls. This work seeks to answer the question: How can the outreach experience of the U.S. government-sponsored export control capacity-building programs (ECCBP) inform best practices for engaging with advanced reactor vendors in the domain of international nuclear safeguards? To answer this question, we interviewed export control subject matter experts with experience working for the U.S. ECCBPs – the Bureau of Industry and Security (BIS), the Export Control and Related Border Security (EXBS) program, and the International Nonproliferation Export Control Program (INECP) – and developed a set of recommendations for industry engagement based on the collective experience of interviewees.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS1981

ATC contingency operations in the en-route flight regime

Air traffic control (ATC) operations were examined to learn what factors of controller performance should be given consideration in the design and development of future automation systems enhancing ATC. Contingencies were of two types: those constraining airspace usage or traffic flow (i.e., weather); and those related to system and equipment usage (i.e., radar/radio status). Examination of controller response to contingencies and workload pressures showed differing effects on controller allocations of effort among the three primary function of planning, monitoring, and informaton transfer. Automation advancements oriented towards aiding the controller in performing monitoring tasks may offer the most substantial safety benefit.

Lyman, E. G.↗
NASA NTRS2006

Leadership Challenges in ISS Operations: Lessons Learned from Junior and Senior Mission Control Personnel

The International Space Station (ISS) is operated by a multi-national, multi-organizational team that is dispersed across multiple locations, time zones, and work schedules. At NASA, both junior and senior mission control personnel have had to find ways to address the leadership challenges inherent in such work, but neither have had systematic training in how to do so. The goals of this study were to examine the major leadership challenges faced by ISS mission control personnel and to highlight the approaches that they have found most effective to surmount them. We pay particular attention to the approaches successfully employed by the senior personnel and to the training needs identified by the junior personnel. We also evaluate the extent to which responses are consistent across the junior and senior samples. Further, we compare the issues identified by our interview survey to those identified by a standardized questionnaire survey of mission control personnel and a contrasting group of space station crewmembers. We studied a sample of 14 senior ISS flight controllers and a contrasting sample of 12 more junior ISS controllers. Data were collected using a semi-structured qualitative interview and content analyzed using an iterative process with multiple coders and consensus meetings to resolve discrepancies. To further explore the meaning of the interview findings, we also conducted new analyses of data from a previous questionnaire study of 13 American astronauts, 17 Russian cosmonauts, and 150 U.S. and 36 Russian mission control personnel supporting the ISS or Mir space stations. The interview data showed that the survey respondents had substantial consensus on several leadership challenges and on key strategies for dealing with them, and they offered a wide range of specific tactics for implementing these strategies. Interview data from the junior respondents will be presented for the first time at the meeting. The questionnaire data showed that the US mission control sample reported a level of support from their management that compared favorably to national norms. American mission control personnel and Russian crewmembers reported higher supervisor support than American crewmembers and Russian mission control personnel. We will present the specific issues underlying these findings and compare and contrast the results from the two datasets. Although specific to space station personnel, our results are consistent with recent management, cultural, and aerospace research. We aim to use our results to improve training for current and future mission control personnel.

Clement, James L.↗
NASA NTRSDate not supplied

Human Systems Integration Approach in Implementing Voice-Control of Future Spacecraft Systems

Crewed spacecraft and habitats of the future will require more automation and autonomy to support complex missions. However, these complex systems require a more efficient command and control input method. Speech recognition along with visual or auditory feedback is an alternative, providing an extra pair of hands and eyes for the crew. Yet, speech recognition demands a highly integrated development approach to ensure a successful system implementation. To ensure the voice control application is developed correctly will require a Human Systems Integration (HSI) approach. This paper provides an insight into the development of a speech/voice control application of a spacecraft system that encompasses automation and autonomy through an HSI approach. Results of the voice control experiment of the Space Shuttle camera system are provided as lessons learned about voice control on a spacecraft. Limitations and challenges of the technology are addressed as well as how HSI can help develop these types of voice control command and control systems.

Voice control↗
DOE OSTI2021

Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency

Behind-the-meter distributed energy resources (DERs), including building solar photovoltaic (PV) technology and electric battery storage, are increasingly being considered as solutions to support carbon reduction goals and increase grid reliability and resiliency. However, dynamic control of these resources in concert with traditional building loads, to effect efficiency and demand flexibility, is not yet commonplace in commercial control products. Traditional rule-based control algorithms do not offer integrated closed-loop control to optimize across systems, and most often, PV and battery systems are operated for energy arbitrage and demand charge management, and not for the provision of grid services. More advanced control approaches, such as MPC control have not been widely adopted in industry because they require significant expertise to develop and deploy. Recent advances in deep reinforcement learning (DRL) offer a promising option to optimize the operation of DER systems and building loads with reduced setup effort. However, there are limited studies that evaluate the efficacy of these methods to control multiple building subsystems simultaneously. Additionally, most of the research has been conducted in simulated environments as opposed to real buildings. This paper proposes a DRL approach that uses a deep deterministic policy gradient algorithm for integrated control of HVAC and electric battery storage systems in the presence of on-site PV generation. The DRL algorithm, trained on synthetic data, was deployed in a physical test building and evaluated against a baseline that uses the current best-in-class rule-based control strategies. Performance in delivering energy efficiency, load shift, and load shed was tested using price-based signals. The results showed that the DRL-based controller can produce cost savings of up to 39.6% as compared to the baseline controller, while maintaining similar thermal comfort in the building. The project team has also integrated the simulation components developed during this work as an OpenAIGym environment and made it publicly available so that prospective DRL researchers can leverage this environment to evaluate alternate DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
DOE OSTI2023

Safety-Related Instrumentation and Control Pilot Upgrade: Conceptual - Detailed Design Phase Report and Lessons Learned

Historically, regulatory barriers have precluded nuclear plant safety-related (SR) Instrumentation and Control (I&C) system modernizations. These barriers have been largely addressed by industry and United States Nuclear Regulatory Commission (NRC) efforts, including development of the License Amendment Request (LAR) Alternate Review (AR) process for digital I&C upgrades. The U.S. Department of Energy (DOE) and the Idaho National Laboratory (INL) are supporting a SR I&C pilot upgrade at the Limerick Generating Station (LGS) to demonstrate this process and capturing associated lessons learned for the benefit of industry. In 2021, a paper summarizing lessons learned as captured in INL/EXT-20-59809, Safety Related Instrumentation & Control Pilot Upgrade Initial Scoping Phase Implementation Report and Lessons Learned was presented at the 12th NPIC & HMIT conference. It is recommended that readers of this paper review reference 1 or the associated Nuclear Journal paper for context. This paper summarizes activities performed and lessons learned for the conceptual design phase and portions of the detailed design phase as captured in INL/RPT-23-XXXX, “Safety-Related Instrumentation and Control Pilot Upgrade: Conceptual - Detailed Design Phase Report and Lessons Learned.” The portions of the effort covered by reference 2 include conceptual design phase activities from December 2020, through completion in November 2021, and detailed design activities through September 2022. These efforts and associated detailed documentation in reference 3 are intended to support similar industry upgrades as a foundation stone for a Digital Transformation that will improve plant safety, reliability, and operational performance while lowering plant total cost of ownership.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

pnnl/MBDRL

Model-based Deep Reinforcement Learning for Real-time Grid Emergency Voltage Control. A model-based Deep Reinforcement Learning (DRL) framework where a deep neural network (DNN)-based surrogate model is utilized within the control policy learning framework, making the learning process faster and more sample efficient

Yin, Tim↗