Search NASA⌕ Search

SEARCH · Search NASA

Results for “Operator learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Deep Reinforcement Learning for Microgrid Cost Optimization Considering Load Flexibility

This paper proposes a novel Soft-Actor-Critic (SAC) based Deep Reinforcement Learning (DRL) method for optimizing the cost of microgrid operation by leveraging load flexibility. The proposed SAC-DRL method is designed to coordinate the control of distributed energy resources (DERs) and flexible load, addressing practical energy billing formation by power distribution utilities. Key contributions include an innovative reward function to mitigate sparse reward challenges and a mixed control strategy for discrete and continuous variables, ensuring radial network topology and minimizing power loss. We evaluate the proposed method on the model of a real microgrid located in Southern California, U.S.. The SAC-DRL model is tested to demonstrate its efficacy in reducing grid dependence, optimizing resource use, and minimizing costs. The results highlight the potential of DRL in modern energy systems, offering a sustainable and economically efficient solution for energy management in microgrids.

deep reinforcement learning↗
DOE OSTI2021

Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search

In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to nonlinear building models. We illustrate the controller performance numerically using a five-zone commercial building.

30 DIRECT ENERGY CONVERSION↗
NASA NTRS2005

Operational behavioral health and performance resources for international space station crews and families

The Behavioral Health and Performance Section (BHP) at NASA Johnson Space Center provides direct and indirect psychological services to the International Space Station (ISS) astronauts and their families. Beginning with the NASA-Mir Program, services available to the crews and families have gradually expanded as experience is gained in long-duration flight. Enhancements to the overall BHP program have been shaped by crewmembers' personal preferences, family requests, specific events during the missions, programmatic requirements, and other lessons learned. The BHP program focuses its work on four areas: operational psychology, behavioral medicine, human-to-system interface, and sleep and circadian. Within these areas of focus are psychological and psychiatric screening for astronaut selection as well as many resources that are available to the crewmembers, families, and other groups such as crew surgeon and various levels of management within NASA. Services include: preflight, in flight, and postflight preparation; training and support; resources from a Family Support Office; in-flight monitoring; clinical care for astronauts and their families; and expertise in the workload and work/rest scheduling of crews on the ISS. Each of the four operational areas is summarized, as are future directions for the BHP program.

Space Flight↗
NASA NTRS2022

Revisiting the Ground Magnetic Field Perturbations Challenge: A Machine Learning Perspective

Forecasting ground magnetic field perturbations has been a long-standing goal of the space weather community. The availability of ground magnetic field data and its potential to be used in geomagnetically induced current studies, such as risk assessment, have resulted in several forecasting efforts over the past few decades. One particular community effort was the Geospace Environment Modeling (GEM) challenge of ground magnetic field perturbations that evaluated the predictive capacity of several empirical and first principles models at both mid- and high-latitudes in order to choose an operative model. In this work, we use three different deep learning models-a feed-forward neural network, a long short-term memory recurrent network and a convolutional neural network-to forecast the horizontal component of the ground magnetic field rate of change (dB H /dt) over 6 different ground magnetometer stations and to compare as directly as possible with the original GEM challenge. We find that, in general, the models are able to perform at similar levels to those obtained in the original challenge, although the performance depends heavily on the particular storm being evaluated. We then discuss the limitations of such a comparison on the basis that the original challenge was not designed with machine learning algorithms in mind.

Victor A. Pinto↗
DOE OSTI2020

Attack-resilient algorithms and testbed federation for wide-area protection and control in smart grid

Today’s energy infrastructure is undergoing a massive transformation across all generation, transmission, and distribution systems to provide reliability, efficiency, and sustainability to the power system network. During the past 10 years, there has been a remarkable enhancement in developing close-loop wide-area protection and control (WAPAC) applications, such as automatic generation control (AGC), wide-area protection scheme (WAPS), synchrophasor-based wide-area voltage control system (WAVCS), etc., in the energy management system (EMS) to provide an adequate situational awareness to control center operators during physical disturbances and natural events. The rapid digitalization, unencrypted communication, and numerous data-sharing devices have increased digital access points that have exposed the grid network to countless cyberattacks pertinent to the WAPAC cybersecurity. Given the amount of complex grid network coupled with legacy infrastructures, it is untrustworthy to completely rely on information technology (IT)-based conventional security measures to defend against evolving cyberattacks. Hence, there is a compelling need to go beyond the traditional paradigm of ‘security by obscurity’ and ‘bolt-on’ security measures, and develop a suite of innovative security solutions at multiple layers to transform the current “fault-resilient” grid into “fault and attack-resilient” grid of the future. This dissertation specifically focuses on developing application-specific attack-resilient solutions for critical WAPAC applications in the smart grid. The first part of the study, Attack-Resilient Wide-Area Protection Scheme, presents a cyber-physical attack-resilient system (CPARS) for WAPS, where two anomaly detection methods are proposed to detect multi-level cyberattacks followed by the rules-based intrusion mitigation system (RIMS) to initiate effective mitigation strategies, tailored to these attacks, to restore the normal grid operation after disturbances. The first anomaly detection component, Machine learning-based Anomaly Detection for Centralized WAPS, discusses a novel architecture and methodology for developing a machine-learning-based anomaly detection system (ADS) for the centralized wide-area protection system (CWAPS) by applying the variational mode decomposition (VMD) technique and decision tree (DT) algorithms using cyber logs and synchrophasor measurements. The second component, Multi-Agent-based Anomaly Detection for Decentralized WAPS, presents a two-level hierarchical multi-agent-based decentralized WAPS against the system-aware stealthy cyber-attacks. Finally, for the RIMS, the state transition diagram-based mitigation measures are defined based on the grid operation states and experimental testing and evaluation are presented in a cyber-physical testbed environment. The second part of the study, Attack-Resilient Wide-Area Voltage Control System, addresses the existing vulnerabilities in the wide-area voltage control system (WAVCS), presents the methodology for performing an impact analysis during data integrity attacks, and proposes a data-driven attackresilient system (DARS) by incorporating machine learning-based ADS and rules-based anomaly mitigation system (RAMS) to detect these attacks and provide necessary mitigation actions to maintain the transient voltage stability after disturbances. Finally, the third part, Networked Federation Testbed (NEFTSec) for Smart Grid Cybersecurity, focuses on developing an interconnected federated testbed through a common network like the internet, describes its conceptual architectures, and also presents its design components for experimental testing, validation, and evaluation. Further, it presents the co-simulation interface algorithm (CIA) to facilitate the geographically-dispersed real-time distributed simulation (GD-RTDS) during the cyber-physical federation. In general, this dissertation presents innovative attack-resilient solutions for WAPAC applications that are experimentally tested and evaluated using the PowerCyber testbed and NEFTSec platform to accelerate the transition of state-of-the-art research works to the real-field deployments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Graph Analytics for CEBAF Operations

We report on the progress achieved during a 2-year Laboratory Directed Research and Development (LDRD) project titled “Graph Analytics for CEBAF Operations”. The objective of this project is to leverage deep learning on graph representations of CEBAF’s injector beamline in order to create a tool for improving the efficiency of beam tuning tasks. Specifically, we use graphs to represent the injector beamline at any arbitrary date and time and invoke a graph neural network (GNN) to extract a low-dimensional, informative representation that can be visualized in two-dimensions. By analyzing years of operational data from the CEBAF archiver, good and bad regions of parameter space can be identified. The goal is to exercise this framework as a real-time tool to aid beam tuning, which represents the dominant source of machine downtime.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search: Preprint

In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to non-linear building models. We illustrate the controller performance numerically using a five-zone commercial building.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2025

Prediction of Silicon Content in a Blast Furnace via Machine Learning: A Comprehensive Processing and Modeling Pipeline

Silicon content plays an important role in determining the operational efficiency of blast furnaces (BFs) and their downstream processes in integrated steelmaking; however, existing sampling methods and first-principles models are somewhat limited in their capability and flexibility. Current data-based prediction models primarily rely on a limited set of manually selected furnace parameters. Additionally, different BFs present a diverse set of operating parameters and state variables that are known to directly influence the hot metal’s silicon content, such as fuel injection, blast temperature, and raw material charge composition, among other process variables that have their own impacts. The expansiveness of the parameter set adds complexity to parameter selection and processing. This highlights the need for a comprehensive methodology to integrate and select from all relevant parameters for accurate silicon content prediction. Providing accurate silicon content predictions would enable operators to adjust furnace conditions dynamically, improving safety and reducing economic risk. To address these issues, a two-stage approach is proposed. First, a generalized data processing scheme is proposed to accommodate diverse furnace parameters. Second, a robust modeling pipeline is used to establish a machine learning (ML) model capable of predicting hot metal silicon content with reasonable accuracy. The method employed herein predicted the average Si content of the upcoming furnace cast with an accuracy of 91% among 200 target predictions for a specific furnace provisioned by the XGBoost model. This prediction is achieved using only the past shift’s operating conditions, which should be available in real time. This performance provides a strong baseline for the modeling approach with potential for further improvement through provision of real-time features.

Chemistry↗
NASA NTRS2020

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation↗
NASA NTRS1996

Early Warning: Development of Confidential Incident Reporting Systems

Accidents hardly ever happen without warning. The combination, or sequence, of failures and mistakes that cause an accident may indeed be unique but the individual failures and mistakes rarely are. In the USA in 1974 the crews on two different aircraft misunderstood the same aeronautical chart and descended towards their destination dangerously early towards a mountain. The first crew were in good weather conditions and could see the mountain and resolved their misinterpretation of the chart. The second crew six weeks later were not so lucky. In cloud they had no clues to point out their mistake nor the presence of the mountain. The resulting crash and the ensuing inquiry, which brought to light the previous incident, shocked the country but gave it the impetus to instigate a safety reporting system. This system eventually became the NASA's Aviation Safety Reporting System (ASRS). The programme collects incident reports from pilots, controllers, mechanics, cabin attendants and many others involved in aviation operations. By disseminating this safety information the ASRS has helped enormously to give US airlines and airspace the highest safety standards. Accident prevention is a goal sought by everyone in the aviation industry and establishing effective incident reporting programmes can go a long way toward achieving that goal. This article will describe the steps and issues required to establish an incident reporting system. The authors summarize the lessons learned from the ASRS, now in its twentieth year of operation and from the Confidential Human Factors Reporting (HER) Programme run by British Airways, an airline that is a recognized world leader in safety reporting and analysis. The differences between government and airline operation of confidential safety reporting systems will be addressed.

OLeary, Mike J.↗
NASA NTRSDate not supplied

An Intern Experience with NASA’s Artemis Program

“3...2...1...Go For Launch”. Nasa’s Kennedy Space Center is projected to launch the first mission of theArtemis Program by the end of this year, 2021. I have had the opportunity to contribute directly to this mission through my internship with their Engineering organization from January to August of 2021.Within their Engineering organization, I am an intern for the Avionics branch. Throughout this experience I have completed two projects with one project still in progress. These projects allowed me to work directly with the Orion vehicle’s software and hardware power ups, checkouts, and tests. I became familiarized with the engineering concepts that allowed this advanced technology to perform as intended.In addition, I became acquainted with the operations and tasks that are completed during a LaunchCountdown, both at a detailed level as well as a higher managerial level. Lastly, I worked directly with the software algorithm that detects and responds to onboard faults and failures. I have analyzed and made conclusions based on data and documentation, allowing my team to benefit from a condensed, centralized informational file. In addition to my technical experience, I have learned a significant amount about how the Kennedy Space Center operates, it’s history, and the history of the agency. The education I received from the Engineering department at Texas A&M University not only allowed me to obtain this internship, but provided me with the required background in computer and electrical engineering principles to be successful in this organization.

Sarah Bianco↗
DOE OSTI2023

A Transductive Graph Neural Network learning for Grid Resilience Analysis

Power grids are critical infrastructures that require robust resilience analysis to ensure reliable and uninterrupted electricity supply. Traditional simulation-based methods for grid resilience analysis suffer from computational complexity and limited ability to capture the full spectrum of potential disruptions. This paper presents a novel approach to enhance grid resilience by leveraging transductive graph neural network (GNN) learning to identify critical nodes and links. By leveraging the graph structure and system features, GNNs effectively learn resilience metrics and accurately identify critical nodes based on actual grid operational behavior. The efficacy of the proposed approach is demonstrated through case studies on node criticality scoring and critical node/line identification in cascading outage scenarios. The results highlight the advantages of learning-based methods over traditional simulation-based approaches and their potential to revolutionize grid resilience analysis. The contributions of this paper include a graph-based scalable approach for fast cascading analysis, an inductive formulation for training GNN models, and a transfer learning-based approach to scale the model to largescale power systems.

grid resilience, graph neural networks, transducti↗
NASA NTRS1975

Study to evaluate the effect of EVA on payload systems. Volume 1: Executive summary

Programmatic benefits to payloads are examined which can result from the routine use of extravehicular activity (EVA) during space missions. Design and operations costs were compared for 13 representative baseline payloads to the costs of those payloads adapted for EVA operations. The EVA-oriented concepts developed in the study were derived from these baseline concepts and maintained mission and program objectives as well as basic configurations. This permitted isolation of cost saving factors associated specifically with incorporation of EVA in a variety of payload designs and operations. The study results were extrapolated to a total of 74 payload programs. Using appropriate complexity and learning factors, net EVA savings were extrapolated to over $551M for NASA and U.S. civil payloads for routine operations. Adding DOD and ESRO payloads increases the net estimated savings of $776M. Planned maintenance by EVA indicated an estimated $168M savings due to elimination of automated service equipment. Contingency problems of payloads were also analyzed to establish expected failure rates for shuttle payloads. The failure information resulted in an estimated potential for EVA savings of $1.9 B.

Patrick, J. W.↗
DOE OSTI2021

A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network

Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2023

Science Objectives for Human Exploration of Mars Workshop Report

The Science Objectives for Human Exploration of Mars Workshop was held in Denver, Colorado on May 4–6, 2022. The workshop was co-sponsored by NASA’s Science Mission Directorate and the Exploration Systems Development Mission Directorate to actively engage the planetary science community to determine what planetary science should be done by human crews on the martian surface and how those science objectives can be achieved. Sessions at the Science Objectives for Human Exploration of Mars Workshop were organized around specific planetary science objectives and mission architecture concepts that were identified during the workshop as the highest priority for human exploration. The intent of this workshop was to synthesize a notional, integrated concept of operations for each scenario to aid in planning and refining the mission architecture for the first human mission to Mars. Results from the Planetary Decadal Survey Report were released xx days before the workshop and were briefed to workshop participants who incorporated the findings in the discussions. With the Artemis missions, humans will return to the Moon using innovative technologies to explore the lunar surface. We will apply what we learn about exploration architecture, surface infrastructure, and science and exploration operations on and around the Moon to conduct successful missions with the first astronauts to Mars. A human mission to Mars will be a landmark achievement and a golden opportunity to make groundbreaking scientific discoveries on Mars. The potential scope of the science activities and impact of achieving those objectives are extraordinary.

Mars↗
DOE OSTI2022

RADAMS: Resilient and adaptive alert and attention management strategy against Informational Denial-of-Service (IDoS) attacks

Attacks exploiting human attentional vulnerability have posed severe threats to cybersecurity. In this work, we identify and formally define a new type of proactive attentional attacks called Informational Denial-of-Service (IDoS) attacks that generate a large volume of feint attacks to overload human operators and hide real attacks among feints. Here, we incorporate human factors (e.g., levels of expertise, stress, and efficiency) and empirical psychological results (e.g., the Yerkes-Dodson law and the sunk cost fallacy) to model the operators’ attention dynamics and their decision-making processes along with the real-time alert monitoring and inspection. To assist human operators in dismissing the feints and escalating the real attacks timely and accurately, we develop a Resilient and Adaptive Data-driven alert and Attention Management Strategy (RADAMS) that de-emphasizes alerts selectively based on the abstracted category labels of the alerts. RADAMS uses reinforcement learning to achieve a customized and transferable design for various human operators and evolving IDoS attacks. The integrated modeling and theoretical analysis lead to the Product Principle of Attention (PPoA), fundamental limits, and the tradeoff among crucial human and economic factors. Experimental results corroborate that the proposed strategy outperforms the default strategy and can reduce the IDoS risk by as much as 20%. Besides, the strategy is resilient to large variations of costs, attack frequencies, and human attention capacities. We have recognized interesting phenomena such as attentional risk equivalency, attacker’s dilemma, and the half-truth optimal attack strategy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data-Enabled Fusion Technology (Final Scientific/Technical Report)

Advancing Scientific Understanding in Fusion Energy and Machine Learning This research represented a significant step forward in machine learning (ML) applications for fusion energy experiments. The project integrated advanced data-driven modeling, optimization techniques, and artificial intelligence to enhance the predictive capabilities and operational efficiency of plasma-based fusion systems. Specifically, tasks focused on ML-enhanced diagnostics, operator guidance tools, and predictive modeling helped improve the ability to interpret complex fusion experiments. Key areas of advancement included: 1) data-driven plasma control, i.e., using ML algorithms to optimize experimental conditions and classify plasma behaviors based on historical data; 2) spectroscopy and diagnostics, i.e., applying AI models to extract previously inaccessible insights from experimental spectroscopy data; and 3) configuration mapping and operator guidance, i.e., developing a predictive framework to assist scientists in identifying the most effective experimental parameters, reducing reliance on manual adjustments. By refining these ML-driven techniques, the project contributed to the broader scientific community’s understanding of plasma dynamics and fusion energy viability. Technical Effectiveness and Economic Feasibility The methods investigated demonstrated high technical effectiveness, as reflected in milestones assessing the predictive accuracy, performance, and optimization of fusion configurations. The development of an Operator Guidance Tool (OGT), for example, led to more precise control of plasma conditions by learning from experimental data and offering real-time adjustments. From an economic standpoint, DeFT provided: 1) the ability to reduce trial-and-error experimentation, which lowered operational costs; 2) improved data interpretation methods, which enabled more efficient resource allocation in large-scale fusion research projects; and 3) the automation of key diagnostic tasks, which reduced manual labor and human error, increasing overall efficiency. 13 The final assessments of predictive models and optimization strategies demonstrated that these approaches were scalable and could be implemented across multiple fusion energy research programs. Public Benefit and Societal Impact This project contributed directly to the broader goal of achieving sustainable and commercially viable fusion energy, which had profound implications for clean energy production and climate change mitigation. The integration of AI-driven solutions into fusion research: 1) sped up scientific discovery, accelerating progress towards achieving energy breakthroughs; 2) reduced the cost of experimentation, making fusion research more accessible; and 3) provided a framework for future AI applications in high-energy physics, benefiting adjacent fields like space exploration, material science, and renewable energy. Additionally, by fostering collaborations between AI researchers and plasma physicists, this project promoted interdisciplinary innovation that could lead to broader applications beyond fusion research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Coordinated Modeling of Electric Grid and Natural Gas Network Operations

Presentation based upon full report of a Colorado case study and coordination framework, which is available at https://www.nrel.gov/docs/fy20osti/77096.pdf. JISEA analysts Brian Sergi, Omar Guerra, and Bri-Mathias Hodge will present on recent JISEA work on coordination between the natural gas and electricity sectors. Power and gas are becoming increasingly interdependent but weren't designed to function together. How does greater coordination impact system operations with different levels of solar and wind penetrations? This free JISEA presentation will take place in a Webinar on Tuesday, May 11, 2021 at 12 p.m.: Learn how greater coordination between the natural gas and electricity sectors impacts system operations with different levels of solar and wind penetrations. Presentation information includes: Historical and projected data for natural gas consumption and power generation in the United States as demonstrated by 2014 East Coast Polar Vortex, 2021 Texas Winter Storm Uri. Showing coupling points, FERC identified need for better coordination, different levels of coordination (decision making and optimization control), coordination framework. Case study on Colorado Front Range, ramping requirements and gas nominations, real-time dispatch (June and December scenarios), results of analysis of impacts of coordination on unserved load. Total real-time gas offtakes by node, impacts on unserved gas, effect on CO2 emissions. Discussions on clusions drawn from Colorado Case study expanded co-simulation via HELICS, objectives of the HELICS+ natural gas use case, modeling of hydrogen blending, techno economic assessment of blending, and blending impacts on energy content and pressure.

analysis↗