Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,729 records · Page 96
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Tuning water dissociation at oxide–electrolyte interfaces with electric fields

Understanding how electric fields influence water dissociation at heterogeneous interfaces is crucial for controlling interfacial chemical reactions and advancing next-generation energy technologies. Herein, ab initio–based machine learning simulations show that even small electric field changes can significantly alter the water dissociation fraction at planar TiO 2 –electrolyte interfaces. The resulting free energy difference between undissociated and dissociated interfacial water exhibits a linear dependence on the field change with a slope of 1.97 eÅ, which far exceeds the dissociation-induced dipole change of a water molecule. Employing a machine-learned collective variable to investigate the reaction statistics of thousands of water dissociation/recombination events, we find that small electric field changes exert minor effects on individual reaction energy barriers but significantly influence the populations of local configurations associated with initial states that are most favorable for reactions. These findings elucidate the pronounced impact of electric fields on interfacial water dissociation and reveal a mechanism for electric-field-controlled chemical reactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Memristors learn to play

A hybrid analogue–digital computing system built upon memristive devices is capable of solving classic control problems with potentially a lower energy consumption and higher speed than fully digital systems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1998

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗
NASA NTRS2002

The Design of Collectives of Agents to Control Non-Markovian Systems

The 'Collective Intelligence' (COIN) framework concerns the design of collectives of reinforcement-learning agents such that their interaction causes a provided 'world' utility function concerning the entire collective to be maximized. Previously, we applied that framework to scenarios involving Markovian dynamics where no re-evolution of the system from counter-factual initial conditions (an often expensive calculation) is permitted. This approach sets the individual utility function of each agent to be both aligned with the world utility, and at the same time, easy for the associated agents to optimize. Here we extend that approach to systems involving non-Markovian dynamics. In computer simulations, we compare our techniques with each other and with conventional-'team games'. We show whereas in team games performance often degrades badly with time, it steadily improves when our techniques are used. We also investigate situations where the system's dimensionality is effectively reduced. We show that this leads to difficulties in the agents' ability to learn. The implication is that 'learning' is a property only of high-enough dimensional systems.

Lawson, John W.↗
NASA NTRS2004

The Design of Collectives of Agents to Control Non-Markovian Systems

The Collective Intelligence (COIN) framework concerns the design of collectives of reinforcement-learning agents such that their interaction causes a provided "world" utility function concerning the entire collective to be maximized. Previously, we applied that framework to scenarios involving Markovian dynamics where no re-evolution of the system from counter-factual initial conditions (an often expensive calculation) is permitted. This approach sets the individual utility function of each agent to be both aligned with the world utility, and at the same time, easy for the associated agents to optimize. Here we extend that approach to systems involving non-Markovian dynamics. In computer simulations, we compare our techniques with each other and with conventional "team games". We show whereas in team games performance often degrades badly with time, it steadily improves when our techniques are used. We also investigate situations where the system's dimensionality is effectively reduced. We show that this leads to difficulties in the agents ability to learn. The implication is that learning is a property only of high-enough dimensional systems.

Lawson, John W.↗
DOE OSTI2021

A Machine Learning-Based Method to Estimate Transformer Primary-Side Voltages with Limited Customer-Side AMI Measurements

Distribution control applications such as volt/var optimization, network reconfiguration, and distribution automation require accurate knowledge of the distribution system state. The lack of sufficient sensors on the primary side of distribution networks often limits the accuracy of the control decisions by these applications. The deployment of advanced metering infrastructure (AMI) provides utilities an opportunity to translate the AMI data on the secondary onto the primary so that it can be used as pseudo-measurements to augment the limited existing measurements on the primary. This paper develops an approach for estimating service transformer primary-side voltages by using limited secondary-side AMI measurements. The estimated primary-side voltages can be used by utilities as pseudo-measurements for distribution control applications. The detailed secondary model topology, which is an essential input data for many existing algorithms, is not required for the proposed method. The performance of the proposed method is validated by using AMI measurements from the field and an actual distribution feeder model of San Diego Gas & Electric Company.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2018

Fuzzy Modeling and Parallel Distributed Compensation for Aircraft Flight Control from Simulated Flight Data

A method is described that combines fuzzy system identification techniques with Parallel Distributed Compensation (PDC) to develop nonlinear control methods for aircraft using minimal a priori knowledge, as part of NASA’s Learn-to-Fly initiative. A fuzzy model was generated with simulated flight data, and consisted of a weighted average of multiple linear time invariant state-space cells having parameters estimated using the equation-error approach and a least-squares estimator. A compensator was designed for each subsystem using Linear Matrix Inequalities (LMI) to guarantee closed-loop stability and performance requirements. This approach is demonstrated using simulated flight data to automatically develop a fuzzy model and design control laws for a simplified longitudinal approximation of the F-16 nonlinear flight dynamics simulation. Results include a comparison of flight data with the estimated fuzzy models and simulations that illustrate the feasibility and utility of the combined fuzzy modeling and control approach.

Weinstein, Rose↗
NASA NTRS2018

Supervised Learning Applied to Air Traffic Trajectory Classification

Given the recent increase of interest in introducing new vehicle types and missions into the National Airspace System, a transition towards a more autonomous air traffic control system is required in order to enable and handle increased density and complexity. This paper presents an exploratory effort of the needed autonomous capabilities by exploring supervised learning techniques in the context of aircraft trajectories. In particular, it focuses on the application of machine learning algorithms and neural network models to a runway recognition trajectory-classification study. It investigates the applicability and effectiveness of various classifiers using datasets containing trajectory records for a month of air traffic. A feature importance and sensitivity analysis are conducted to challenge the chosen time-based datasets and the ten selected features. The study demonstrates that classification accuracy levels of 90% and above can be reached in less than 40 seconds of training for most machine learning classifiers when one track data point, described by the ten selected features at a particular time step, per trajectory is used as input. It also shows that neural network models can achieve similar accuracy levels but at higher training time costs.

Bosson, Christabelle S.↗
NASA NTRS2018

Supervised Learning Applied to Air Traffic Trajectory Classification

Given the recent increase of interest in introducing new vehicle types and missions into the National Airspace System, a transition towards a more autonomous air traffic control system is required in order to enable and handle increased density and complexity. This paper presents an exploratory effort of the needed autonomous capabilities by exploring supervised learning techniques in the context of aircraft trajectories. In particular, it focuses on the application of machine learning algorithms and neural network models to a runway recognition trajectory-classification study. It investigates the applicability and effectiveness of various classifiers using datasets containing trajectory records for a month of air traffic. A feature importance and sensitivity analysis are conducted to challenge the chosen time-based datasets and the ten selected features. The study demonstrates that classification accuracy levels of 90% and above can be reached in less than 40 seconds of training for most machine learning classifiers when one track data point, described by the ten selected features at a particular time step, per trajectory is used as input. It also shows that neural network models can achieve similar accuracy levels but at higher training time costs.

Bosson, Christabelle↗
NASA NTRS2016

Lessons Learned from the Advanced Topographic Laser Altimeter System

The ICESat-2 Advanced Topographic Laser Altimeter System (ATLAS) instrument is an upcoming Earth Science mission focusing on the effects of climate change. The flight instrument passed all environmental testing at GSFC (Goddard Space Flight Center) and is now ready to be shipped to the spacecraft vendor for integration and testing. This presentation walks through the lessons learned from design, hardware, analysis and testing perspective. ATLAS lessons learned include general thermal design, analysis, hardware, and testing issues as well as lessons specific to laser systems, two-phase thermal control, and optical assemblies with precision alignment requirements.

Thermal↗
NASA NTRS2014

Anomalous Thrust Production from an RF Test Device Measured on a Low-Thrust Torsion Pendulum

This paper describes the eight-day August 2013 test campaign designed to investigate and demonstrate viability of using classical magnetoplasmadynamics to obtain a propulsive momentum transfer via the quantum vacuum virtual plasma. This paper will not address the physics of the quantum vacuum plasma thruster, but instead will describe the test integration, test operations, and the results obtained from the test campaign. Approximately 30-50 micro-Newtons of thrust were recorded from an electric propulsion test article consisting primarily of a radio frequency (RF) resonant cavity excited at approximately 935 megahertz. Testing was performed on a low-thrust torsion pendulum that is capable of detecting force at a single-digit micronewton level, within a stainless steel vacuum chamber with the door closed but at ambient atmospheric pressure. Several different test configurations were used, including two different test articles as well as a reversal of the test article orientation. In addition, the test article was replaced by an RF load to verify that the force was not being generated by effects not associated with the test article. The two test articles were designed by Cannae LLC of Doylestown, Pennsylvania. The torsion pendulum was designed, built, and operated by Eagleworks Laboratories at the NASA Johnson Space Center of Houston, Texas. Approximately six days of test integration were required, followed by two days of test operations, during which, technical issues were discovered and resolved. Integration of the two test articles and their supporting equipment was performed in an iterative fashion between the test bench and the vacuum chamber. In other words, the test article was tested on the bench, then moved to the chamber, then moved back as needed to resolve issues. Manual frequency control was required throughout the test. Thrust was observed on both test articles, even though one of the test articles was designed with the expectation that it would not produce thrust. Specifically, one test article contained internal physical modifications that were designed to produce thrust, while the other did not (with the latter being referred to as the "null" test article). Test data gathered includes torsion pendulum displacement measurements which are used to calculate generated force, still imagery in the visible spectrum to document the physical configuration, still imagery in the infrared spectrum to characterize the thermal environment, and video imagery. Post-test data includes static and animated graphics produced during RF resonant cavity characterization using the COMSOL Multiphysics® software application. Excerpts from all of the above are included and discussed in this paper. Lessons learned from test integration and operations include identification of the need to replace manual control of the resonant cavity target frequency with an automated frequency control capability. Future test plans include the development of an automatic frequency control circuit. Test results indicate that the RF resonant cavity thruster design, which is unique as an electric propulsion device, is producing a force that is not attributable to any classical electromagnetic phenomenon and therefore is potentially demonstrating an interaction with the quantum vacuum virtual plasma. Future test plans include independent verification and validation at other test facilities.

Brady, David↗
DOE OSTI2021

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗
NASA NTRSDate not supplied

Comparison of Radiation-induced Damage Between Livers from Control and Chimeric Mice

Assessment of human health risks associated with space radiation exposure is based largely on the knowledge learned from studies in which animals, mostly rodents, are exposed to high-LET radiation on the ground. It has been recognized that translation of animal results to meaningful implications for human disease can be challenging, particularly for certain risk categories such as the high-LET radiation effects in the central nervous system (CNS). Considering limitations in utilizing non-human primates and clinical studies in humans, chimeric animals can potentially bridge the knowledge gap between rodents and humans. In a chimeric animal, a specific organ or a cell type is replaced with respective human cells that are functional. In this pilot study, we use PXB mice whose livers contain >90% human cells. These mice are exposed to gamma rays for investigations of DNA damage and transcriptomics changes in the humanized livers. Results obtained from PXB mice were compared non-engrafted control animals from the same background strain that are exposed to identical conditions. Staining of the liver tissues with H&E indicated that the human liver tissue in chimeric mice responds differently than the mouse liver tissue to gamma radiation on the cellular level, as evidenced by differences in inflammation and cellular damage seen on histopathology. The gene expression data collected from the liver samples will also be presented, which potentially offers an explanation for the differential responses.

Honglu Wu↗
DOE OSTI2020

Layer-Parallel Training of Deep Residual Neural Networks

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers with a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Finally, using numerical examples from supervised classification, we demonstrate that the new approach achieves a training performance similar to that of traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Building thermal model development of typical house in U.S. for virtual storage control of aggregated building loads based on limited available information

A building thermal model is an essential component for achieving optimal control of a building’s heating, ventilation, and air conditioning (HVAC). A self-learning grey-box model has been developed by implementing data-driven techniques and utilizing limited available information about the building. In previous research, detailed foreknown knowledge/information, e.g. physical and thermal properties of building materials, as well as sufficient number of observation points, e.g. indoor temperature sensors in different zones, are available for making the decision of model structure and parameters searching range easier. The availability of measured data and a narrowed searching range of model parameters, e.g. resistance and capacity for each wall, made the adopted algorithms quickly achieve near optimal values, which closely approximate the actual heat transfer of a building. Meanwhile, the details of pre-processing of the key inputs were seldom explained. Compared to the previous research, in this study, the available information is assumed to be limited which can fill the requirement for large scale virtual storage control of real residential community in near future, where it may be impractically or over time-consuming to acquire the detailed information for each single house. The developed model is validated by comparing the results to measured data in a recently-built home typical of the southeastern United States.

Cui, Borui↗
DOE OSTI2017

Data analytics and optimization of an ice-based energy storage system for commercial buildings

Ice-based thermal energy storage (TES) systems can shift peak cooling demand and reduce operational energy costs (with time-of-use rates) in commercial buildings. The accurate prediction of the cooling load, and the optimal control strategy for managing the charging and discharging of a TES system, are two critical elements to improving system performance and achieving energy cost savings. This study utilizes data-driven analytics and modeling to holistically understand the operation of an ice–based TES system in a shopping mall, calculating the system’s performance using actual measured data from installed meters and sensors. Results show that there is significant savings potential when the current operating strategy is improved by appropriately scheduling the operation of each piece of equipment of the TES system, as well as by determining the amount of charging and discharging for each day. A novel optimal control strategy, determined by an optimization algorithm of Sequential Quadratic Programming, was developed to minimize the TES system’s operating costs. Three heuristic strategies were also investigated for comparison with our proposed strategy, and the results demonstrate the superiority of our method to the heuristic strategies in terms of total energy cost savings. Specifically, the optimal strategy yields energy costs of up to 11.3% per day and 9.3% per month compared with current operational strategies. A one-day-ahead hourly load prediction was also developed using machine learning algorithms, which facilitates the adoption of the developed data analytics and optimization of the control strategy in a real TES system operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗