Search NASA⌕ Search

SEARCH · Search NASA

Results for “adaptive learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Controlling a truck with an adaptive critic temporal difference CMAC design

In this study, CMAC (Cerebellar Model Articulated Controller) neural architectures are shown to be viable for the purposes of real-time learning and control. An adaptive critic temporal difference neurocontrol design has been implemented that learns in real-time how to back up a trailer truck along a fixed straight line trajectory. The truck backer-upper experiment is a standard performance measure in the neural network literature, but previously the training of the controllers was done off-line. With the CMAC neural architectures, it was possible to train the neurocontrollers on-line in real-time on a MS-DOS PC 386.

Shelton, Robert O.↗

Gravity Sensor Plasticity in the Space Environment

The ability of the brain to learn from experience and to adapt to new environments is recognized to be profound. This ability, called 'neural plasticity,' depends directly on properties of neurons (nerve cells) that permit them to change in dimension, sprout new parts called spines, change the shape and/or size of existing parts, and to generate, alter, or delete synapses. (Synapses are communication sites between neurons.) These neuronal properties are most evident during development, when evolution guides the laying down of a general plan of the nervous system. However, once a nervous system is established, experience interacts with cellular and genetic mechanisms and the internal milieu to produce unique neuronal substrates that define each individual. The capacity for experience-related neuronal growth in the brain, as measured by the potential for synaptogenesis, is speculated to be in the trillions of synapses, but the range of increment possible for any one part of the nervous system is unknown. The question has been whether more primitive endorgans such as gravity sensors of the inner ear have a capacity for adaptive change, since this is a form of learning from experience.

Ross, Muriel D.↗

Completing and Adapting Models of Biological Processes

We present a learning-based method for model completion and adaptation, which is based on the combination of two approaches: 1) R2D2C, a technique for mechanically transforming system requirements via provably equivalent models to running code, and 2) automata learning-based model extrapolation. The intended impact of this new combination is to make model completion and adaptation accessible to experts of the field, like biologists or engineers. The principle is briefly illustrated by generating models of biological procedures concerning gene activities in the production of proteins, although the main application is going to concern autonomic systems for space exploration.

Margaria, Tiziana↗

Nonlinear functional approximation with networks using adaptive neurons

A novel mathematical framework for the rapid learning of nonlinear mappings and topological transformations is presented. It is based on allowing the neuron's parameters to adapt as a function of learning. This fully recurrent adaptive neuron model (ANM) has been successfully applied to complex nonlinear function approximation problems such as the highly degenerate inverse kinematics problem in robotics.

Tawel, Raoul↗

Adaptive Protection of Scientific Backbone Networks Using Machine Learning

In this article, we propose a new protection scheme for backbone networks to guarantee high service availability. The presented scheme does not require any reconfiguration immediately after the failure (i.e., it is proactive). At the same time, it does not require any reserved backup network resources either. To achieve these seemingly contradictory goals, we utilize the recent advancements in Machine Learning (ML) to implement a network intelligence that periodically re-allocates the unused capacity as protection bandwidth to meet the service availability requirements of each connection. Our goal is achieved by two components (1) predicting the traffic for the next period on each link, and (2) intelligently selecting the best fit dedicated protection scheme for the next period depending on the estimated unused (spare) bandwidth and the previous service availability violations. Note that re-allocating protection bandwidth affects neither the operational connections nor the current best practice of operators to over-provision network bandwidth to support elephant flows. Finally, we provide a case study on the real traffic from Energy Sciences Network (ESnet), a high-speed, international scientific backbone network. The key benefit of our framework is that adaptively utilizing the over-provisioned bandwidth for spare capacity is sufficient to improve the availability from three-nines to five-nines (in ESnet for the 30 examined connections). The drawback is negligible bandwidth limitations; the user perceives a minor and very temporal bandwidth limitation in less than 0.1% of the time.

42 ENGINEERING↗

Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs

The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.

Ravi, Abhijith↗

Closing the Certification Gaps in Adaptive Flight Control Software

Over the last five decades, extensive research has been performed to design and develop adaptive control systems for aerospace systems and other applications where the capability to change controller behavior at different operating conditions is highly desirable. Although adaptive flight control has been partially implemented through the use of gain-scheduled control, truly adaptive control systems using learning algorithms and on-line system identification methods have not seen commercial deployment. The reason is that the certification process for adaptive flight control software for use in national air space has not yet been decided. The purpose of this paper is to examine the gaps between the state-of-the-art methodologies used to certify conventional (i.e., non-adaptive) flight control system software and what will likely to be needed to satisfy FAA airworthiness requirements. These gaps include the lack of a certification plan or process guide, the need to develop verification and validation tools and methodologies to analyze adaptive controller stability and convergence, as well as the development of metrics to evaluate adaptive controller performance at off-nominal flight conditions. This paper presents the major certification gap areas, a description of the current state of the verification methodologies, and what further research efforts will likely be needed to close the gaps remaining in current certification practices. It is envisioned that closing the gap will require certain advances in simulation methods, comprehensive methods to determine learning algorithm stability and convergence rates, the development of performance metrics for adaptive controllers, the application of formal software assurance methods, the application of on-line software monitoring tools for adaptive controller health assessment, and the development of a certification case for adaptive system safety of flight.

Jacklin, Stephen A.↗

Development of Machine Learning-Derived Microbiological and Immune Signatures: Applications in Adaptive Risk Assessment of Infectious Disease During Spaceflight

Infectious diseases represent an urgent risk for spaceflight with consequences ranging from loss in crew performance to crew incapacitation or loss of life should an outbreak occur. The resident environmental microbiome on the International Space Station has been monitored through routine surveillance over almost twenty years, beginning with culture-based microbial detection which has advanced to molecular methods in recent years. This has created a wealth of data that we have begun mining to define the microbial ecology of the ISS. Summarized here is our analysis of data from the historical microbial population defined by culture-based monitoring from the past two decades, organized by their likelihood to cause disease into clinical categories. As expected, many residents of the normal microflora in environments where people work and live were detected. However, some known pathogens were also detected. As the spaceflight environment can predispose humans to infection, crew health records were used to source additional data for the set to uncover clinical relevance. Data mining was performed on crew health records to capture adverse health events that may be related to infectious disease. Machine learning, specifically Random Forest analysis, was used to analyze the microbial and crew health datasets. The symptom categories were not explained by the ranked bacteria, due to lack of sufficient data for some categories and due to poor ranking of the pathogens for others. Poor ranking of the bacteria could be due to the clinical symptoms being linked to other disease-causing factors, such as allergy or viral infection. These findings suggest a lack of relationship between bacteria detected on surfaces in the ISS and historical health events experienced by astronauts.

Kristyn Hoffman↗

Neuromorphic learning of continuous-valued mappings in the presence of noise: Application to real-time adaptive control

The ability of feed-forward neural net architectures to learn continuous-valued mappings in the presence of noise is demonstrated in relation to parameter identification and real-time adaptive control applications. Factors and parameters influencing the learning performance of such nets in the presence of noise are identified. Their effects are discussed through a computer simulation of the Back-Error-Propagation algorithm by taking the example of the cart-pole system controlled by a nonlinear control law. Adequate sampling of the state space is found to be essential for canceling the effect of the statistical fluctuations and allowing learning to take place.

Troudet, Terry↗

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation↗

Adaptive Load Shedding for Grid Emergency Control via Deep Reinforcement Learning

Emergency control, typically such as under-voltage load shedding (UVLS), is broadly used to grapple with low voltage and voltage instability issues in real-world power systems under contingencies. However, existing emergency control schemes are rule-based and cannot be adaptively applied to uncertain and floating operating conditions. Here, we propose an adaptive UVLS algorithm for emergency control via deep reinforcement learning (DRL) and expert systems. We first construct dynamic components for picturing the power system operation as the environment. The transient voltage recovery criteria, which poses time-varying requirements to UVLS, is integrated into the states and reward function to advise the learning of deep neural networks. The proposed method has no tuning issue of coefficients in reward functions, and this issue was regarded as a deficiency in the existing DRL-based algorithms. Case studies illustrate that the proposed method outperforms the traditional UVLS relay in both the timeliness and efficacy for emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗

AGFormer: Adaptive Spatiotemporal graph informed transformer for multi-reservoir inflow forecasting

Accurate reservoir inflow forecasting is crucial for effective water resource management, yet most machine learning models focus on single-reservoir prediction and overlook spatial dependencies among hydrologically connected reservoirs. Here, we propose AGFormer (Adaptive Graph-Informed Transformer), an end-to-end framework that integrates adaptive graph learning with temporal sequence modeling for multi-reservoir inflow forecasting. A shared encoder and graph attention mechanism generate reservoir-specific embeddings, which are then processed by the Transformer-based encoder–decoder for multi-step inflow forecasting. We also introduce a pretraining paradigm to learn robust temporal embeddings from misaligned historical records. Evaluated on 30 reservoirs in the Upper Colorado River Basin, AGFormer achieves superior seven-day-ahead forecasts, with NSE > 0.75 for 20 reservoirs—outperforming Encoder–Decoder LSTM, GCN+LSTM, and Transformer baselines. Adaptive graph learning captures dynamic inter-reservoir dependencies, and feature attribution aligns with snowmelt-driven hydrology. Incorporating forecasted meteorological inputs further enhances accuracy, demonstrating AGFormer’s potential to support reservoir management under dynamic hydrological conditions.

Adaptive graph learning↗

Reinforcement Learning in a Nonstationary Environment: The El Farol Problem

This paper examines the performance of simple learning rules in a complex adaptive system based on a coordination problem modeled on the El Farol problem. The key features of the El Farol problem are that it typically involves a medium number of agents and that agents' pay-off functions have a discontinuous response to increased congestion. First we consider a single adaptive agent facing a stationary environment. We demonstrate that the simple learning rules proposed by Roth and Er'ev can be extremely sensitive to small changes in the initial conditions and that events early in a simulation can affect the performance of the rule over a relatively long time horizon. In contrast, a reinforcement learning rule based on standard practice in the computer science literature converges rapidly and robustly. The situation is reversed when multiple adaptive agents interact: the RE algorithms often converge rapidly to a stable average aggregate attendance despite the slow and erratic behavior of individual learners, while the CS based learners frequently over-attend in the early and intermediate terms. The symmetric mixed strategy equilibria is unstable: all three learning rules ultimately tend towards pure strategies or stabilize in the medium term at non-equilibrium probabilities of attendance. The brittleness of the algorithms in different contexts emphasize the importance of thorough and thoughtful examination of simulation-based results.

Bell, Ann Maria↗

Revisiting the ODE Method for Recursive Algorithms: Fast Convergence Using Quasi Stochastic Approximation

Several decades ago, Profs. Sean Meyn and Lei Guo were postdoctoral fellows at ANU, where they shared interest in recursive algorithms. It seems fitting to celebrate Lei Guo's 60th birthday with a review of the ODE Method and its recent evolution. The method has been regarded as a technique for algorithm analysis. It is argued that this viewpoint is backwards: The original stochastic approximation method was surely motivated by an ODE, and tools for analysis came much later (based on establishing robustness of Euler approximations). The paper presents a brief survey of recent research in machine learning that shows the power of algorithm design in continuous time, following by careful approximation to obtain a practical recursive algorithm. While these methods are usually presented in a stochastic setting, this is not a prerequisite. In fact, recent theory shows that rates of convergence can be dramatically accelerated by applying techniques inspired by quasi Monte-Carlo. Subject to conditions, the optimal rate of convergence can be obtained by applying the averaging technique of Polyak and Ruppert. The conditions are not universal, but theory suggests alternatives to achieve acceleration. The theory is illustrated with applications to gradient-free optimization, and policy gradient algorithms for reinforcement learning.

learning and adaptive systems in artificial intell↗

Robust integrated neurocontroller for complex dynamic systems

The goal of this research effort is to develop an integrated control software environment for the purpose of creating an intelligent neurocontrol system. The system will be capable of estimating states, identifying parameters, diagnosing conditions, planning control strategies, and producing intelligent control actions. The distinct features of such control system are adaptability and on-line learning capability. The proposed system will be flexible to allow structure adaptability to account for changes in the dynamic system such as sensory failures and/or component degradations. The developed system should learn system uncertainties and changes, as they occur, while maintaining minimal control level on the dynamic system. The research activities set to achieve the research objective are summarized by the following general items: (1) Development of a system identifier or diagnostic system; (2) Development of a robust neurocontroller system, and; (3) Integration of above systems to create a robust Integration Control system (RIC-system). Two contrary approaches are investigated in this research: classical (traditional) design approach, and the simultaneous design approach. However, in both approaches neural network is the base for the development of different functions of the system. The two resulting designs will be tested and simulation results will be compared for better possible implementation.

Zein-Sabbato, S.↗

Robust Integrated Neurocontroller for Complex Dynamic Systems

The goal of this research effort is to develop an integrated control software environment for the purpose of creating an intelligent neurocontrol system. The system will be capable of estimating states, identifying parameters, diagnosing conditions, planning control strategies, and producing intelligent control actions. The distinct features of such control system are: adaptability and on-line learning capability. The proposed system will be flexible to allow structure adaptability to account for changes in the dynamic system such as: sensory failures and/or component degradations. The developed system should learn system uncertainties and changes, as they occur, while maintaining minimal control level on the dynamic system. The research activities set to achieve the research objective are summarized by the following general items: (1) Development of a system identifier or diagnostic system, (2) Development of a robust neurocontroller system, and 3. Integration of above systems to create a Robust Integrated Control system (RIC-system). Two contrary approaches are investigated in this research: classical (traditional) design approach, and the simultaneous design approach. However, in both approaches neural network is the base for the development of different functions of the system. The two resulting designs will be tested and simulation results will be compared for better possible implementation.

Zein-Sabatto, S.↗