SEARCH · Search NASA
Results for “Learning control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Flexible doorway controlled Na + ion diffusion in NaPSO glassy electrolytes from machine-learning force field simulations
Oxygen doping reduces free volume yet paradoxically enhances amorphous framework flexibility, facilitating Na + ion diffusion. This balance leads to an initial increase, then a decline, observed in conductivity of NaPSO electrolytes.
Exploring Gradient-Based Multi-directional Controls in GANs
Generative Adversarial Networks (GANs) have been widely applied in modeling diverse image distributions. However, despite its impressive applications, the structure of the latent space in GANs largely remains as a black-box, leaving its controllable generation an open problem, especially when spurious correlations between different semantic attributes exist in the image distributions. To address this problem, previous methods typically learn linear directions or individual channels that control semantic attributes in the image space. However, they often suffer from imperfect disentanglement, or are unable to obtain multi-directional controls. Furthermore, in this work, in light of the above challenges, we propose a novel approach that discovers nonlinear controls, which enables multi-directional manipulation as well as effective disentanglement, based on gradient information in the learned GAN latent space. More specifically, we first learn interpolation directions by following the gradients from classification networks trained separately on the attributes, and then navigate the latent space by exclusively controlling channels activated for the target attribute in the learned directions. Empirically, with small training data, our approach is able to gain fine-grained controls over a diverse set of bi-directional and multi-directional attributes, and we showcase its ability to achieve disentanglement significantly better than state-of-the-art methods both qualitatively and quantitatively.
Device-Centric Ransomware Detection using Machine Learning-Based Memory Forensics for Smart Inverters
Ransomware attacks are the fastest-growing form of cyberattacks worldwide. Recently, ransomware attacks have targeted industrial control systems (ICSs), including power grids. Lessons learned from recent incidents in ICSs show that ransomware groups can deliver ransomware into not only the organization’s control servers, but also the operational technology (OT) devices such as smart inverters and smart grid devices. This paper proposes a machine learning (ML)- based memory forensics method enabling the detection of ransomware binaries stored in the memory of a commercial smart inverter. Device firmware binary files are extracted from a Serial Peripheral Interface (SPI) flash memory, and samples of both benign and ransomware binaries are generated by a binary manipulation method and a real-world ransomware encryption, separately. A deep transfer learning (DTL) method is used to retrain a convolutional neural network (CNN)-based ransomware detection algorithm using the generated samples. The experimental result validates that the proposed ML-based memory forensics method can accurately detect ransomware files.
Stabilization of the 81-channel coherent beam combination using machine learning
We develop a rapidly converging algorithm for stabilizing a large channel-count diffractive optical coherent beam combination. An 81-beam combiner is controlled by a novel, machine-learning based, iterative method to correct the optical phases, operating on an experimentally calibrated numerical model. A neural-network is trained to detect phase errors based on interference pattern recognition of uncombined beams adjacent to the combined one. Due to the non-uniqueness of solutions in the full space of possible phases, the network is trained within a limited phase perturbation/error range. This also reduces the number of samples needed for training. Simulations have proven that the network can converge in one step for small phase perturbations. When the trained neural-network is applied to a realistic case of 360 degree full range, an iterative scheme exploits random walking at the beginning, with the accuracy of prediction on phase feedback direction, to allow the neural-network to step into the training range for fast convergence. This neural-network-based iterative method of phase detection works tens of times faster than the commonly used stochastic parallel gradient descent approach (SPGD) using a single-detector and random dither when both are tested with random phase perturbations.
Method to Investigate Cognitive Aging Effects in Nuclear Operations Using the Rancor Microworld Simulator
Many nuclear power plant (NPP) operators are close to, or at retirement age. Since NPP operators are responsible for tasks that engage different and complex cognitive abilities, human factors researchers recently called for the need to examine cognitive aging effects and potential interactions with modernized control rooms. One way control rooms are being modernized is by replacing paper-based procedures with computer-based procedures (CBPs). This paper outlines a method for testing preference and usability of three types of CBPs as a function of age, in simulated NPP operations. The Rancor Microworld Simulator will be used, which mimics an NPP control room, allowing novices to learn to perform operator tasks quickly. The experiment will have two independent variables: age (young and old) and CBP-type (Type 1, Type 2, and Type 3). The three types of CBPs vary in levels of embedded instrumentation and controls. Adults aged 18–25 years (young group) and =60 years (older group) will be recruited from the general population. The dependent variables will be preference and usability metrics via survey and simulator logs. All participants will perform two scenario types (startup and loss of feedwater operations) and will be randomly assigned to one type of CBP to guide them through the scenario. Analyses of variance (ANOVAs) will be used to determine whether there is a main effect of age and any age x CBP-type interactions on the preference and usability dependent variables.
Designing Monte Carlo Simulation and an Optimal Machine Learning to Optimize and Model Space Missions
This paper investigates applying artificial intelligence (AI) algorithms to attitude control system of satellites to optimally tune the controller using high performance computing. This methodology is applied to the Virtual Telescope for X-ray Observation mission, which is a precise formation of two separate spacecraft observing multiple objects in the space in the X-ray domain. The mission is divided into phases based on the instrumentation and the mission goal. To reach an stable precise formation robust to stochastic slew and slew rate (i.e., Euler angles and angular velocities) in a minimal constrained time T , consumed energy of the attitude control system, denoted as E, and root-mean-square state error of attitude control system, denoted as e, are minimized. Monte-Carlo simulation is used for the sensitivity analysis of optimization and designing a controller. Deep neural networks (DNN), Gaussian processes (GP), and support vector regression (SVR) learn this optimization as a surrogate model, while their hyperparameters are optimized in a novel approach. THETA supercomputer at Argonne Leadership Computing Facility (ALCF) is used for optimizing the hyperparameters of DNN. The surrogate model meets the requirements of the mission, and it shows a better performance over the optimization and Monte-Carlo. The optimal DNN can satisfy the mission requirements e and T while reducing E for 90% compared to the other given methods.
Adaptive PID Gain Scheduling Control for Hydropower Turbine Using Neural CDE and Stochastic Distribution Shaping
This paper introduces a gain-scheduling PID controller design strategy for hydroturbine frequency control mode. This scheme first uses real data to learn the nonlinear dynamics of the hydroturbine using neural controlled differential equations and then perturbs the obtained nonlinear system at different equilibrium points, based on which a static output feedback adaptive dynamic programming algorithm is then used to optimize the PID gains for each equilibrium point. Moreover, a continuous-time version of stochastic distribution control is proposed to further fine-tune the optimized PID gains. Finally, the controller is obtained by implementing linear interpolation between the optimized PID control gains. The simulation results show that the proposed gain-scheduling PID controller can control a larger range of operation points compared with the given fixed PID controller and the baseline method. Compared with the given fixed PID controller, the proposed gain-scheduling PID controller can regulate hydroturbine frequency against disturbances induced by power-load variation with over 50% less overshoot for some operation points.
Study of MHD Stability, Active Mode Control, and Disruption Avoidance in NSTX-U
Learning how to achieve sustained operation at high plasma pressure is an important goal of the National Spherical Torus Upgrade (NSTX-U) program and is critically important to any future fusion power plant. This research addresses high priority stability and control issues outlined by the NSTX-U Program. Reduced models of the stabilization physics learned continue to be created and tested against actual plasma operation in NSTX, NSTX-U, and other tokamak devices. This research is forming physics-based models for disruption forecasting that aims to unify results across different fusion machines for more confident extrapolation of the techniques studied to future devices including ITER. The Disruption Event Characterization and Forecasting (DECAF) code was expanded to analyze data from the TCV tokamak, including real-time MHD analysis signals. The presence of a mode and likelihood of its m/n numbers can be tracked, as well as the mode frequency, although a limitation at low frequency does not yet allow tracking modes to the locking point in TCV. Initial analyses of NSTX-U operation were conducted using the DECAF code. The timing of sawtooth oscillations was investigated taking advantage of the more extensive L-mode operation of the device in its first run campaign. An initial investigation of thermal collapses in NSTX-U was also conducted with specific comparison to the Greenwald density limit. A local island power balance model to explain the density limit in tokamaks has been tested with NSTX data and is being evaluated as a potential disruption forecaster in the DECAF code, presently showing similarities to the global Greenwald density evolution and limit. A module was written for the DECAF code that generalizes magnetic spectrogram analysis of MHD modes. This new portable code, written in Python, discriminates the toroidal mode number of instabilities and can track each mode as a DECAF event, monitoring the evolution of the mode through bifurcation and locking points. This is a critical new capability to allow disruption event characterization for general tokamak databases.
Development and Validation of Home Comfort System for Total Performance Deficiency/Fault Detection and Optimal Comfort Control
In this project, we developed and tested a learning-based home thermal model that facilitates the operation of a model predictive control (MPC)-based optimization agent and an automated fault detection and diagnosis (AFDD) agent. The home thermal model was constructed using a two-node resistor-capacitor model. Moreover, two accompanying parameter identification methods were introduced, least-squares and optimization. Based on the home thermal model, the MPC-based optimization agent was developed to optimize residential HVAC operation. Using two FDD methods, the AFDD agent was constructed to detect and diagnose two prevalent residential AC faults, airflow reduction and refrigerant undercharge. The home thermal model, along with the MPC-based optimization agent and AFDD agent, were tested at the Norman Test House, Miami Test House, Pacific Northwest National Laboratory (PNNL) Test House A, and PNNL Test House B. Finally, they were also field tested in nine demonstration homes with real occupants.
Automated Control of Transactive HVACs in Energy Distribution Systems
Heating, Ventilation, and Air Conditioning (HVAC) systems contribute significantly to a building’s energy consumption. In the recent years, there is an increased interest in developing transactive approaches which could enable automated and flexible scheduling of HVAC systems based on the customer demand and the electricity prices decided by the suppliers. Flexible and automated scheduling of the HVAC systems make it a prime source for participation in residential demand response or transactive energy systems. Therefore, it is of significant interest to identify an optimal strategy to control the HVAC systems. Here, reducing the energy cost while keeping the comfort level acceptable to the users, we argue that such a control strategy should consider both the energy cost and user comfort simultaneously. Accordingly, we develop the control strategy through the solution of an optimization problem that balances between the energy cost and consumer’s dissatisfaction. This optimization enables us to solve a decision-making problem through first price prediction and then choosing HVAC temperature settings throughout the day based on the predicted price, history of the price and HVAC settings, and outside temperature. More specifically, we formulate the control design as a Markov decision process (MDP) using deep neural networks and use Deep Deterministic Policy Gradients (DDPG)-based deep reinforcement learning algorithm to find the optimal control strategy for HVAC systems that balances between electricity cost and user comfort.
Final report for LDRD project 11-0783 : directed robots for increased military manpower effectiveness.
The purpose of this LDRD is to develop technology allowing warfighters to provide high-level commands to their unmanned assets, freeing them to command a group of them or commit the bulk of their attention elsewhere. To this end, a brain-emulating cognition and control architecture (BECCA) was developed, incorporating novel and uniquely capable feature creation and reinforcement learning algorithms. BECCA was demonstrated on both a mobile manipulator platform and on a seven degree of freedom serial link robot arm. Existing military ground robots are almost universally teleoperated and occupy the complete attention of an operator. They may remove a soldier from harm's way, but they do not necessarily reduce manpower requirements. Current research efforts to solve the problem of autonomous operation in an unstructured, dynamic environment fall short of the desired performance. In order to increase the effectiveness of unmanned vehicle (UV) operators, we proposed to develop robots that can be 'directed' rather than remote-controlled. They are instructed and trained by human operators, rather than driven. The technical approach is modeled closely on psychological and neuroscientific models of human learning. Two Sandia-developed models are utilized in this effort: the Sandia Cognitive Framework (SCF), a cognitive psychology-based model of human processes, and BECCA, a psychophysical-based model of learning, motor control, and conceptualization. Together, these models span the functional space from perceptuo-motor abilities, to high-level motivational and attentional processes.
Safety-Related Instrumentation and Control Upgrade Pilot Project: NUREG- 0711 Process Development, Planning and Analysis Activities, and Lessons Learned
Constellation Energy and the United States Department of Energy (DOE) have established a public/private partnership to implement a pilot digital upgrade to replace legacy analog, safety-related reactor protection and emergency safety feature actuation systems (RPS/ESFAS) with modern digital systems. This effort is occurring at Constellation’s Limerick Generating Station. This project is being performed in accordance with industry processes that have been adapted to better support digital upgrades. These processes include IP-ENG-001, Standard Design Process, NISP-EN-04, Standard Digital Engineering Process and Electric Power Research Institute Report 3002011816, Digital Engineering Guide. The Light Water Reactor Sustainability (LWRS) Program at the Idaho National Laboratory (INL) has been supporting this effort. Latest INL HFE efforts in support of this project have focused on definition and implementation of the Human Factors Engineering (HFE) Program in support of Constellation design and related licensing efforts. This paper presents the development execution of the HFE Program through completion of the Planning and Analysis Phase as defined by NUREG-0711 as well as HFE lessons learned.
Target Control and Protection Systems Lessons from SNS Operations
The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory has been in operations since 2006 and proposes a project to build a Second Target Station (STS) to effectively double potential scientific output. The SNS target controls operate in a harsh environment which includes high radiation, exposure to gaseous radionuclides, and activated liquid mercury and mercury vapor. These conditions necessitate protective interlocks and credited controls for protection functions to ensure proper response to off-normal conditions. In order to inform the design of target controls for the STS, we have examined lessons learned during SNS operations regarding the design and implementation of the control and protection systems for the first target station (FTS). This paper will examine various aspects of the performance of the target control and protection systems including reliability, maintainability and sustainability given the challenging environment created by 1.4 MW operations. Specific topics include distributed control of various target subsystems, response to loss of power, selection of nuclear grade instrumentation, and applying these lessons to the design for the STS project.
Data-Enabled Fusion Technology (Final Scientific/Technical Report)
Advancing Scientific Understanding in Fusion Energy and Machine Learning This research represented a significant step forward in machine learning (ML) applications for fusion energy experiments. The project integrated advanced data-driven modeling, optimization techniques, and artificial intelligence to enhance the predictive capabilities and operational efficiency of plasma-based fusion systems. Specifically, tasks focused on ML-enhanced diagnostics, operator guidance tools, and predictive modeling helped improve the ability to interpret complex fusion experiments. Key areas of advancement included: 1) data-driven plasma control, i.e., using ML algorithms to optimize experimental conditions and classify plasma behaviors based on historical data; 2) spectroscopy and diagnostics, i.e., applying AI models to extract previously inaccessible insights from experimental spectroscopy data; and 3) configuration mapping and operator guidance, i.e., developing a predictive framework to assist scientists in identifying the most effective experimental parameters, reducing reliance on manual adjustments. By refining these ML-driven techniques, the project contributed to the broader scientific community’s understanding of plasma dynamics and fusion energy viability. Technical Effectiveness and Economic Feasibility The methods investigated demonstrated high technical effectiveness, as reflected in milestones assessing the predictive accuracy, performance, and optimization of fusion configurations. The development of an Operator Guidance Tool (OGT), for example, led to more precise control of plasma conditions by learning from experimental data and offering real-time adjustments. From an economic standpoint, DeFT provided: 1) the ability to reduce trial-and-error experimentation, which lowered operational costs; 2) improved data interpretation methods, which enabled more efficient resource allocation in large-scale fusion research projects; and 3) the automation of key diagnostic tasks, which reduced manual labor and human error, increasing overall efficiency. 13 The final assessments of predictive models and optimization strategies demonstrated that these approaches were scalable and could be implemented across multiple fusion energy research programs. Public Benefit and Societal Impact This project contributed directly to the broader goal of achieving sustainable and commercially viable fusion energy, which had profound implications for clean energy production and climate change mitigation. The integration of AI-driven solutions into fusion research: 1) sped up scientific discovery, accelerating progress towards achieving energy breakthroughs; 2) reduced the cost of experimentation, making fusion research more accessible; and 3) provided a framework for future AI applications in high-energy physics, benefiting adjacent fields like space exploration, material science, and renewable energy. Additionally, by fostering collaborations between AI researchers and plasma physicists, this project promoted interdisciplinary innovation that could lead to broader applications beyond fusion research.
Real-time avoidance of the L-mode and H-mode density limit via machine-learned stability metrics
Reliable operation of burning plasma tokamaks will require robust control strategies to avoid macroscopic instability limits such as the L-mode and H-mode density limits (LDL, HDL). In this work, we explore closed-loop avoidance of these phenomena at DIII-D using machine-learned risk metrics. Feedback control is implemented via the ‘DL Supervisor’ scheme, which regulates the chosen risk metric by reducing the density target or increasing NBI heating in real-time. Using the LDL 25 risk metric, the LDL is reproducibly suppressed. We also introduce an HDL risk metric in this study, HDL 25 , which reduces the False Positive Rate by 2x compared to the Greenwald fraction. Applying this scaling to a plasma current ramp-down, we successfully avoid an HDL-driven H/L back-transition. These experiments constitute the first demonstration of real-time DL avoidance using machine-learned risk metrics. These instability metrics outline a path to safer high-density operation, more reliable ramp-down scenarios, and improved off-normal control for next-step devices such as ITER and SPARC.
Entanglement engineering of optomechanical systems by reinforcement learning
Entanglement is fundamental to quantum information science and technology, yet controlling and manipulating entanglement—so-called entanglement engineering—for arbitrary quantum systems remains a formidable challenge. There are two difficulties: the fragility of quantum entanglement and its experimental characterization. We develop a model-free deep reinforcement-learning (RL) approach to entanglement engineering, in which feedback control together with weak continuous measurement and partial state observation is exploited to generate and maintain desired entanglement. We employ quantum optomechanical systems with linear or nonlinear photon–phonon interactions to demonstrate the workings of our machine-learning-based entanglement engineering protocol. In particular, the RL agent sequentially interacts with one or multiple parallel quantum optomechanical environments, collects trajectories, and updates the policy to maximize the accumulated reward to create and stabilize quantum entanglement over an arbitrary amount of time. The machine-learning-based model-free control principle is applicable to the entanglement engineering of experimental quantum systems in general.
Adaptive stretching of representations across brain regions and deep learning model layers
Prefrontal cortex (PFC) is known to modulate the visual system to favor goal-relevant information by accentuating task-relevant stimulus dimensions. Does the brain broadly re-configures itself to optimize performance by stretching visual representations along task-relevant dimensions? We considered a task that required monkeys to selectively attend on a trial-by-trial basis to one of two dimensions (color or motion direction) to make a decision. Although effects were most prominent in frontal areas, representations stretched along task-relevant dimensions in all sites considered: V4, MT, lateral PFC, frontal eye fields (FEF), lateral intraparietal cortex (LIP), and inferotemporal cortex (IT). Spike timing was crucial to this code. A deep learning model was trained on the same visual input and rewards as the monkeys. Despite lacking an explicit selective attention or other control mechanism, by minimizing error during learning, the model’s representations stretched along task-relevant dimensions, indicating that stretching is an adaptive strategy.