SEARCH · Search NASA

Results for “Imitation network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2021

POINT: Partially Observable Imitation Network for Traffic Signal Control

Smart traffic signals bring together transportation infrastructure and advance technologies to improve the mobility and efficiency of urban transportation network. Adaptive traffic signal control studies can be categorized into modeling-based approaches and learning-based approaches. In order to take advantages of these two systems, this study developed an offline-online combined Partial Observable Imitation Network for Traffic signal control (POINT). In the offline system, the traffic signal timing optimization problem was formulated as a Mixed Integer Nonlinear Programming (MINLP) given complete traffic information, i.e., second-by-second speeds and locations of all vehicles. Furthermore, the objective of MINLP is to minimize total travel delays considering individual vehicle trajectories under Connected Vehicle (CV) environment. The calculated optimal solutions under various traffic conditions were considered as the ”expert” decisions. In the online system, an imitation neural network model was developed to learn the ”expert” signal plans generated from offline system. Given partial observable traffic conditions in real time, e.g., the aggregate-level of traffic volume, the POINT model can compute the signal timing parameters in the online system. The numerical results demonstrated that the proposed method outperformed other state-of-the-art signal control method under high and unbalanced traffic demand levels in terms of reducing travel delays and queue length.

33 ADVANCED PROPULSION SYSTEMS↗

NASA NTRS1987

Teaching artificial neural systems to drive: Manual training techniques for autonomous systems

A methodology was developed for manually training autonomous control systems based on artificial neural systems (ANS). In applications where the rule set governing an expert's decisions is difficult to formulate, ANS can be used to extract rules by associating the information an expert receives with the actions taken. Properly constructed networks imitate rules of behavior that permits them to function autonomously when they are trained on the spanning set of possible situations. This training can be provided manually, either under the direct supervision of a system trainer, or indirectly using a background mode where the networks assimilates training data as the expert performs its day-to-day tasks. To demonstrate these methods, an ANS network was trained to drive a vehicle through simulated freeway traffic.

Shepanski, J. F.↗

NASA NTRS1996

Modular, Hierarchical Learning By Artificial Neural Networks

Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.

Baldi, Pierre F.↗

DOE OSTI2023

Emulation automation and model checking

A method of automating emulations is provided. The method comprising collecting publicly available network data over a predefined time interval, wherein the collected network data might comprise structured and unstructured data. Any unstructured data is converted into structured data. The original and converted structured data is stored in a database and compared to known network vulnerabilities. An emulated network is created according to the collected network data and the comparison of the structured data with known vulnerabilities. Virtual machines are created to run on the emulated network. Director programs and guest actor programs are run on the virtual machines, wherein the actor programs imitate real user behavior on the emulated network. The director programs deliver task commands to the guest actor programs to imitate real user behavior. The imitated behavior is presented to a user via an interface.

Urias, Vincent↗

DOE OSTI2020

Cutting the Deployment Costs of Physics-Based MPC in Buildings by Simulation-Based Imitation Learning

It has been shown that model predictive control (MPC) is a promising solution for energy-efficient building operations. However, the deployment of MPC in a large portion of the building stock has not been possible partially because of high installation costs. Every building is unique and requires a tailored MPC solution. The best performing solutions are often based on physics-based modeling, which is, however, computationally expensive and requires dedicated software. A promising direction that tackles this problem is to train a neural network-based optimal control policy to imitate the behavior of physics-based MPC from the simulation data generated offline. The neural networks give control actions that closely approximate those produced by physics-based MPC, but with a fraction of the computational and memory requirements and without the need for licensed software. The main advantage of the proposed approach stems from simple evaluation at execution time, leading to low computational footprints and easy deployment on embedded HW platforms. In the case study, we present the energy savings potential of physics-based MPC applied to an office building in Belgium. We demonstrate how neural network approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the remaining challenges and open research questions.

Drgona, Jan↗

NASA NTRS2008

Spiking Neurons for Analysis of Patterns

Artificial neural networks comprising spiking neurons of a novel type have been conceived as improved pattern-analysis and pattern-recognition computational systems. These neurons are represented by a mathematical model denoted the state-variable model (SVM), which among other things, exploits a computational parallelism inherent in spiking-neuron geometry. Networks of SVM neurons offer advantages of speed and computational efficiency, relative to traditional artificial neural networks. The SVM also overcomes some of the limitations of prior spiking-neuron models. There are numerous potential pattern-recognition, tracking, and data-reduction (data preprocessing) applications for these SVM neural networks on Earth and in exploration of remote planets. Spiking neurons imitate biological neurons more closely than do the neurons of traditional artificial neural networks. A spiking neuron includes a central cell body (soma) surrounded by a tree-like interconnection network (dendrites). Spiking neurons are so named because they generate trains of output pulses (spikes) in response to inputs received from sensors or from other neurons. They gain their speed advantage over traditional neural networks by using the timing of individual spikes for computation, whereas traditional artificial neurons use averages of activity levels over time. Moreover, spiking neurons use the delays inherent in dendritic processing in order to efficiently encode the information content of incoming signals. Because traditional artificial neurons fail to capture this encoding, they have less processing capability, and so it is necessary to use more gates when implementing traditional artificial neurons in electronic circuitry. Such higher-order functions as dynamic tasking are effected by use of pools (collections) of spiking neurons interconnected by spike-transmitting fibers. The SVM includes adaptive thresholds and submodels of transport of ions (in imitation of such transport in biological neurons). These features enable the neurons to adapt their responses to high-rate inputs from sensors, and to adapt their firing thresholds to mitigate noise or effects of potential sensor failure. The mathematical derivation of the SVM starts from a prior model, known in the art as the point soma model, which captures all of the salient properties of neuronal response while keeping the computational cost low. The point-soma latency time is modified to be an exponentially decaying function of the strength of the applied potential. Choosing computational efficiency over biological fidelity, the dendrites surrounding a neuron are represented by simplified compartmental submodels and there are no dendritic spines. Updates to the dendritic potential, calcium-ion concentrations and conductances, and potassium-ion conductances are done by use of equations similar to those of the point soma. Diffusion processes in dendrites are modeled by averaging among nearest-neighbor compartments. Inputs to each of the dendritic compartments come from sensors. Alternatively or in addition, when an affected neuron is part of a pool, inputs can come from other spiking neurons. At present, SVM neural networks are implemented by computational simulation, using algorithms that encode the SVM and its submodels. However, it should be possible to implement these neural networks in hardware: The differential equations for the dendritic and cellular processes in the SVM model of spiking neurons map to equivalent circuits that can be implemented directly in analog very-large-scale integrated (VLSI) circuits.

Huntsberger, Terrance↗

DOE OSTI2021

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗

DOE OSTI2020

A caloritronics-based Mott neuristor

Abstract Machine learning imitates the basic features of biological neural networks at a software level. A strong effort is currently being made to mimic neurons and synapses with hardware components, an approach known as neuromorphic computing. While recent advances in resistive switching have provided a path to emulate synapses at the 10 nm scale, a scalable neuron analogue is yet to be found. Here, we show how heat transfer can be utilized to mimic neuron functionalities in Mott nanodevices. We use the Joule heating created by current spikes to trigger the insulator-to-metal transition in a biased VO 2 nanogap. We show that thermal dynamics allow the implementation of the basic neuron functionalities: activity, leaky integrate-and-fire, volatility and rate coding. This approach could enable neuromorphic hardware to take full advantage of the rapid advances in memristive synapses, allowing for much denser and complex neural networks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2023

Reinforcement Learning Approach to Cybersecurity in Space (RELACSS)

Securing satellite groundstations against cyber-attacks is vital to national security missions. However, these cyber threats are constantly evolving. As vulnerabilities are discovered and patched, new vulnerabilities are discovered and exploited. In order to automate the process of discovering existing vulnerabilities and the means to exploit them, a reinforcement learning framework is presented in this report. We demonstrate that this framework can learn to successfully navigate an unknown network and detect nodes of interest despite the presence of a moving target defense. The agent then exfiltrates a file of interest from the node as quickly as possible. This framework also incorporates a defensive software agent that learns to impede the attacking agents progress. This setup allows for the agents to work against each other and improve their abilities. We anticipate that this capability will help uncover unforeseen vulnerabilities and the means to mitigate them. The modular nature of the framework enables users to swap out learning algorithms and modify the reward functions in order to adapt the learning tasks to various use cases and environments. Several algorithms, viz., tabular Q learning, deep Q networks, proximal policy optimization, advantage actor-critic, generative adversarial imitation learning, are explored for the agents and the results highlighted. The agent learns to solve the tasks in a light-weight abstract environment. Once the agent learns to perform sufficiently well, it can be deployed in a minimega virtual machine environment (or a real network) with wrappers that map abstract actions to software commands. The agent also uses a local representation of the actions called a ‘slot-mechanism’. This allows the agent to learn in a certain network and generalize it to different networks. The defensive agent learns to predict the actions taken by an offensive agent and uses that information to anticipate the threat. This information can then either be used to raise an alarm or to take actions to thwart the attack. We believe that with the appropriate reward design, a representative environment, and action set, this framework can be generalized to tackle other cybersecurity tasks. By sufficiently training these agents, we can anticipate vulnerabilities leading to robust future designs. We can also deploy automated defensive agents that can help secure satellite groundstation and their vital national security missions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2022

Adversarial methods to reduce simulation bias in neutrino interaction event filtering at liquid argon time projection chambers

For current and future neutrino oscillation experiments using large liquid argon time projection chambers (LAr-TPCs), a key challenge is identifying neutrino interactions from the pervading cosmic-ray background. Rejection of such background is often possible using traditional cut-based selections, but this typically requires the prior use of computationally expensive reconstruction algorithms. This work demonstrates an alternative approach of using a 3D submanifold sparse convolutional network trained on low-level information from the scintillation light signal of interactions inside LAr-TPCs. This technique is applied to example simulations from ICARUS, the far detector of the short baseline neutrino program at Fermilab. The results of the network, show that cosmic background is reduced by up to 76.3% whilst neutrino interaction selection efficiency remains over 98.9%. We further present a way to mitigate potential biases from imperfect input simulations by applying domain adversarial neural networks (DANNs), for which modified simulated samples are introduced to imitate real data and a small portion of them are used for adversarial training. A series of mock-data studies are performed and demonstrate the effectiveness of using DANNs to mitigate biases, showing neutrino interaction selection efficiency performances significantly better than that achieved without the adversarial training.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS1996

CI Controls for Energy and Environment

Computational intelligence (CI) is a rapidly evolving field that utilizes life imitating metaphors for guiding model building including, but not limited to neural networks, fuzzy logic, genetic algorithms, artificial life, and hybrid CI paradigms. Although the boundaries between artificial intelligence (AI) and CI are not distinct, their research communities are separate and distinct. CI researchers tend to focus on processing numerical data from sensors, while the AI community generally relies on symbolic computing to capture human knowledge. In both areas, there is a great deal of interest and activity in hybrid systems that can offset the limitations of individual methods, extend their capabilities, and create new capabilities. Examples of the benefits that can accrue from hybrid systems are contained.

Biondo, Samuel J.↗

DOE OSTI2020

Heterogeneous Machine Learning on High Performance Computing for End to End Driving of Autonomous Vehicles

Current artificial intelligence techniques for end to end driving of autonomous vehicles typically rely on a single form of learning or training processes along with a corresponding dataset or simulation environment. Relatively speaking, success has been shown for a variety of learning modalities in which it can be shown that the machine can successfully “drive” a vehicle. However, the realm of real-world driving extends significantly beyond the realm of limited test environments for machine training. This creates an enormous gap in capability between these two realms. With their superior neural network structures and learning capabilities, humans can be easily trained within a short period of time to proceed from limited test environments to real world driving. For machines though, this gap is guarded by at least two challenges: 1) machine learning techniques remain brittle and unable to generalize to a wide range of scenarios, and 2) effective training data that enhances generalization and generates the desired driving behavior. Further, each challenge can be computationally intensive on its own thereby exasperating the gap. Moreover, is has not yet been shown that a single form of learning or training is capable of addressing a large range of scenarios. As a result, solving the first challenge does not inherently solve the second and vice versa. The work described here discusses an approach to address the first challenge that would also provide a foundation for solving the second. Our approach utilizes a combination of conditional imitation learning with a static dataset, reinforcement learning with a simulation environment, and high-performance computing to train a neural network. As a result, this reduces the “time to solution” from to the existing techniques for autonomous driving and provides an extensible framework to address the second key challenge.

autonomous vehicles↗

DOE OSTI2022

Evolutionary vs imitation learning for neuromorphic control at the edge*

Abstract Neuromorphic computing offers the opportunity to implement extremely low power artificial intelligence at the edge. Control applications, such as autonomous vehicles and robotics, are also of great interest for neuromorphic systems at the edge. It is not clear, however, what the best neuromorphic training approaches are for control applications at the edge. In this work, we implement and compare the performance of evolutionary optimization and imitation learning approaches on an autonomous race car control task using an edge neuromorphic implementation. We show that the evolutionary approaches tend to achieve better performing smaller network sizes that are well-suited to edge deployment, but they also take significantly longer to train. We also describe a workflow to allow for future algorithmic comparisons for neuromorphic hardware on control applications at the edge.

Schuman, Catherine↗

NASA NTRS1988

Compound-taper feed horn for NASA 70-m antennas

A novel X-band feedhorn was designed for the Deep Space Network (DSN) 70-meter antennas. The feedhorn is a compound-taper structure consisting of a corrugated flared section and a corrugated straight section. This feedhorn is designed to closely imitate the characteristics of the standard feedhorn, while providing the proper phase center location, without adding any significant loss to the system. The use of the existing feedhorn and the ease of manufacturing the corrugated straight section have resulted in major overall cost savings.

Manshadi, Farzin↗

DOE OSTI2021

Control and Simulation of a Grid-Forming Inverter for Hybrid PV-Battery Plants in Power System Black Start

Power system restoration is an important part of system planning. Power utilities are required to maintain black start capable generators that can energize the transmission system and provide cranking power to non-blackstart capable generators. Traditionally, hydro and diesel units are used as black start capable generators. With the increased penetration of bulk size solar farms, inverter based generation can play an important role in faster and parallel black start thus ensuring system can be brought back into service without the conventional delays that can be expected with limited black start generators. Inverter-based photovoltaic (PV) power plants have advantages that are suitable for black start. This paper proposes the modeling, control, and simulation of a grid-forming inverter-based PV-battery power plant that can be used as a black start unit. The inverter control includes both primary and secondary control loops to imitate the control of a conventional synchronous machine. The proposed approach is verified using a test system modified from the IEEE 9-bus system in the time-domain electromagnetic transient simulation tool PSCAD. The simulation results shows voltage and frequency stability during a multi-step black-start and network energization process.

Nguyen, Quan H.↗

DOE OSTI2021

Application of machine learning in the determination of impact parameter in the 132 Sn+ 124 Sn system

Here, 132 Sn + 124 Sn collisions at a beam energy of 270 MeV/nucleon were performed at the Radioactive Isotope Beam Factory (RIBF) in RIKEN to investigate the nuclear equation of state. Reconstructing the impact parameter is one of the important tasks in the experiment as it relates to many observable. In this work, we employ three commonly used algorithms in machine learning, the artificial neural network (ANN), the convolutional neural network (CNN), and the light gradient boosting machine (LightGBM), to determine the impact parameter by analyzing either the charged particle spectra or several features simulated with events from the ultrarelativistic quantum molecular dynamics (UrQMD) model. To closely imitate experimental data and investigate the generalizability of the trained machine learning algorithms, incompressibility of nuclear equation of state and the in-medium nucleon-nucleon cross sections are varied in the UrQMD model to generate the training data. The mean absolute error Δb between the true and the predicted impact parameter is smaller than 0.45 fm if training and testing sets are sampled from the UrQMD model with the same parameter set. However, if training and testing sets are sampled with different parameter sets, Δb would increase to 0.8 fm. The generalizability of the trained machine learning algorithms suggests that these machine learning algorithms can be used reliably to reconstruct the impact parameter in experiment.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS2018

Occluded Object Reconstruction for First Responders with Augmented Reality Glasses Using Conditional Generative Adversarial Networks

Firefighters suffer a variety of life-threatening risks, including line-of-duty deaths, injuries, and exposures to hazardous substances. Support for reducing these risks is important. We built a partially occluded object reconstruction method on augmented reality glasses for first responders. We used a deep learning based on conditional generative adversarial networks to train associations between the various images of flammable and hazardous objects and their partially occluded counterparts. Our system then reconstructed an image of a new flammable object. Finally, the reconstructed image was superimposed on the input image to provide "transparency". The system imitates human learning about the laws of physics through experience by learning the shape of flammable objects and the flame characteristics.

Chow, Edward↗

DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗