Discrete Deep Reinforcement Learning For Online Distribution Power System Cybersecurity Protection.
Abstract not provided.
Bailey, Tyson↗
SEARCH · Search NASA
Results for “Online learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Towards Online Machine Learning in DUNE Data Acquisition
Processing the large volumes of data produced by liquid argon time projection chamber (LArTPC) experiments presents a significant challenge, especially those at the scale of DUNE. This is a particular challenge when aiming to trigger on low-energy neutrinos from core-collapse supernovae, which are typically buried in a high-rate radiological background. To enable real-time event selection suitable for such rare signals, we are developing machine learning based data filtering methods. In order to demonstrate the feasibility of this approach, we implemented such pipeline using the ICEBERG detector at Fermilab as a small-scale LArTPC, with a focus on identifying Michel electrons as a proxy for low-energy neutrino interactions. This poster will present the current status of integrating these machine learning models into the data acquisition (DAQ) system of this detector.
Dalager, Olivia [Fermilab]↗
An Online Reinforcement Learning Control Design for Mars Ascent Vehicle
Explore the source record for details and available documents.
Han Woong Bae↗
Hybrid Deep Reinforcement Learning For Online Distribution Power System Optimization and Control.
Abstract not provided.
Bailey, Tyson↗
Robust Online Sequential RVFLNs for Data Modeling of Dynamic Time-Varying Systems with Application of an Ironmaking Blast Furnace
In a world where the increasing complexity of modern industrial processes brings difficulties for accurate mathematical modeling, taking advantage of data has become an efficient solution to complex dynamic process modeling issue. In this paper, we develop a novel robust online sequential version of random vector functional-link networks (RVFLNs) for data-driven modeling of dynamic time-varying system and applied it in a blast furnace (BF) ironmaking process. First, to overcome the time-varying dynamics of process and to enable the RVFLNs to learn online with avoiding data saturation, an improved online sequential version of RVFLNs (OS-RFVLNs) is first presented by online sequential learning with forgetting factor. This improved OS-RVFLNs algorithm is not only suitable for the real-time and large data transfer situation, but also can adjust the sensitivity of the algorithm to different samples with the help of the introduced forgetting factor. Second, since the output weights of the improved OS-RVFLNs as well as other RVFLNs algorithms are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with various outliers. To solve this problem, a Cauchy distribution weighted M-estimator is introduced to improve the robustness of the improved OS- RVFLNs. For this proposed robust OS-RVFLNs (R-OS- RVFLNs), since the weights of different outlier data are properly determined by the Cauchy distribution function, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Experiments using actual industrial data of BF ironmaking process and comparative studies have demonstrated that the proposed method produces a better estimation accuracy and stronger robustness than other methods.
Blast furnace (BF), Modelling, Dynamic systems↗
Space Operations Learning Center
The Space Operations Learning Center (SOLC) is a tool that provides an online learning environment where students can learn science, technology, engineering, and mathematics (STEM) through a series of training modules. SOLC is also an effective media for NASA to showcase its contributions to the general public. SOLC is a Web-based environment with a learning platform for students to understand STEM through interactive modules in various engineering topics. SOLC is unique in its approach to develop learning materials to teach schoolaged students the basic concepts of space operations. SOLC utilizes the latest Web and software technologies to present this educational content in a fun and engaging way for all grade levels. SOLC uses animations, streaming video, cartoon characters, audio narration, interactive games and more to deliver educational concepts. The Web portal organizes all of these training modules in an easily accessible way for visitors worldwide. SOLC provides multiple training modules on various topics. At the time of this reporting, seven modules have been developed: Space Communication, Flight Dynamics, Information Processing, Mission Operations, Kids Zone 1, Kids Zone 2, and Save The Forest. For the first four modules, each contains three components: Flight Training, Flight License, and Fly It! Kids Zone 1 and 2 include a number of educational videos and games designed specifically for grades K-6. Save The Forest is a space operations mission with four simulations and activities to complete, optimized for new touch screen technology. The Kids Zone 1 module has recently been ported to Facebook to attract wider audience.
Lui, Ben↗
Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation
Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.
adaptation models↗
Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation: Preprint
Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.
distribution system operator↗
Toward Justifiable Trust in Autonomous Systems Incorporating Human Knowledge in Autonomous Systems through Machine Learning
Trust in Autonomous Systems is largely about humans trusting the decisions made by autonomous systems. This trust can be increased through learning from domain experts. In particular, autonomous systems can learn offline from past mission operations before conducting any operations of its own. Additionally, autonomous systems can learn online by obtaining human feedback during operations. We will discuss several classes of machine learning methods and our application of them to autonomous systems. The first class of methods is anomaly detection, which uses operations data to identify examples of anomalous operations. The second class of methods is inverse reinforcement learning, also known as apprenticeship learning, that takes past operations data as input and yields a controller that is able to duplicate the operations described by the data. The third class is active learning, which identifies examples on which the model is most uncertain and requests domain expert feedback.
Oza, Nikunj C.↗
Efficient Learning of Accurate Surrogates for Simulations of Complex Systems
Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse, or dynamic data. We introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation of state model can be reliably auto-generated from expensive calculations using few model evaluations.
97 MATHEMATICS AND COMPUTING↗
Unsupervised learning-enabled pulsed infrared thermographic microscopy of subsurface defects in stainless steel
Metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing method (AM) frequently contain microscopic porosity defects, with typical approximate size distribution from one to 100 microns. Presence of such defects could lead to premature failure of the structure. In principle, structural integrity assessment of LPBF metals can be accomplished with nondestructive evaluation (NDE). Pulsed infrared thermography (PIT) is a non-contact, one-sided NDE method that allows for imaging of internal defects in arbitrary size and shape metallic structures using heat transfer. PIT imaging is performed using compact instrumentation consisting of a flash lamp for deposition of a heat pulse, and a fast frame infrared (IR) camera for measuring surface temperature transients. However, limitations of imaging resolution with PIT include blurring due to heat diffusion, sensitivity limit of the IR camera. We demonstrate enhancement of PIT imaging capability with unsupervised learning (UL), which enables PIT microscopy of subsurface defects in high strength corrosion resistant stainless steel 316 alloy. PIT images were processed with UL spatial–temporal separation-based clustering segmentation (STSCS) algorithm, refined by morphology image processing methods to enhance visibility of defects. The STSCS algorithm starts with wavelet decomposition to spatially de-noise thermograms, followed by UL principal component analysis (PCA), fine-tuning optimization, and neural learning-based independent component analysis (ICA) algorithms to temporally compress de-noised thermograms. The compressed thermograms were further processed with UL-based graph thresholding K-means clustering algorithm for defects segmentation. The STSCS algorithm also includes online learning feature for efficient re-training of the model with new data. For this study, metallic specimens with calibrated microscopic flat bottom hole defects, with diameters in the range from 203 to 76 µm, were produced using electro discharge machining (EDM) drilling. While the raw thermograms do not show any material defects, using STSCS algorithm to process PIT images reveals defects as small as 101 µm in diameter. To the best of our knowledge, this is the smallest reported size of a sub-surface defect in a metal imaged with PIT, which demonstrates the PIT capability of detecting defects in the size range relevant to quality control requirements of LPBF-printed high-strength metals.
36 MATERIALS SCIENCE↗
Efficient learning of accurate surrogates for simulations of complex systems
Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.
79 ASTRONOMY AND ASTROPHYSICS↗
Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses
Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Deep medical image analysis with representation learning and neuromorphic computing
Deep learning is increasingly used in medical imaging, improving many steps of the processing chain, from acquisition to segmentation and anomaly detection to outcome prediction. Yet significant challenges remain: (i) image-based diagnosis depends on the spatial relationships between local patterns, something convolution and pooling often do not capture adequately; (ii) data augmentation, the de facto method for learning three-dimensional pose invariance, requires exponentially many points to achieve robust improvement; (iii) labelled medical images are much less abundant than unlabelled ones, especially for heterogeneous pathological cases; and (iv) scanning technologies such as magnetic resonance imaging can be slow and costly, generally without online learning abilities to focus on regions of clinical interest. To address these challenges, novel algorithmic and hardware approaches are needed for deep learning to reach its full potential in medical imaging.
60 APPLIED LIFE SCIENCES↗
Smart Congestion Control for Delay- and Disruption Tolerant Networks
In this paper, we propose a novel congestion control framework for delay- and disruption tolerant networks (DTNs). The proposed framework, called Smart-DTN-CC, adjusts its operation automatically as a function of the dynamics of the underlying network. It employs reinforcement learning, a machine learning technique known to be well suited to problems in which the environment, in this case the network, plays a crucial role; yet, no prior knowledge about the target environment can be assumed, i.e., the only way to acquire information about the environment is to interact with it through continuous online learning. Smart-DTN-CC nodes get input from the environment (e.g., its buffer occupancy, set of neighbors, etc), and, based on that information, choose an action to take from a set of possible actions. Depending on an action’s effectiveness in controlling congestion, it will be given a reward. Smart-DTN-CC’s goal is to maximize the overall reward which translates to minimizing congestion. To our knowledge, Smart-DTN-CC is the first DTN congestion control framework that has the ability to automatically and continuously adapt to the dynamics of the target environment which allows Smart-DTNCC to deliver adequate performance in a variety of DTN applications and scenarios. As demonstrated by our experimental evaluation, Smart-DTN-CC is able to consistently outperform existing DTN congestion control mechanisms under a wide range of network conditions and characteristics.
Hirata, Celso M.↗
A Hardware and Software Co-design Framework for Energy Efficient Neuromorphic Systems
Neuromorphic systems can be realized by a variety of algorithms and architectures. A common understanding is that spiking neuromorphic designs, which encode information into spatio-temporal spiking events, are both a biologically-accurate and efficient way of processing information. However, representing the information through timing relationships induces sophisticated circuit designs in traditional CMOS-based implementations. In recent years, high-capacity resistive memory (RRAM, aka, memristor) has demonstrated great potential in mimicking synaptic behaviors. Several RRAM-based spiking neuromorphic designs exist, most of which focus on rate coding schemes. These designs simplify circuit implementations of neuron models and explore challenges such as unsatisfactory speed, resolution, and performance. As an alternative, we will explore temporal coding spiking neuromorphic systems that encode information as the relative timing of neuron activations (spikes), which have been proven to be more adaptive and energy-efficient. Developing a neuromorphic system for spiking neural network (SNN) inference and online training, however, faces some major technical challenges: (1) It lacks circuit implementation support for temporal-coding SNN to achieve satisfying power efficiency and accuracy; (2) Although existing research works have investigated memristive synapse and neuron designs for spike-timing-dependent plasticity, the non-ideal conditions in implementation, such as device variations and signal degradation, degrade online learning accuracy of large scale systems; and (3) Non-optimized, inter-layer data traffic in SNNs, leads to unnecessary data communication costs. In this project, we plan to address these challenges by a hardware and software co-design framework that incorporates solutions at the circuit, architecture, and algorithm levels. At the circuit-level, we will elaborate on the in-situ SNN processing element designs for supporting both inference and online training modes. Variation-aware schemes will be studied to improve reliability. At the architecture level, we propose a pipelined, asynchronous architecture to retain the timing resolution of spikes. At the algorithm level, we will investigate an innovative SNN training algorithm for enabling activation sparsification and reducing unnecessary data communication costs. This neuromorphic system will provide an effective solution to real-life energy-constrained applications and significantly contribute to the exploration of next-generation high-performance computing systems under the DOE context.
97 MATHEMATICS AND COMPUTING↗
Superconducting Hyperdimensional Associative Memory Circuit for Scalable Machine Learning
Here we propose a generalized architecture for the first rapid-single-flux-quantum (RSFQ) associative memory circuit. The circuit employs hyperdimensional computing (HDC), a machine learning (ML) paradigm utilizing vectors with dimensionality in the thousands to represent information. HDC designs have small memory footprints, simple computations, and simple training algorithms compared to superconducting neural network accelerators (SNNAs), making them a better option for scalable SFQ machine learning (ML) solutions. The proposed superconducting HDC (SHDC) circuit uses entirely on-chip RSFQ memory which is tightly integrated with logic, operates at 33.3 GHz, is applicable to general ML tasks, and is manufacturable at practically useful scales given current SFQ fabrication limits. Tailored to a language recognition task, SHDC consists of ~ 2-20 M Josephson junctions (JJs) and consumes up to three times less power than an analogous CMOS HDC circuit while achieving 78-84% higher throughput. SHDC is capable of outperforming the state of the art RSFQ SNNA, SuperNPU, by 48-99% for all benchmark NN architectures tested while occupying up to 90% less area and consuming up to nine times less power. To the best of the authors' knowledge, SHDC is currently the only superconducting ML approach feasible at practically useful scales for real-world ML tasks and capable of online learning.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
Shape‐Dependent Multi‐Weight Magnetic Artificial Synapses for Neuromorphic Computing
Abstract In neuromorphic computing, artificial synapses provide a multi‐weight (MW) conductance state that is set based on inputs from neurons, analogous to the brain. Herein, artificial synapses based on magnetic materials that use a magnetic tunnel junction (MTJ) and a magnetic domain wall (DW) are explored. By fabricating lithographic notches in a DW track underneath a single MTJ, 3–5 stable resistance states that can be repeatably controlled electrically using spin‐orbit torque are achieved. The effect of geometry on the synapse behavior is explored, showing that a trapezoidal device has asymmetric weight updates with high controllability, while a rectangular device has higher stochasticity, but with stable resistance levels. The device data is input into neuromorphic computing simulators to show the usefulness of application‐specific synaptic functions. Implementing an artificial neural network (NN) applied to streamed Fashion‐MNIST data, the trapezoidal magnetic synapse can be used as a metaplastic function for efficient online learning. Implementing a convolutional NN for CIFAR‐100 image recognition, the rectangular magnetic synapse achieves near‐ideal inference accuracy, due to the stability of its resistance levels. This work shows MW magnetic synapses are a feasible technology for neuromorphic computing and provides design guidelines for emerging artificial synapse technologies.
Leonard, Thomas↗