Search NASA⌕ Search

SEARCH · Search NASA

Results for “Working memory training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Piecewise empirical model (PEM) of resistive memory for pulsed analog and neuromorphic applications

With the end of Dennard scaling and the ever-increasing need for more efficient, faster computation, resistive switching devices (ReRAM), often referred to as memristors, are a promising candidate for next generation computer hardware. These devices show particular promise for use in an analog neuromorphic computing accelerator as they can be tuned to multiple states and be updated like the weights in neuromorphic algorithms. Modeling a ReRAM-based neuromorphic computing accelerator requires a compact model capable of correctly simulating the small weight update behavior associated with neuromorphic training. These small updates have a nonlinear dependence on the initial state, which has a significant impact on neural network training. Consequently, we propose an empirically derived general purpose model that can accurately capture the nonlinearity of an arbitrary two-terminal device to match pulse measurements important for neuromorphic computing applications. By defining the state of the device to be proportional to its current, the model parameters can be extracted from a series of voltages pulses that mimic the behavior of a device in an analog neuromorphic computing accelerator. This allows for a general, accurate, and intuitive compact circuit model that is applicable to different resistance switching device technologies. In this work we explain the details of the model, implement the model in the circuit simulator Xyce, and give an example of its usage to model a specific Ta/TaO x device.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Mixed Delay/Nondelay Embeddings Based Neuromorphic Computing with Patterned Nanomagnet Arrays

Patterned nanomagnet arrays (PNAs) have been shown to exhibit a strong geometrically frustrated dipole interaction. Some PNAs have also shown emergent domain wall dynamics. Previous works have demonstrated methods to physically probe these magnetization dynamics of PNAs to realize neuromorphic reservoir systems that exhibit chaotic dynamical behavior and high-dimensional nonlinearity. These PNA reservoir systems from prior works leverage echo state properties and linear/nonlinear short-term memory of component reservoir nodes to map and preserve the dynamical information of the input time-series data into nondelay spatial embeddings. Such mappings enable these PNA reservoir systems to imitate and predict/forecast the input time series data. However, these prior PNA reservoir systems are based solely on the nondelay spatial embeddings obtained at component reservoir nodes. As a result, they require a massive number of component reservoir nodes, or a very large spatial embedding (i.e., high-dimensional spatial embedding) per reservoir node, or both, to achieve acceptable imitation and prediction accuracy. These requirements reduce the practical feasibility of such PNA reservoir systems. To address this shortcoming, we present a mixed delay/nondelay embeddings-based PNA reservoir system. Our system uses a single PNA reservoir node with the ability to obtain a mixture of delay/nondelay embeddings of the dynamical information of the time-series data applied at the input of a single PNA reservoir node. Our analysis shows that when these mixed delay/nondelay embeddings are used to train a perceptron at the output layer, our reservoir system outperforms existing PNA-based reservoir systems for the imitation of NARMA 2, NARMA 5, NARMA 7, and NARMA 10 time series data, and for the short-term and long-term prediction of the Mackey Glass time series data.

Ti, Changpeng↗
NASA NTRS1988

KAM (Knowledge Acquisition Module): A tool to simplify the knowledge acquisition process

Analysts, knowledge engineers and information specialists are faced with increasing volumes of time-sensitive data in text form, either as free text or highly structured text records. Rapid access to the relevant data in these sources is essential. However, due to the volume and organization of the contents, and limitations of human memory and association, frequently: (1) important information is not located in time; (2) reams of irrelevant data are searched; and (3) interesting or critical associations are missed due to physical or temporal gaps involved in working with large files. The Knowledge Acquisition Module (KAM) is a microcomputer-based expert system designed to assist knowledge engineers, analysts, and other specialists in extracting useful knowledge from large volumes of digitized text and text-based files. KAM formulates non-explicit, ambiguous, or vague relations, rules, and facts into a manageable and consistent formal code. A library of system rules or heuristics is maintained to control the extraction of rules, relations, assertions, and other patterns from the text. These heuristics can be added, deleted or customized by the user. The user can further control the extraction process with optional topic specifications. This allows the user to cluster extracts based on specific topics. Because KAM formalizes diverse knowledge, it can be used by a variety of expert systems and automated reasoning applications. KAM can also perform important roles in computer-assisted training and skill development. Current research efforts include the applicability of neural networks to aid in the extraction process and the conversion of these extracts into standard formats.

Gettig, Gary A.↗
NASA NTRSDate not supplied

Early Information Parameter-Set Analysis for Satellite Close Approaches using Machine Learning

Understanding orbital mechanics is essential in space flight and navigation applications, and leveraging modern force models for flight path projection remains an important aspect in space mission design and operation. However, force models do not capture all the dynamics or perturbations in the space environment and thus are subject to errors in predicting the state vectors. The further out the predicted miss distance between spacecraft is from the time of closest approach (TCA), the larger the propagated errors in the predicted miss distance at TCA is. The dependency on these force models for spacecraft flight state prediction calls for a more reliable method that can quantify, or even reduce, these propagated errors. With recent advances in the field artificial intelligence, specifically in machine and deep learning algorithms, a model that implements these approaches can improve on the modern force model approach. The goal for this work is to provide an early-information decision-making threshold, in order to prioritize risk assessment implementation, given the ongoing increase of space objects. In analyzing the relationship of several parameters from conjunction data messages(CDMs) and solar information, early information becomes viable in miss distance prediction with unsupervised learning techniques, which learn the parameters that are linked together with miss distance and probability of collision (Pc) variables. Another approach implemented for identifying relationships within CDMs is supervised learning, in which a shallow neural network binary classifier learns to distinguish events with Pc values¡108. These parameters detected in the unsupervised process are then applied to a regression neural network, which predicts the miss distance at TCA for a specific event within a given uncertainty bound. For the regression neural network, a Long Short Term Memory (LSTM) neural network is implemented, which yields memory about each time step in an event. Using an LSTM network, the model learns to predict miss distance within 0.2km of the value measured at TCA. Although there is a limited amount of "close miss" data to train a network, the network learns to associate parameters, like large energy dissipation rates with the secondary object, with an elevated Pc

Brianna I. Robertson↗
NASA NTRS2006

Development and Characterization of Improved NiTiPd High-Temperature Shape-Memory Alloys by Solid-Solution Strengthening and Thermomechanical Processing

The need for compact, solid-state actuation systems for use in the aerospace, automotive, and other transportation industries is currently motivating research in high-temperature shape-memory alloys (HTSMA) with transformation temperatures greater than 100 C. One of the basic high-temperature alloys investigated to fill this need is Ni(19.5)Ti(50.5)Pd30. Initial testing has indicated that this alloy, while having acceptable work characteristics, suffers from significant permanent deformation (or ratcheting) during thermal cycling under load. In an effort to overcome this deficiency, various solid-solution alloying and thermomechanical processing schemes were investigated. Solid-solution strengthening was achieved by substituting 5at% gold or platinum for palladium in Ni(19.5)Ti(50.5)Pd30, the so-called baseline alloy, to strengthen the martensite and austenite phases against slip processes and improve thermomechanical behavior. Tensile properties, work behavior, and dimensional stability during repeated thermal cycling under load for the ternary and quaternary alloys were compared. The relative difference in yield strength between the martensite and austenite phases and the dimensional stability of the alloy were improved by the quaternary additions, while work output was only minimally impacted. The three alloys were also thermomechanically processed by cycling repeatedly through the transformation range under a constant stress. This so-called training process dramatically improved the dimensional stability in these samples and also recovered the slight decrease in work output caused by quaternary alloying. An added benefit of the solid-solution strengthening was maintenance of enhanced dimensional stability of the trained material to higher temperatures compared to the baseline alloy, providing a greater measure of over-temperature capability.

Bigelow, Glen↗
DOE OSTI2025

CRCNS US-France Research Proposal: Collaborative Research: Encoding reward expectation in Drosophilia

The fruit fly Drosophila melanogaster has been a valuable model for investigating the genetic and neural bases that underlie learning and memory. Early and most current studies use basic behavior conditioning protocols to study learning in controlled laboratory settings. More recently, the ability to transgenically manipulate many of the brain neurons in the fruit fly with exquisite specificity, and the recent knowledge of the synaptic ‘connectome’ of the fruit fly brain, makes these animals almost unique as a comprehensive model for studies of learning, memory and motivated behavior. In fact, the connectome has revealed many types of new connections that had until now been overlooked. Within this context, the thesis of this proposal is that studies of learning and memory will be greatly enhanced by using more sophisticated means for evaluating memory representations, such as have been developed in vertebrates, and combining those studies with information from the connectome guided by computational modelling. We propose to push beyond the boundaries of existing conditioning protocols for fruit flies to investigate more complex memory representations. In particular, we will investigate the function of reinforcement pathways in relation to the absence of expected reinforcement. More specifically, we propose a series of experiments designed to investigate the memory representations in fruit flies when an expected consequence of a Conditioned Stimulus (CS) fails to occur. Although studies have evaluated how this failure can establish extinction memory for the CS, our studies will go beyond studying extinction. Specifically, we predict that in Drosophila when a CS is associated with a failed expectation of an appetitive food reinforcement it will acquire aversive value, and vice versa for a failed expectation of an aversive reinforcer. We combine these studies with manipulations of reinforcement pathways in the CNS inspired from the connectome, iteratively knitted in with established computational models. Intellectual Merit: The concept of reinforcement expectation and incentive contrast have been influential in the development of studies of associative learning in mammals. These questions are particularly challenging to answer in vertebrates because they require exquisite cellular, temporal, and genetic specificity of experimental manipulations. The recent development of work with identified neurons and their connectomes makes the larval and adult fly brains ripe as models for pushing our understanding of neural bases for these higher- order conditioning phenomena. Broader Impacts: Public health: These analyses and the conceptual framework of prediction error processing underlying them have a profound impact on our understanding of reinforcement-related behavior in humans, including monetary rewards and the mnemonic consequences of traumatic experiences, and for pathologies of the dopamine reinforcement system. Educational: This project will provide interdisciplinary training for postdoctoral researchers, Ph.D. and undergraduate students. The PIs will act as co-supervisors or mentors of students working in the different labs via face-to-face and internet-based technologies. We will also work with ASU’s award-winning Ask- A-Biologist program. This is an online science program designed to enrich the learning experiences of students of all ages and to provide classroom material for use by K-12 teachers. We will develop an extension of a game developed under a prior NSF award, and the new game will include modules to teach K-12 students about how insects learn. We will also integrate into the AAB site a program developed by a collaborator (B Gerber) at the Leibniz Institut für Neurobiologie, Magdeburg, and now in use in schools in Germany, to teach K-12 students how to train animals using the fruit fly larval learning paradigm. Underrepresented groups: All PIs will work with their university offices of Academic Diversity and Equal Opportunity for reaching underrepresented students.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Differentiable, Learnable, Regionalized Process-Based Models With Multiphysical Outputs can Approach State-Of-The-Art Hydrologic Prediction Accuracy

Predictions of hydrologic variables across the entire water cycle have significant value for water resources management as well as downstream applications such as ecosystem and water quality modeling. Recently, purely data-driven deep learning models like long short-term memory (LSTM) showed seemingly insurmountable performance in modeling rainfall runoff and other geoscientific variables, yet they cannot predict untrained physical variables and remain challenging to interpret. Here, we show that differentiable, learnable, process-based models (called δ models here) can approach the performance level of LSTM for the intensively observed variable (streamflow) with regionalized parameterization. We use a simple hydrologic model HBV as the backbone and use embedded neural networks, which can only be trained in a differentiable programming framework, to parameterize, enhance, or replace the process-based model's modules. Without using an ensemble or post-processor, δ models can obtain a median Nash-Sutcliffe efficiency of 0.732 for 671 basins across the USA for the Daymet forcing data set, compared to 0.748 from a state-of-the-art LSTM model with the same setup. For another forcing data set, the difference is even smaller: 0.715 versus 0.722. Meanwhile, the resulting learnable process-based models can output a full set of untrained variables, for example, soil and groundwater storage, snowpack, evapotranspiration, and baseflow, and can later be constrained by their observations. Both simulated evapotranspiration and fraction of discharge from baseflow agreed decently with alternative estimates. The general framework can work with models with various process complexity and opens up the path for learning physics from big data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Optimized pulsed write schemes improve linearity and write speed for low-power organic neuromorphic devices

Neuromorphic devices are becoming increasingly appealing as efficient emulators of neural networks used to model real world problems. However, no hardware to date has demonstrated the necessary high accuracy and energy efficiency gain over CMOS in both training via backpropagation and in read via vector matrix multiplication. Such shortcomings are due to device non-idealities, particularly asymmetric conductance tuning in response to uniform voltage pulse inputs. Here, by formulating a general circuit model for capacitive ion-exchange neuromorphic devices, we show that asymmetric nonlinearity in organic electrochemical neuromorphic devices (ENODes) can be suppressed by an appropriately chosen write scheme. Simulations based upon our model suggest that a nonlinear write-selector could reduce the switching voltage and energy, enabling analog tuning via a continuous set of resistance states (100 states) with extremely low switching energy (~170 fJ • µm –2 ). Furthermore, this work clarifies the pathway to neural algorithm accelerators capable of parallelism during both read and write operations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2008

Effects of Palladium Content, Quaternary Alloying, and Thermomechanical Processing on the Behavior of Ni-Ti-Pd Shape Memory Alloys for Actuator Applications

The need for compact, solid-state actuation systems for use in the aerospace, automotive, and other transportation industries is currently driving research in high-temperature shape memory alloys (HTSMA) having transformation temperatures above 100 C. One of the basic high temperature systems under investigation to fill this need is NiTiPd. Prior work on this alloy system has focused on phase transformations and respective temperatures, no-load shape memory behavior (strain recovery), and tensile behavior for selected alloys. In addition, a few tests have been done to determine the effect of boron additions and thermomechanical treatment on the aforementioned properties. The main properties that affect the performance of a solid state actuator, namely work output, transformation strain, and permanent deformation during thermal cycling under load have mainly been neglected. There is also no consistent data representing the mechanical behavior of this alloy system over a broad range of compositions. For this thesis, ternary NiTiPd alloys containing 15 to 46 at.% palladium were processed and the transformation temperatures, basic tensile properties, and work characteristics determined. However, testing reveals that at higher levels of alloying addition, the benefit of increased transformation temperature begins to be offset by lowered work output and permanent deformation or "walking" of the alloy during thermal cycling under load. In response to this dilemma, NiTiPd alloys have been further alloyed with gold, platinum, and hafnium additions to solid solution strengthen the martensite and parent austenite phases in order to improve the thermomechanical behavior of these materials. The tensile properties, work behavior, and dimensional stability during repeated thermal cycling under load for the ternary and quaternary alloys were compared and discussed. In addition, the benefits of more advanced thermomechanical processing or training on the dimensional stability of these alloys during repeated actuation were investigated. Finally, the effect of quaternary alloying on the thermal stability of NiTiPdX alloys is determined via thermal cycling of the materials to increasing temperatures under load. It was found that solid solution additions of platinum and gold resulted in about a 30 C increase in upper use temperature compared to the baseline NiTiPd alloy, providing an added measure of over-temperature protection.

Bigelow, Glen↗
NASA NTRS2005

Training Software in Artificial-Intelligence Computing Techniques

The Artificial Intelligence (AI) Toolkit is a computer program for training scientists, engineers, and university students in three soft-computing techniques (fuzzy logic, neural networks, and genetic algorithms) used in artificial-intelligence applications. The program promotes an easily understandable tutorial interface, including an interactive graphical component through which the user can gain hands-on experience in soft-computing techniques applied to realistic example problems. The tutorial provides step-by-step instructions on the workings of soft-computing technology, whereas the hands-on examples allow interaction and reinforcement of the techniques explained throughout the tutorial. In the fuzzy-logic example, a user can interact with a robot and an obstacle course to verify how fuzzy logic is used to command a rover traverse from an arbitrary start to the goal location. For the genetic algorithm example, the problem is to determine the minimum-length path for visiting a user-chosen set of planets in the solar system. For the neural-network example, the problem is to decide, on the basis of input data on physical characteristics, whether a person is a man, woman, or child. The AI Toolkit is compatible with the Windows 95,98, ME, NT 4.0, 2000, and XP operating systems. A computer having a processor speed of at least 300 MHz, and random-access memory of at least 56MB is recommended for optimal performance. The program can be run on a slower computer having less memory, but some functions may not be executed properly.

Howard, Ayanna↗
DOE OSTI2020

PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM

The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Additionally, numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars at performing matrix-vector multiplication operations that are prevalent in training. However, they still suffer from inefficiency due to the use of serial reads and writes for performing the weight gradient and update step. A few works have demonstrated the possibility of performing outer products in crossbars, which can be used to realize the weight gradient and update step without the use of serial reads and writes. However, these works have been limited to low precision operations which are not sufficient for typical training workloads. Moreover, they have been confined to a limited set of training algorithms for fully-connected layers only. To address these limitations, we propose a bit-slicing technique for enhancing the precision of ReRAM-based outer products, which is substantially different from bit-slicing for matrix-vector multiplication only. We incorporate this technique into a crossbar architecture with three variants catered to different training algorithms. To evaluate our design on different types of layers in neural networks (fully-connected, convolutional, etc.) and training algorithms, we develop PANTHER, an ISA-programmable training accelerator with compiler support. Our design can also be integrated into other accelerators in the literature to enhance their efficiency. Our evaluation shows that PANTHER achieves up to 8.02×, 54.21×, and 103× energy reductions as well as 7.16×, 4.02×, and 16× execution time reductions compared to digital accelerators, ReRAM-based accelerators, and GPUs, respectively.

42 ENGINEERING↗
DOE OSTI2024

Dynamic data-driven multiscale modeling for predicting the degradation of a 316L stainless steel nuclear cladding material

Here, we have developed a long short-term memory stacked ensemble (LSTM-SE) surrogate modeling approach that can provide rapid predictions of microstructural evolution and the resultant mechanical properties of American Iron and Steel Institute (AISI) 316L series stainless steel (316LSS) fuel cladding under conditions of varying temperature and radiation dose rate. To acquire training data, we developed and implemented a kinetic Monte Carlo (KMC) model to simulate precipitation kinetics of M 23 C 6 , γ', and G phases within SS316L cladding. Experimentally reported precipitation kinetics of SS316L in literature were linked to the kinetic parameters of the simulated precipitation in our KMC model. The model was then used to simulate microstructure evolution under synthetically generated treatments of varying temperature and radiation dose rate, for periods of up to 3000 hours. Changes in volume fraction, number density, and particle size of precipitates were recorded, and particle area fractions were correlated using statistical methods to develop the surrogate model. Simultaneously, the mechanical properties of the simulated microstructures were evaluated using microstructure-based finite element method (FEM) analysis to determine the elastic modulus, yield stress, ultimate tensile strength, and elongation to failure of the aged microstructures. Using this approach, our surrogate model can predict precipitation behavior within 0.25% volume fraction and mechanical properties within 6% relative error from the values predicted by the KMC and FEM models using 50 training simulations as input. The trained recurrent neural network-based model can return estimations of precipitation kinetics and mechanical properties ~1000 times faster than the physics-based codes. This work demonstrates, as a proof of concept, that reactor material service lifetimes under variable service conditions can be predicted for a statistics-based model from a practicably obtainable dataset.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Dynamic Data-Driven Multiscale Modeling for Predicting Microstructure and Mechanical Properties of 316L Stainless Steel

We have developed a long short-term memory stacked ensemble (LSTM-SE) surrogate modeling approach that can provide rapid predictions of microstructural evolution and the resultant mechanical properties of American Iron and Steel Institute (AISI) 316L series stainless steel (316LSS) fuel cladding under conditions of varying temperature and radiation dose rate. To acquire training data, we developed and implemented a kinetic Monte Carlo (KMC) model to simulate precipitation kinetics of M 23 C 6 , γ', and G phases within SS316L cladding. Experimentally reported precipitation kinetics of SS316L in literature were linked to the kinetic parameters of the simulated precipitation in our KMC model. The model was then used to simulate microstructure evolution under synthetically generated treatments of varying temperature and radiation dose rate, for periods of up to 3000 hours. Changes in volume fraction, number density, and particle size of precipitates were recorded, and particle area fractions were correlated using statistical methods to develop the surrogate model. Simultaneously, the mechanical properties of the simulated microstructures were evaluated using microstructure-based finite element method (FEM) analysis to determine the elastic modulus, yield stress, ultimate tensile strength, and elongation to failure of the aged microstructures. Using this approach, our surrogate model can predict precipitation behavior within 0.25% volume fraction and mechanical properties within 6% relative error from the values predicted by the KMC and FEM models using 50 training simulations as input. The trained recurrent neural network-based model can return estimations of precipitation kinetics and mechanical properties ~1000 times faster than the physics-based codes. This work demonstrates, as a proof of concept, that reactor material service lifetimes under variable service conditions can be predicted for a statistics-based model from a practicably obtainable dataset.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Recurrent Neural Network Language Models for Open Vocabulary Event-Level Cyber Anomaly Detection

Automated analysis methods are crucial aids for monitoring and defending a network to protect the sensitive or confidential data it hosts. This work introduces a flexible, powerful, and unsupervised approach to detecting anomalous behavior in computer and network logs; one that largely eliminates domain-dependent feature engineering employed by existing methods. By treating system logs as threads of interleaved ``sentences'' (event log lines) to train online unsupervised neural network language models, our approach provides an adaptive model of normal network behavior. We compare the effectiveness of both standard and bidirectional recurrent neural network language models at detecting malicious activity within network log data. Extending these models, we introduce a tiered recurrent architecture, which provides context by modeling sequences of users' actions over time. Compared to isolation forests and Principal Components Analysis (PCA), two popular anomaly detection algorithms, we observe superior performance on the Los Alamos National Laboratory Cyber Security dataset. For log-line-level predictions, our best performing character-based model provides test set anomaly scores ranking red team events in the 98th percentile on average, and yields 0.97 area under the receiver operator characteristic curve, demonstrating the strong fine-grained anomaly detection performance of this approach on open vocabulary logging sources.

Cyber Analysis, Deep Learning, language analysis, ↗
DOE OSTI2021

Forecasting Solar-Thermal Systems Performance under Transient Operation Using a Data-Driven Machine Learning Approach Based on the Deep Operator Network Architecture

Modeling and prediction of the dynamic behavior of thermal systems operating under intermittent energy input and variable load requirements represent one of the greatest challenges in the development of efficient and reliable renewable-based power generation technologies. In this work, a data-driven machine learning modeling framework was developed based on a modified version of the Deep Operator Network architecture where the time coordinate in the trunk net is replaced with historical data of the predicting quantity. The modeling framework can be used to accurately predict the performance of renewable-based energy conversion technologies including wind- and solar-based power plants. This novel framework was applied on a solar-thermal system that consists of a solar collection loop using a flat plate collector, a power generation loop comprising an Organic Rankine Cycle, and a thermal energy storage tank connecting both loops. Variable solar irradiance, air temperature, and power load profiles were used by the Deep Operator Network to predict the State-of-Charge and the efficiency of the thermal system for several days. The results were compared with the State-of-Charge and efficiency functions calculated using a physics-based model. For a simple operation scenario, characterized by a clear sky solar irradiance profile and constant load, the standard deviation in the State-of-Charge prediction by Deep Operator Network is below 0.9% during a seven-day prediction time horizon. For the most realistic operation scenario that considers real solar irradiance and a rough load profile, the maximum standard deviation in the predictions for the State-of-Charge and efficiency are below 6.8% and 2.5%, respectively. A comparison between Deep Operator Network and Long Short Term Memory network was also performed. In general, both networks predict very well the State-of-Charge for different data density conditions; however, a higher accuracy, with a standard deviation below 2.0%, is obtained by the Deep Operator Network during three and half days using sparser training data of 20-minute points. The same accuracy for the State-of-Charge prediction with the Long Short Term Memory network is achieved only for 14 h. Average standard deviations for the State-of-Charge prediction of 1.1% with the Deep Operator Network and 1.5% with the Long Short Term Memory network are obtained for a four-day prediction time using a denser training data of 5-minute points.

DeepONet↗
NASA NTRS2016

Transitioning Enhanced Land Surface Initialization and Model Verification Capabilities to the Kenya Meteorological Department (KMD)

Flooding, severe weather, and drought are key forecasting challenges for the Kenya Meteorological Department (KMD), based in Nairobi, Kenya. Atmospheric processes leading to convection, excessive precipitation and/or prolonged drought can be strongly influenced by land cover, vegetation, and soil moisture content, especially during anomalous conditions and dry/wet seasonal transitions. It is thus important to represent accurately land surface state variables (green vegetation fraction, soil moisture, and soil temperature) in Numerical Weather Prediction (NWP) models. The NASA SERVIR and the Short-term Prediction Research and Transition (SPoRT) programs in Huntsville, AL have established a working partnership with KMD to enhance its regional modeling capabilities. SPoRT and SERVIR are providing experimental land surface initialization datasets and model verification capabilities for capacity building at KMD. To support its forecasting operations, KMD is running experimental configurations of the Weather Research and Forecasting (WRF; Skamarock et al. 2008) model on a 12-km/4-km nested regional domain over eastern Africa, incorporating the land surface datasets provided by NASA SPoRT and SERVIR. SPoRT, SERVIR, and KMD participated in two training sessions in March 2014 and June 2015 to foster the collaboration and use of unique land surface datasets and model verification capabilities. Enhanced regional modeling capabilities have the potential to improve guidance in support of daily operations and high-impact weather and climate outlooks over Eastern Africa. For enhanced land-surface initialization, the NASA Land Information System (LIS) is run over Eastern Africa at ~3-km resolution, providing real-time land surface initialization data in place of interpolated global model soil moisture and temperature data available at coarser resolutions. Additionally, real-time green vegetation fraction (GVF) composites from the Suomi-NPP VIIRS instrument is being incorporated into the KMD-WRF runs, using the product generated by NOAA/NESDIS. Model verification capabilities are also being transitioned to KMD using NCAR's Model *Corresponding author address: Jonathan Case, ENSCO, Inc., 320 Sparkman Dr., Room 3008, Huntsville, AL, 35805. Email: Jonathan.Case-1@nasa.gov Evaluation Tools (MET; Brown et al. 2009) software in conjunction with a SPoRT-developed scripting package, in order to quantify and compare errors in simulated temperature, moisture and precipitation in the experimental WRF model simulations. This extended abstract and accompanying presentation summarizes the efforts and training done to date to support this unique regional modeling initiative at KMD. To honor the memory of Dr. Peter J. Lamb and his extensive efforts in bolstering weather and climate science and capacity-building in Africa, we offer this contribution to the special Peter J. Lamb symposium. The remainder of this extended abstract is organized as follows. The collaborating international organizations involved in the project are presented in Section 2. Background information on the unique land surface input datasets is presented in Section 3. The hands-on training sessions from March 2014 and June 2015 are described in Section 4. Sample experimental WRF output and verification from the June 2015 training are given in Section 5. A summary is given in Section 6, followed by Acknowledgements and References.

land surface modeling↗
DOE OSTI2022

SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms

Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Automated Framework for Groundwater Monitoring Using DWT with LSTM and Transformers

Environmental monitoring is critical for safeguarding public health and ecological well-being. Traditional data structuring and workflow monitoring methods consume significant time and effort, hindering timely insights and effective decision-making. Our study addresses this challenge by presenting an AI framework that automates data cleaning, structuring, and modeling processes, specifically targeting applications in groundwater monitoring. By leveraging automation for data processing and model training, our framework establishes a novel and efficient paradigm for environmental monitoring, with its potential application to the vast network of over a hundred Department of Energy Environmental Management (DoE-EM) cleanup sites across the country. It analyzes data streams from a network of groundwater Internet-of-Things (IoT) sensors deployed at the Savannah River Site (SRS) for prediction modeling. This allows human experts to focus on analysis and decision-making, ultimately leading to better environmental outcomes.The framework employs multivariate time-series forecasting methods to study and model the behavior of varying chemical analytes. The continuous learning process is enabled by utilizing deep learning techniques. It allows the framework to become more nuanced in its analysis over time, adapting to the specific characteristics of the environmental site and the evolving nature of contaminant behavior. Deep learning models known for sequence modeling, LSTM, and Transformers are employed for time series forecasting. Data processing and structuring are essential components significantly impacting the final model's performance. This hypothesis was proven by presenting a comparative analysis of model performance with processed and unprocessed data. The feature engineering approach utilized was the Discrete Wavelet Transform, which works well with time series data.

Discrete Wavelet Transform (DWT)↗