Search NASA⌕ Search

SEARCH · Search NASA

Results for “Working memory training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Bridging the Gap Between LLMs and LNS with Dynamic Data Format and Architecture Codesign

Deep neural networks (DNNs) have achieved tremendous success in the past few years. However, their training and inference demand exceptional computational and memory resources. Quantization has been shown as an effective approach to mitigate the cost, with the mainstream data types reduced from FP32 to FP16/BF16 and recently FP8 in the latest NVIDIA H100 GPUs. With increasingly aggressive quantization, however, the conventional floating-point formats suffer from limited precision in representing numbers around zero. Recently, NVIDIA demonstrated the potential of using a Logarithmic Number System (LNS) for the next generation of tensor cores. While LNS mitigates the hurdles in representing small numbers, in this work we observed a mismatch between LNS and the emerging Large Language Models (LLM), where LLM exhibits significant outliers when directly adopting the LNS format. In this paper, we present a data-format/architecture codesign to bright this gap. On the format side, we propose a dynamic LNS format to flexibly represent outliers at a higher precision, by exploiting asymmetry in the LNS representation and identifying outliers through a per-vector basis. On the architecture side, for demonstration, we realize the dynamic LNS format in a systolic array, which can handle the irregularity of the outliers at runtime. We implement our approach on an Alveo U280 FPGA as a prototype. Experimental results show that our design can effectively handle the outliers and resolve the mismatch between LNS and LLM, contributing to an accuracy improvement of 15.4% and 16% over the floating-point and the original LNS baselines, using four state-of-the-art LLM models. Our observation and design lay a solid foundation for the large-scale adoption of the LNS format in the next-generation deep learning hardware.

Haghi, Pouya↗
DOE OSTI2025

Physics-informed latent neural operator for real-time predictions of time-dependent parametric PDEs

Deep operator network (DeepONet) has shown significant promise as surrogate models for systems governed by partial differential equations (PDEs), enabling accurate mappings between infinite-dimensional function spaces. However, when applied to systems with high-dimensional input-output mappings arising from large numbers of spatial and temporal collocation points, these models often require heavily overparameterized networks, leading to long training times. Latent DeepONet addresses some of these challenges by introducing a two-step approach: first learning a reduced latent space using a separate model, followed by operator learning within this latent space. While efficient, this method is inherently data-driven and lacks mechanisms for incorporating physical laws, limiting its robustness and generalizability in data-scarce settings. Here, in this work, we propose PI-Latent-NO, a physics-informed latent neural operator framework that integrates governing physics directly into the learning process. Our architecture features two coupled DeepONets trained end-to-end: a Latent-DeepONet that learns a low-dimensional representation of the solution, and a Reconstruction-DeepONet that maps this latent representation back to the physical space. By embedding PDE constraints into the training via automatic differentiation, our method eliminates the need for labeled training data and ensures physics-consistent predictions. The proposed framework is both memory and compute-efficient, exhibiting near-constant scaling with problem size and demonstrating significant speedups over traditional physics-informed operator models. We validate our approach on a range of parametric PDEs, showcasing its accuracy, scalability, and suitability for real-time prediction in complex physical systems.

Latent representations↗
DOE OSTI2025

Navigating the Noise: Bringing Clarity to ML Parameterization Design With O $\boldsymbol{\mathcal{O}}$(100) Ensembles

Abstract Machine‐learning (ML) parameterizations of subgrid processes (here of turbulence, convection, and radiation) may one day replace conventional parameterizations by emulating high‐resolution physics without the cost of explicit simulation. However, uncertainty about the relationship between offline and online performance (i.e., when integrated with a large‐scale general circulation model) hinders their development. Much of this uncertainty stems from limited sampling of the noisy, emergent effects of upstream ML design decisions on downstream online hybrid simulation. Our work rectifies the sampling issue via the construction of a semi‐automated, end‐to‐end pipeline for size ensembles of hybrid simulations, revealing important nuances in how systematic reductions in offline error manifest in changes to online error and online stability. For example, removing dropout and switching from a Mean Squared Error to a Mean Absolute Error loss both reduce offline error, but they have opposite effects on online error and online stability. Other design decisions, like incorporating memory, converting moisture input from specific humidity to relative humidity, using batch normalization, and training on multiple climates do not come with any such compromises. Finally, we show that ensemble sizes of may be necessary to reliably detect causally relevant differences online. By enabling rapid online experimentation at scale, we can empirically settle debates regarding subgrid ML parameterization design that would have otherwise remained unresolved in the noise.

Lin, Jerry [Department of Earth System Sciences Un↗
DOE OSTI2015

Spin-glass-like freezing of inner and outer surface layers in hollow γ-Fe 2 O 3 nanoparticles

Disorder among surface spins largely dominates the magnetic response of ultrafine magnetic particle systems. In this work, we examine time-dependent magnetization in high-quality, monodisperse hollow maghemite nanoparticles (NPs) with a 14.8±0.5 nm outer diameter and enhanced surface-to-volume ratio. The nanoparticle ensemble exhibits spin-glass-like signatures in dc magnetic aging and memory protocols and ac magnetic susceptibility. The dynamics of the system slow near 50 K, and becomes frozen on experimental time scales below 20 K. Remanence curves indicate the development of magnetic irreversibility concurrent with the freezing of the spin dynamics. A strong exchange-bias effect and its training behavior point to highly frustrated surface spins that rearrange much more slowly than interior spins with bulk coordination. Monte Carlo simulations of a hollow particle reproducing the experimental morphology corroborate strongly disordered surface layers with complex energy landscapes that underlie both glass-like dynamics and magnetic irreversibility. Calculated hysteresis loops reveal that magnetic behavior is not identical at the inner and outer surfaces, with spins at the outer surface layer of the 15 nm hollow particles exhibiting a higher degree of frustration. Lastly, our combined experimental and simulated results shed light on the origin of spin-glass-like phenomena and the important role played by the surface spins in magnetic hollow nanostructures.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2013

Fast MAP Search for Compact Additive Tree Ensembles (CATE)

Since their inception, additive ensembles have been shown to provide a highly effective and robust frame-work for a variety of classification and prediction tasks. Their successful application in a practical setting, however, usually depends on specifying a priori several operational parameters that can be difficult or time-consuming to estimate. In particular, the number of constituent models (e.g., decision trees, logistic models) that comprise the ensemble induces a tradeoff between performance and efficiency that may not be readily characterized a priori. This dichotomy varies with several key factors (e.g., the nature of the underlying models, availability of training data, etc.) and is especially critical in contexts where memory is at a premium (e.g., limited hardware [19]) and/or computational speed is paramount (e.g., real-time stock trading, cybersecurity). We have frequently observed that the classification performance of the ensemble tends to plateau (i.e., achieve saturation) as the number of members in the ensemble (i.e., ensemble size) increases, so that eventually the marginal gain from adding further models falls below a desired tolerance. This observation suggests that superior performance may be achieved via overspecification of the ensemble size. In practice, however, this strategy can easily produce simultaneously redundant and overly complex ensembles at a significant expense to computational efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Day-Ahead Forecasting with Federated LSTM to Plan Energy Sharing in a Community Microgrid

Energy balancing in microgrids is a key enabler of resilience. Community microgrids located close to each other have the added benefit of networking and sharing surplus energy, if available. Such complex decision-making runs on optimization that requires reliable short-term (up to very-short-term) forecasts of energy generation and consumption for scheduling or trading. Each microgrid may also opt to not expose their sensitive data such as consumption patterns of individual businesses or residences. This paper investigates a federated approach to dayahead forecasting that trains naive long short-term memory (LSTM) at each business in a microgrid and aggregates weights at the microgrid controller using proximal regularization. This approach ensures that the controller has access only to energy surplus/deficit and not the actual generation or consumption values, avoiding unwanted exposure of sensitive data. A community microgrid in Adjuntas, Puerto Rico with 3 businesses is selected as a case study with a laboratory-scale computing setup. A central LSTM forecaster, where sensitive data from businesses are aggregated at the controller, is implemented as a baseline for qualifying the results. This work serves as a proof-of-concept for scaling the approach to networked and nested microgrids with more complex control options.

Sundararajan, Aditya [ORNL] (ORCID:000000033577854↗
DOE OSTI2022

Design and Challenges of Edge Computing ASICs on Front-End Electronics

In situ or hardware-embedded data processing of raw signals, close to their source, in radiation detectors is expected to provide dramatic improvements in data quality and volumes. However, the implementation of artificial neural networks (ANNs) in the front-end electronics, and the design of custom integrated circuits (ASICs), comes with challenges. In addition, detectors have to operate with limited power budget and implement complex functionalities in a very dense space. They often are exposed to extreme conditions as they work in high-radiation environments and/or cryogenic temperatures. This paper presents examples of applications and design methodologies for in-situ ANNs, along with the challenges of retaining the fidelity of the trained networks. For illustration, we use the problem of estimating the energy deposited by the radiation from digitized waveforms. The proposed implementation starts with an ML algorithm trained in Qkeras and eventually leads to an equivalent ASIC implementation. The associated design challenges in realizing energy and area efficient implementations in CMOS processes are reviewed. Novel approaches that employ hybrid technologies (combination of CMOS with memristors), in-memory computing models, and bio-inspired spiking neural networks are also highlighted

47 OTHER INSTRUMENTATION↗
DOE OSTI2019

A deep learning approach to anomaly detection in geological carbon sequestration sites using pressure measurements

Carbon capture and storage (CCS) has been extensively investigated as a potential engineering measure to reduce anthropogenic carbon emission to the atmosphere. Real-time monitoring of the safety and integrity of carbon storage reservoirs is a critical aspect of any commercial-scale CCS deployment. Pressure-based sensing is cost effective, suitable for real-time monitoring, and scalable to large monitoring networks. However, questions remain on how to best harness intelligent information from the high-frequency pressure monitoring sensors to support real-time decisions. Here this work presents a deep-learning-based framework for analyzing and detecting anomalies in pressure data streams by using a convolutional long short-term memory (ConvLSTM) neural network model, which allows for the fusion of both static and dynamic reservoir data. In ConvLSTM, the convolutional neural network (CNN) is used for spatial pattern mining and the LSTM is used for temporal pattern recognition. The performance of the ConvLSTM model for real-time anomaly detection is demonstrated using a set of pressure monitoring data collected from Cranfield, Mississippi, an active enhanced-oil-recovery field. The anomaly detection model is trained using bottom-hole pressure data acquired from the base experiment (without leak event) and then tested on pressure data collected during a series of controlled CO 2 release experiments (with artificially created leak events). Results show that the ConvLSTM neural network model successfully detected anomalies in the pressure time series obtained from the controlled release experiments. Inclusion of static information into the model further improves the robustness of ConvLSTM.

13 HYDRO ENERGY↗
DOE OSTI2025

Efficient Training of Deep Neural Operator Networks via Randomized Sampling

Neural operators (NOs) employ deep neural networks to learn the mappings between infinitedimensional function spaces. Deep operator network (DeepONet), a popular NO architecture, has demonstrated success in the real-time prediction of complex dynamics across various scientific and engineering applications. In this work, we introduce a random sampling technique to be adopted during the training of DeepONet, aimed at improving the generalization ability of the model, while significantly reducing the computational time. The proposed approach targets the trunk network of the DeepONet model that outputs the basis functions corresponding to the spatiotemporal locations of the bounded domain on which the physical system is defined. While constructing the loss function, DeepONet training traditionally considers a uniform grid of spatiotemporal points at which all the output functions are evaluated for each iteration. This approach leads to a larger batch size, resulting in poor generalization and increased memory demands, due to the limitations of the stochastic gradient descent (SGD) optimizer. The proposed random sampling over the inputs of the trunk net mitigates these challenges, improving generalization and reducing the memory requirements during training, resulting in significant computational gains. We validate our hypothesis through three benchmark examples, demonstrating substantial reductions in training time while achieving comparable or lower overall test errors relative to the traditional training approach. Our results indicate that incorporating randomization in the trunk network inputs during training enhances the efficiency and robustness of DeepONet, offering a promising avenue for improving the framework’s performance in modeling complex physical systems.

Karumuri, Sharmila [Department of Civil & Systems ↗
NASA NTRS2008

Characterization of Ni19.5Ti50.5Pd25Pt5 High-Temperature Shape Memory Alloy Springs and their Potential Application in Aeronautics

Shape memory alloys (SMAs) have been used as actuators in many different industries since the discovery of the shape memory effect, but the use of SMAs as actuation devices in aeronautics has been limited due to the temperature constraints of commercially available materials. Consequently, work is being done at NASA's Glenn Research Center to develop new SMAs capable of being used in high temperature environments. One of the more promising high-temperature shape memory alloys (HTSMAs) is Ni19.5Ti50.5Pd25Pt5. Recent work has shown that this material is capable of being used in operating environments of up to 250 C. This material has been shown to have very useful actuation capabilities, demonstrating repeatable strain recoveries up to 2.5% in the presence of an externally applied load. Based on these findings, further work has been initiated to explore potential applications and alternative forms of this alloy, such as springs. Thus, characterization of Ni19.5Ti50.5Pd25Pt5 springs, including their mechanical response and how variations in this response correlate to changes in geometric parameters, are discussed. The effects of loading history, or training, on spring behavior were also investigated. A comparison of the springs with wire actuators is made and the benefits of using one actuator form as opposed to the other discussed. These findings are used to discuss design considerations for a surge-control mechanism that could be used in the centrifugal compressor of a T-700 helicopter engine.

Stebner, Aaron↗
NASA NTRS2008

Effect of Upper-Cycle Temperature on the Load-Biased, Strain-Temperature Response of NiTi

Over the past decade, interest in shape memory alloy based actuators has increased as the primary benefits of these solid-state devices have become more apparent. However, much is still unknown about the characteristic behavior of these materials when used in actuator applications. Recently we have shown that the maximum temperature reached during thermal cycling under isobaric conditions could significantly affect the observed mechanical response of NiTi (55 wt% Ni), especially the amount of transformation strain available for actuation and thus work output. This investigation extends that original work to ascertain whether further increases in the upper-cycle temperature would produce additional improvement in the work output of the material, which has a stress-free Af of 113 oC, and to determine the optimum cyclic conditions. Thus, isobaric, thermal-cycle experiments were conducted in the aforementioned alloy at various stress levels from 50-300 MPa using upper-cycle temperatures of 165, 200, 230, 260, 290, 320 and 350 oC. The data indicated that the amount of applied stress influenced the transformation strain available in the system, as would be expected. However, the maximum temperature reached during the thermal excursion also plays a role in determining the transformation strain, with the maximum transformation strain being developed by thermal cycling to 290 oC. In situ, neutron diffraction showed that the differences in transformation strain were related to differences in martensite texture within the microstructure when cycling to different upper-cycle temperatures. Hence, understanding this effect is important to optimizing the operation of SMA-based actuators and could lead to new methods for processing and training shape memory alloys for optimal performance.

Padula, Santo, II↗
NASA NTRS2011

Effect of Upper-Cycle Temperature on the Load-Biased, Strain-Temperature Response of NiTi

Over the past decade, interest in shape memory alloy based actuators has increased as the primary benefits of these solid-state devices have become more apparent. However, much is still unknown about the characteristic behavior of these materials when used in actuator applications. Recently we have shown that the maximum temperature reached during thermal cycling under isobaric conditions could significantly affect the observed mechanical response of NiTi (55 wt% Ni), especially the amount of transformation strain available for actuation and thus work output. The investigation we report here extends that original work to ascertain whether further increases in the upper-cycle temperature would produce additional changes in the work output of the material, which has a stress-free austenite finish temperature of 113 C, and to determine the optimum cyclic conditions. Thus, isobaric, thermal-cycle experiments were conducted on the aforementioned alloy at various stresses from 50-300 MPa using upper-cycle temperatures of 165, 200, 230, 260, 290, 320 and 350 C. The data indicated that the amount of applied stress influenced the transformation strain, as would be expected. However, the maximum temperature reached during the thermal excursion also plays an equally significant role in determining the transformation strain, with the maximum transformation strain observed during thermal cycling to 290 C. In situ neutron diffraction at stress and temperature showed that the differences in transformation strain were mostly related to changes in martensite texture when cycling to different upper-cycle temperatures. Hence, understanding this effect is important to optimizing the operation of SMA-based actuators and could lead to new methods for processing and training shape memory alloys for optimal performance.

Padula, Santo, II↗
DOE OSTI2021

Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration

To support the increasing demands for efficient deep neural network processing, accelerators based on analog in-memory computation of matrix multiplication have recently gained significant attention for reducing the energy of neural network inference. However, analog processing within memory arrays must contend with the issue of parasitic voltage drops across the metal interconnects, which distort the results of the computation and limit the array size. This work analyzes how parasitic resistance affects the end-to-end inference accuracy of state-of-the-art convolutional neural networks, and comprehensively studies how various design decisions at the device, circuit, architecture, and algorithm levels affect the system's sensitivity to parasitic resistance effects. Here, a set of guidelines are provided for how to design analog accelerator hardware that is intrinsically robust to parasitic resistance, without any explicit compensation or re-training of the network parameters.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Deep Learning Modeling Framework to Capture Mixing Patterns in Reactive-Transport Systems

Prediction and control of chemical mixing are vital for many scientific areas such as subsurface reactive transport, climate modeling, combustion, epidemiology, and pharmacology. Due to the complex nature of mixing in heterogeneous and anisotropic media, the mathematical models related to this phenomenon are not analytically tractable. Numerical simulations often provide a viable route to predict chemical mixing accurately. However, contemporary modeling approaches for mixing cannot utilize available spatial-temporal data to improve the accuracy of the future prediction and can be compute-intensive, especially when the spatial domain is large and for long-term temporal predictions. To address this knowledge gap, in this work we will present in this paper a deep learning (DL) modeling framework applied to predict the progress of chemical mixing under fast bimolecular reactions. This framework uses convolutional neural networks (CNN) for capturing spatial patterns and long short-term memory (LSTM) networks for forecasting temporal variations in mixing. By careful design of the framework—placement of non-negative constraint on the weights of the CNN and the selection of activation function, the framework ensures non-negativity of the chemical species at all spatial points and for all times. Our DL-based framework is fast, accurate, and requires minimal data for training. The time needed to obtain a forecast using the model is a fraction (≈ O(-6)) of the time needed to obtain the result using a high-fidelity simulation. To achieve an error of 10% (measured using the infinity norm) for capturing local-scale mixing features such as interfacial mixing, only 24% to 32% of the sequence data for model training is required. To achieve the same level of accuracy for capturing global-scale mixing features, the sequence data required for model training is 64% to 70% of the total spatial-temporal data. Hence, the proposed approach—a fast and accurate way to forecast long-time spatial-temporal mixing patterns in heterogeneous and anisotropic media—will be a valuable tool for modeling reactive-transport in a wide range of applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRSDate not supplied

A Machine Learning Ready Dataset of Acoustic Power Maps for Detection of Active Region Emergence

The development of an accurate forecast for solar eruptive activity has become increasingly important in order to prevent any potential impact on activities in space and the Earth's environment. It is therefore crucial to detect active regions before they appear on the solar surface and create early warning capabilities for upcoming Space Weather disturbances. In this work, 9TB of solar data (SDO/HMI dopplergrams, magnetograms and continuum intensity maps) involving the emergence of 61 NOAA solar active regions since 2010 were processed using the NASA HECC capabilities. An acoustic power maps time-series dataset was created (for four different frequency ranges and processed to take into account the solar sphere geometric effect ) which can be used for understanding the dynamics of the solar surface and train a variety of ML models. The calculated acoustic power maps carry precursor information associated with the decrease in continuum intensity on the solar surface, verifying older helioseismology research. Our results show that a Long Short-Term Memory (LSTMs) model, with a modest layer depth and the right hyperparameters tuned, when trained on this solar acoustic power maps dataset can predict without false negatives a drop in intensity (associated with the emergence of the active region), up to 18 hours in advance.

SMD↗
DOE OSTI2021

Transfer learning of memory kernels for transferable coarse-graining of polymer dynamics

The present work concerns the transferability of coarse-grained (CG) modeling in reproducing the dynamic properties of the reference atomistic systems across a range of parameters. In particular, we focus on implicit-solvent CG modeling of polymer solutions. The CG model is based on the generalized Langevin equation, where the memory kernel plays the critical role in determining the dynamics in all time scales. Thus, we propose methods for transfer learning of memory kernels. The key ingredient of our methods is Gaussian process regression. By integration with the model order reduction via proper orthogonal decomposition and the active learning technique, the transfer learning can be practically efficient and requires minimum training data. Through two example polymer solution systems, we demonstrate the accuracy and efficiency of the proposed transfer learning methods in the construction of transferable memory kernels. The transferability allows for out-of-sample predictions, even in the extrapolated domain of parameters. Built on the transferable memory kernels, the CG models can reproduce the dynamic properties of polymers in all time scales at different thermodynamic conditions (such as temperature and solvent viscosity) and for different systems with varying concentrations and lengths of polymers.

Ma, Zhan↗
NASA NTRS2001

Stabilized NADH as a Countermeasure for Jet Lag

Current remedies for jet lag (phototherapy, melatonin, stimulant, and sedative medications) are limited in efficacy and practicality. The efficacy of a stabilized, sublingual form of reduced nicotin amide adenine dinucleotide (NADH, ENADAlert, Menuco Corp.) as a countermeasure for jet lag was examined. Because NADH increases cellular production of ATP and facilitates dopamine synthesis, it may counteract the effects of jet lag on cognitive functioning and sleepiness. Thirty-five healthy, employed subjects participated in this double-blind, placebo-controlled study. Training and baseline testing were conducted on the West Coast before subjects flew overnight to the East Coast, where they would experience a 3-hour time difference. Upon arrival, individuals were randomly assigned to receive either 20 mg of sublingual stabilized ADH (n=18) or identical placebo tablets (n=17). All participants completed computer-administered tests (including CogScreen7) to assess changes in cognitive functioning, mood, and sleepiness in the morning and afternoon. Jet lag resulted in increased sleepiness for over half the participants and deterioration of cognitive functioning for approximately one third. The morning following the flight, subjects experienced lapses of attention in addition to disruptions in working memory, divided attention, and visual perceptual speed. Individuals who received NADH performed significantly better on 5 of 8 cognitive and psychomotor test measures (P less than or equal to 0.5) and showed a trend for better performance on the other three measures (P less than or equal to .l0). Subjects also reported less sleepiness compared with those who received placebo. No adverse effects were observed with NADH treatment. Stabilized NADH significantly reduced jet lag-induced disruptions of cognitive functioning, was easily administered, and was found to have no adverse side effects.

Kay, Gary G.↗
DOE OSTI2022

Dimensionally Reduced Model for Rapid and Accurate Prediction of Gas Saturation, Pressure, and Brine Production in a CO 2 Storage Application: Case Study Using the SACROC Field as Part of SMART Task 5

This technical report presents work conducted by the sub surface analysis team of the Strategic Systems Analysis & Engineering group at NETL for Task 5 of SMART Phase 1. This study involved the development of deep learning models for CO 2 geologic storage that are capable of accurate prediction of spatio-temporal outputs of CO 2 saturation, pressure, and brine production in three dimensional space over a storage operation's injection and post-injection timeframes. The model framework involves ensembling multi-layer encoder networks that provide dimesionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction This approach offers a means to maximize training time efficiency, reduce computational memory burden, and minimize prediction turnaround.

54 ENVIRONMENTAL SCIENCES↗