Search NASA⌕ Search

SEARCH · Search NASA

Results for “Working memory training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗
DOE OSTI2016

Limited‐memory adaptive snapshot selection for proper orthogonal decomposition

Summary Reduced order models are useful for accelerating simulations in many‐query contexts, such as optimization, uncertainty quantification, and sensitivity analysis. However, offline training of reduced order models (ROMs) can have prohibitively expensive memory and floating‐point operation costs in high‐performance computing applications, where memory per core is limited. To overcome this limitation for proper orthogonal decomposition, we propose a novel adaptive selection method for snapshots in time that limits offline training costs by selecting snapshots according an error control mechanism similar to that found in adaptive time‐stepping ordinary differential equation solvers. The error estimator used in this work is related to theory bounding the approximation error in time of proper orthogonal decomposition‐based ROMs, and memory usage is minimized by computing the singular value decomposition using a single‐pass incremental algorithm. Results for a viscous Burgers' test problem demonstrate convergence in the limit as the algorithm error tolerances go to zero; in this limit, the full‐order model is recovered to within discretization error. A parallel version of the resulting method can be used on supercomputers to generate proper orthogonal decomposition‐based ROMs, or as a subroutine within hyperreduction algorithms that require taking snapshots in time, or within greedy algorithms for sampling parameter space. Copyright © 2016 John Wiley & Sons, Ltd.

Oxberry, Geoffrey M.↗
DOE OSTI2024

A crystal-plasticity-informed Gaussian Process Regression model to capture anisotropy in single crystal shape memory alloys

This work presents a machine learning (ML) framework that model the anisotropic actuation responses in a shape memory alloy. A Gaussian Process Regression (GPR) based ML model is trained on a set of different crystal orientations subjected to different actuation conditions. The training employed thermo-mechanical responses from a crystal-plasticity model that captures phase-transformation, stress-induced plasticity, and transformation-induced plasticity. Further, on training the GPR-ML model at fixed stress level for different orientations, it captured the thermo-mechanical responses accounting for the anisotropy, and predicted responses for new orientations with good accuracy. The GPR-ML model is able to capture the transformation temperature variations even when trained using multiple stress levels, and the transformation strain showed significant deviations. The developed GPR-ML model gave reasonable predictions for an unexplored sample set of orientations and loading conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load

Scientific computing workloads at HPC facilities have been shifting from traditional numerical simulations to AI/ML applications for training and inference while processing and producing ever-increasing amounts of scientific data. To address the growing need for increased storage capacity, lower access latency, and higher bandwidth, emerging technologies such as non-volatile memory are integrated into supercomputer I/O subsystems. With these emerging trends, we need a better understanding of the multilayer supercomputer I/O systems and ways to use these subsystems efficiently. In this work, we study the I/O access patterns and performance characteristics of two representative supercomputer I/O subsystems. Through an extensive analysis of year-long I/O logs on each system, we report new observations in I/O reads and writes, unbalanced use of storage system layers, and new trends in user behaviors at the HPC I/O middleware stack.

Bez, JL↗
DOE OSTI2025

Properties of Electronic Materials

This final technical report summarizes the research conducted under DOE Grant DE-SC0002623, "Properties of Electronic Materials," led by Principal Investigator Shengbai Zhang at Rensselaer Polytechnic Institute. Over the 16-year period, the project employed first-principles computational methods to investigate the structural, electronic, and dynamic properties of a wide range of electronic materials, with applications in energy technologies, optoelectronics, and data storage. Key areas included topological insulators, phase-change materials, graphene and two-dimensional systems, perovskites for photovoltaics, defect engineering in semiconductors, kagome lattices, and ultrafast carrier dynamics. The research resulted in 115 peer-reviewed publications, advancing fundamental understanding of material behaviors at the atomic scale and contributing to innovations in renewable energy, memory devices, and quantum materials. Findings have implications for improving energy efficiency, developing lead-free solar cells, and enabling high-speed data processing. The work has trained numerous graduate students and postdocs, fostering the next generation of computational materials scientists. The original goals were to develop theoretical models and computational tools to predict and optimize electronic properties of materials for energy applications. All objectives were accomplished, with no major departures from planned methodologies. Challenges in computational scaling were addressed through access to high-performance computing resources.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Temperature outweighs light and flow as the predominant driver of dissolved oxygen in US rivers

The concentration of dissolved oxygen (DO), an important measure of water quality and river metabolism, varies tremendously in time and space. Riverine DO is commonly perceived as regulated by interacting and competing drivers (light, temperature and flow) that define rivers’ climate. Its continental-scale drivers, however, have remained elusive, partly due to the scarcity and spatio-temporal inconsistency of water quality data. Here we show, via a deep learning model (long short-term memory) trained using data from 580 rivers, that temperature predominantly drives daily DO dynamics in the contiguous United States. Light comes a close second, whereas flow imparts minimal influence. This work showcases the promise of using deep learning models for data filling that enables large-scale systematic analysis of patterns and drivers. Results show fairly accurate prediction of DO by temperature alone, and declining DO in warming rivers, which has important implications for water security and ecosystem health in the future climate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Teacher-student training improves the accuracy and efficiency of machine learning interatomic potentials

Machine learning interatomic potentials (MLIPs) are revolutionizing the field of molecular dynamics (MD) simulations. Recent MLIPs have tended towards more complex architectures trained on larger datasets. The resulting increase in computational and memory costs may prohibit the application of these MLIPs to perform large-scale MD simulations. Herein, we present a teacher-student training framework in which the latent knowledge from the teacher (atomic energies) is used to augment the students' training. We show that the light-weight student MLIPs have faster MD speeds at a fraction of the memory footprint compared to the teacher models. Remarkably, the student models can even surpass the accuracy of the teachers, even though both are trained on the same quantum chemistry dataset. Our work highlights a practical method for MLIPs to reduce the resources required for large-scale MD simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Embedding Neural Thermal Scattering (NeTS) Modules in SERPENT for Higher Fidelity Advanced Reactor Analysis

When a neutron born in fission thermalizes to the order of $k$ $B$ $T$, it’s de-Broglie wavelength and energy approach the order of inter-atomic spacing and elementary lattice oscillations, respectively. $S$($a,β,t$) or the scattering law, uuantify these temperature-dependent crystallographic contributions to total cross section (or reaction rate). In a Monte Carlo analysis, cumulative distribution functions (CDFs) of $S$($a,β,t$) are loaded to memory from “A Compact ENDF” (ACE) files for stochastically selecting thermal scattered neutron trajectories. In this work, novel neural thermal scattering (NeTS) modules for $S$($a,β,t$) CDFs are designed, trained, serialized and embedded within SERPENT using Python’s limited C-API for on-the-fly deployment of crystalline graphite $S$($a,β,t$) sampling. Torchscript tracing and Numba just-in-time (JIT) compilation streamline neural inference on NVIDIA GPUs with CUDA libraries. Demonstrations of bare sphere thermalization of fast and thermal sources show excellent agreement between embedded NeTS in SERPENT and MCNP. With an explicit model of the reactor, NeTS can predict on-the-fly changes in TREAT neutron spectra as a function of local temperature, which can serve to improve transient and accident predictions in a multiphysics analysis framework. This framework can be further extended to account on-the-fly for changes in local graphitic microstructure to scattering cross sections, and outlines a novel coupling of modern machine learning with state-of-the-art reactor physics methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Analysis and Benchmarking of feature reduction for classification under computational constraints

Abstract Machine learning is most often expensive in terms of computational and memory costs due to training with large volumes of data. Current computational limitations of many computing systems motivate us to investigate practical approaches, such as feature selection and reduction, to reduce the time and memory costs while not sacrificing the accuracy of classification algorithms. In this work, we carefully review, analyze, and identify the feature reduction methods that have low costs/overheads in terms of time and memory. Then, we evaluate the identified reduction methods in terms of their impact on the accuracy, precision, time, and memory costs of traditional classification algorithms. Specifically, we focus on the least resource intensive feature reduction methods that are available in Scikit-Learn library. Since our goal is to identify the best performing low-cost reduction methods, we do not consider complex expensive reduction algorithms in this study. In our evaluation, we find that at quadratic-scale feature reduction, the classification algorithms achieve the best trade-off among competitive performance metrics. Results show that the overall training times are reduced 61%, the model sizes are reduced 6×, and accuracy scores increase 25% compared to the baselines on average with quadratic scale reduction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Accurate Compression of Tabulated Chemistry Models with Partition of Unity Networks

Tabulated chemistry models are widely used to simulate large-scale turbulent fires in applications including energy generation and fire safety. Tabulation via piecewise Cartesian interpolation suffers from the curse-of-dimensionality, leading to a prohibitive exponential growth in parameters and memory usage as more dimensions are considered. Artificial neural networks (ANNs) have attracted attention for constructing surrogates for chemistry models due to their ability to perform high-dimensional approximation. However, due to well-known pathologies regarding the realization of suboptimal local minima during training, in practice they do not converge and provide unreliable accuracy. Partition of unity networks (POUnets) are a recently introduced family of ANNs which preserve notions of convergence while performing high-dimensional approximation, discovering a mesh-free partition of space which may be used to perform optimal polynomial approximation. In this work, we assess their performance with respect to accuracy and model complexity in reconstructing unstructured flamelet data representative of nonadiabatic pool fire models. Our results show that POUnets can provide the desirable accuracy of classical spline-based interpolants with the low memory footprint of traditional ANNs while converging faster to significantly lower errors than ANNs. For example, we observe POUnets obtaining target accuracies in two dimensions with 40 to 50 times less memory and roughly double the compression in three dimensions. We also address the practical matter of efficiently training accurate POUnets by studying convergence over key hyperparameters, the impact of partition/basis formulation, and the sensitivity to initialization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Software-Hardware Co-design of Heterogeneous SmartNIC System for Recommendation Models Inference and Training

Deep Learning Recommendation Models (DLRMs) are critical applications in various domains and have evolved as one of the single largest machine learning applications. Trillions of DLRM parameters exceed the on-chip memory capacity of GPUs. Large-scale multi-node systems are required for distributed DLRM inference and training, which suffer from the all-to-all communication bottleneck, mainly limiting the scalability of ever-growing DLRMs. In recent years, SmartNICs have evolved with coupled computation and communication capabilities providing opportunities for a powerful heterogeneous device in the system. However, there isn't such a distributed system that fully leverages the abundant smartNIC resources that resolve the scalability issue of DLRMs. In this work, we proposed a software-hardware co-design of a heterogeneous smartNIC system that resolves the communication bottleneck of distributed DLRMs, mitigates the memory bandwidth pressure, and improves computation efficiency. We provide a set of smartNIC designs of cache systems (including local cache and remote cache) and smartNIC computation kernels which reduce data movement, relieve memory lookup intensity, and improve the GPU's computation efficiency. In addition, we propose a graph algorithm that improves the data locality of queries within batches which optimizes the overall system performance with higher data reuse. Our evaluation shows that our system achieves 2.1x latency speedup for inference and 1.6x throughput speedup for training.

Guo, Anqi↗
NASA NTRS2013

From "Below Dignity" to "Above It All": Origins and Early History of Underwater Neutral Buoyancy Simulation of Weightlessness for EVA Procedures Development and Training

An attempt to clarify some vague memories of underwater studies of astronaut capabilities in space led Dr. John Charles to become acquainted with Sam Mattingly, one of the pioneers in the field, and to greater insights into Mattingly's work simulating Gemini EVAs in the mid-1960s. Charles recounted major accomplishments by Environmental Research Associates (ERA), Mattingly's company for contracting with NASA Langley on several early studies. ERA's work was considered within the context of contemporary efforts to simulate weightlessness and the widespread development of neutral buoyancy facilities after ERA's successful demonstration for Gemini 12.

Charles, John B.↗
DOE OSTI2018

User-level File Systems Specialized for HPC Workloads Year-End Report

High-performance computing (HPC) clusters attract the attention of Deep Learning (DL) training users due to the clusters' powerful computation capabilities. While there have been many existing efforts to enable deep neural works to leverage the powerful CPU and GPU processors from leadership high-performance computing (HPC) systems, large-scale deep learning with larger datasets requires efficient I/O support from the underlying file and storage systems. As some current and upcoming HPC clusters have large on-node memory or are equipped with NVMe SSD on compute nodes, more distributed DL trainings are considered to leverage those storages to store dataset for efficient dataset access. Our project goal is to design a specialized DL-oriented file system to improve datasets loading performance of any DL training applications. Over the past year, the team at Florida State University has performed research activities in three different aspects, including 1) completing a specialized memory-based I/O framework (DeepIO) for improving dataset loading performance on DL applications; 2) proposing a more generalized file system (DLFS) for DL applications on node-local SSDs; 3) completing the BeeGFS performance evaluation project for Deep Neural Networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A domain wall-magnetic tunnel junction artificial synapse with notched geometry for accurate and efficient training of deep neural networks

Inspired by the parallelism and efficiency of the brain, several candidates for artificial synapse devices have been developed for neuromorphic computing, yet a nonlinear and asymmetric synaptic response curve precludes their use for backpropagation, the foundation of modern supervised learning. Spintronic devices—which benefit from high endurance, low power consumption, low latency, and CMOS compatibility—are a promising technology for memory, and domain-wall magnetic tunnel junction (DW-MTJ) devices have been shown to implement synaptic functions such as long-term potentiation and spike-timing dependent plasticity. In this work, we propose a notched DW-MTJ synapse as a candidate for supervised learning. Using micromagnetic simulations at room temperature, we show that notched synapses ensure the non-volatility of the synaptic weight and allow for highly linear, symmetric, and reproducible weight updates using either spin transfer torque (STT) or spin–orbit torque (SOT) mechanisms of DW propagation. We use lookup tables constructed from micromagnetics simulations to model the training of neural networks built with DW-MTJ synapses on both the MNIST and Fashion-MNIST image classification tasks. Accounting for thermal noise and realistic process variations, the DW-MTJ devices achieve classification accuracy close to ideal floating-point updates using both STT and SOT devices at room temperature and at 400 K. Our work establishes the basis for a magnetic artificial synapse that can eventually lead to hardware neural networks with fully spintronic matrix operations implementing machine learning.

42 ENGINEERING↗
DOE OSTI2025

CRCNS22 Learning Rules in the Hippocampus and their Mapping to Neuromorphic Systems (Final Technical Report)

Large scale biologically-realistic computational models are key to investigating the interplay between structure and function in nervous systems, thus paving the way to new clinical methods and neuro-inspired computing solutions. This project focuses on the hippocampus, in particular the CA3-CA1 regions, due to their role in associative learning and memory, pattern separation and completion, and spatial navigation. Investigations into the neuronal organization and learning rule(s) of this circuit can shed light into how declarative memories are formed, stored, recalled and forgotten and inform computational, experimental and clinical neuroscience work. Our project aims at developing a novel data-driven methodology supported by a broad heterogeneous base of neuroscience experimental knowledge and inspired from advances in computer science and engineering. Specifically, this work will benchmark existing and new learning rules within a full-scale spiking neural network simulation of the CA3-CA1 region. The model will be based on an open-source repository, called the Hippocampome, which contains neuronal morphologies, firing patterns, synapse probabilities, and most other required parameters for all known neuron types in the rodent hippocampal formation. The model will be first trained in a supervised fashion for associative memory tasks using backpropagation through time traditionally used in computer science, enhanced with a new technique called the surrogate gradient method. This optimization method will be used to obtain a global loss minimization, but it is not biologically inspired as it assumes the use of data not locally available to the synapses. However, we propose its use as a benchmarking tool, to compare the training performance of local biologically plausible and hardware-mappable learning rules at scale. New rules or combinations will be proposed and tested as needed, based on the obtained results. Progress in this area will also drive the development of novel hardware-mappable algorithms for continual lifelong learning and categorization of new events from few presented examples. This project goes beyond the existing state-of-the-art by looking at large scale realistic neuronal circuits as networks trainable via global optimization methods such as surrogate gradient descent. The objective function of the brain that supports learning is largely unknown, but it is likely that it operates through local learning rules. Studying network trajectories around local minima as proposed in this work represents a useful strategy for understanding whether a network is training by using a specific (set of) learning rule(s). Starting from a completely untrained network is a challenging test since it is difficult to determine how the learning rule affects the trajectory of the network. This interdisciplinary project will help understand what rule governs learning in these regions or if multiple learning rules are involved. The work will develop a robust methodology to measure if the network is converging to the target solution, oscillating around it, or diverging away.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Dual-ion ECRAM as a stable and accurate analog synapse

Electrochemical random-access memory (ECRAM) works by tuning the bulk electronic conductance of functional materials via reversible, electrochemical insertion of ions, resulting in stable analog resistive switching, attractive for analog in-memory and neuromorphic computing. However, achieving fast programming for training and long retention for inference has been elusive. Protonic ECRAM demonstrates fast programming but insufficient retention, while oxygen-based ECRAM with excellent retention requires elevated programming temperatures. Cu-based ECRAM offers a compromise, with an activation energy (E A ) of ≈0.76 eV between protons (E A ≈ 0.4 eV) and oxygen (E A > 1 eV), enabling extensive retention and room temperature programming. Combining Cu 2+ ions with protons to form a dual-ion ECRAM, we demonstrate two distinct switching behaviors: fast switching at ≤5 V, (E A ≈ 0.45 eV) via protons, and nonvolatile, room temperature switching at ≥8 V, with E A ≈ 0.76 eV via Cu 2+ ions. In conclusion, the Cu-based state exhibits a wide conductance range, with excellent retention, low noise, and linear current-voltage behavior, achieving digital-equivalent ImageNet inference accuracy.

analog in-memory computing↗
DOE OSTI2019

Application of deep artificial neural networks to multi-dimensional flamelet libraries and spray flames

The “curse of dimensionality” has limited the applicability and expansion of tabulated combustion models. While the tabulated flamelet model and other multi-dimensional manifold approaches have shown predictive capability, the associated tabulation involves the storage of large lookup tables, requiring large memory as well as multi-dimensional interpolation subroutines, all implemented during runtime. As such, this work investigates the use of deep artificial neural networks to replace lookup tables in order to reduce the memory footprint and increase the computational speed of tabulated flamelets and related approaches. Specifically, different strategic approaches to training the artificial neural network models are explored and a grouped multi-target artificial neural network is introduced, which takes advantage of the ability of artificial neural networks to map an input space to multiple targets by classifying the species based on their correlation to one another. The grouped multi-target artificial neural network approach is validated by applying it to an n-dodecane spray flame using conditions of the Spray A flame from the Engine Combustion Network and comparing global flame characteristics for different ambient conditions using a well-established large-eddy simulation framework. The same framework is then extended to the simulations of methyl decanoate combustion in a compression ignition engine. The validation studies show that the grouped multi-target artificial neural networks are able to accurately capture flame liftoff, autoignition, two-stage heat release and other quantitative trends over a range of conditions. The use of neural networks in conjunction with the grouping mechanism as performed in the grouped multi-target artificial neural network produces a significant reduction in the memory footprint and computational costs for the code and, thus, widens the operating envelope for higher fidelity engine simulations with detailed mechanisms.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

Plasma membrane ordering agent pluronic F-68 (PF-68) reduces neurotransmitter uptake and release and produces learning and memory deficits in rats

A substantial body of evidence indicates that aged-related changes in the fluidity and lipid composition of the plasma membrane contribute to cellular dysfunction in humans and other mammalian species. In the CNS, reductions in neuronal plasma membrane order (PMO) (i.e., increased plasma membrane fluidity) have been attributed to age as well as the presence of the beta-amyloid peptide-25-35, known to play an important role in the neuropathology of Alzheimer's disease (AD). These PMO increases may influence neurotransmitter synthesis, receptor binding, and second messenger systems as well as signal transduction pathways. The effects of neuronal PMO on learning and memory processes have not been adequately investigated, however. Based on the hypothesis that an increase in PMO may alter a number of aspects of synaptic transmission, we investigated several neurochemical and behavioral effects of the membrane ordering agent, PF-68. In cell culture, PF-68 (nmoles/mg SDS extractable protein) reduced [3H]norepinephrine (NE) uptake into differentiated PC-12 cells as well as reduced nicotine stimulated [3H]NE release. The compound (800-2400 microg/kg, i.p., resulting in nmoles/mg SDS extractable protein in the brain) decreased step-through latencies and increased the frequencies of crossing into the unsafe side of the chamber in inhibitory avoidance training. In the Morris water maze, PF-68 increased the latencies and swim distances required to locate a hidden platform and reduced the time spent and distance swam in the previous target quadrant during transfer (probe) trials. PF-68 did not impair performance of a well-learned working memory task, the rat delayed stimulus discrimination task (DSDT), however. Studies with 14C-labeled PF-68 indicated that significant (pmoles/mg wet tissue) levels of the compound entered the brain from peripheral (i.p.) injection. No PF-68 related changes were observed in swim speeds or in visual acuity tests in water maze experiments, rotorod performance, or in tests of general locomotor activity. Furthermore, latencies to select a lever in the DSDT were not affected. These results suggest that PF-68 induced deficits in learning and memory without confounding peripheral motor, sensory, or motivational effects at the tested doses. Furthermore, none of the doses induced a conditioned taste aversion to a novel 0.1% saccharin solution indicating a lack of nausea or gastrointestinal malaise induced by the compound. The data indicate that increases in neuronal plasma membrane order may have significant effects on neurotransmitter function as well as learning and memory processes. Furthermore, compounds such as PF-68 may also offer novel tools for studying the role of neuronal PMO in mnemonic processes and changes in PMO resulting from age-related disorders such as AD.

Poloxamer/pharmacology↗