Search NASA⌕ Search

SEARCH · Search NASA

Results for “latent space optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

28 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Divertor Plasma Detachment Control Neural Network

DivControlNN is a state-of-the-art software tool that leverages advanced machine learning techniques to predict and control divertor plasma behavior in fusion reactors. Plasma, a highly energetic and electrically charged gas, requires meticulous management to protect reactor components and maintain optimal energy production. Conventional simulation methods, although extremely detailed, typically demand extensive computational time-making them unsuitable for real-time control scenarios. DivControlNN addresses this challenge by learning from tens of thousands of high-fidelity simulations, thereby creating a rapid surrogate model that can deliver near-instantaneous predictions. At the core of its functionality is a sophisticated technique known as latent space mapping, which condenses complex, high-dimensional plasma data into a compact, lower-dimensional representation. This streamlined representation enables the system to quickly forecast essential plasma properties and determine the precise conditions required for effective detachment. Detachment is a crucial process in which the plasma is cooled before reaching the divertor plates, thereby reducing heat loads and mitigating material erosion. In recent experiments conducted on the KSTAR tokamak in South Korea, DivControlNN successfully guided the detachment process without any fine-tuning-even when applied to a new tungsten divertor configuration. By achieving a computational speed-up of over one hundred million times compared to traditional simulation methods while maintaining low prediction errors, DivControlNN stands to significantly enhance real-time control and diagnostic capabilities in future fusion reactors. This breakthrough paves the way for safer, more reliable reactor operation and represents a major advancement toward realizing fusion energy as a practical, sustainable, and clean power source.

Xu, Xueqiao [Lawrence Livermore National Laborator↗
DOE OSTI2025

Understanding latent timescales in neural ordinary differential equation models of advection-dominated dynamical systems

The neural ordinary differential equation (ODE) framework has shown considerable promise in recent years in developing highly accelerated surrogate models for complex physical systems characterized by partial differential equations (PDEs). For PDE-based systems, state-of-the-art neural ODE strategies leverage a two-step procedure to achieve this acceleration: a nonlinear dimensionality reduction step provided by an autoencoder, and a time integration step provided by a neural-network based model for the resultant latent space dynamics (the neural ODE). This work explores the applicability of such autoencoder-based neural ODE strategies for PDEs in which advection terms play a critical role. More specifically, alongside predictive demonstrations, physical insight into the sources of model acceleration (i.e., how the neural ODE achieves its acceleration) is the scope of the current study. Such investigations are performed by quantifying the effects of both autoencoder and neural ODE components on latent system time-scales using eigenvalue analysis of dynamical system Jacobians. To this end, the sensitivity of various critical training parameters – de-coupled versus end-to-end training, latent space dimensionality, and the role of training trajectory length, for example – to both model accuracy and the discovered latent system timescales is quantified. Furthermore, this work specifically uncovers the key role played by the training trajectory length (the number of rollout steps in the loss function during training) on the latent system timescales: larger trajectory lengths correlate with an increase in limiting neural ODE time-scales, and optimal neural ODEs are found to recover the largest time-scales of the full-order (ground-truth) system. Demonstrations are performed across fundamentally different unsteady fluid dynamics configurations influenced by advection: (1) the Kuramoto–Sivashinsky equations (2) Hydrogen-Air channel detonations (the compressible reacting Navier–Stokes equations with detailed chemistry), and (3) 2D Atmospheric flow.

Advection-dominated dynamical systems↗
DOE OSTI2025

Enhancing Unknown Waveform Detection by Learning Intra and Inter-domain Dependencies with Advanced Attention Fusion Mechanisms

Detection of unknown waveforms in mission-critical communications is a crucial area of interest for the Department of Energy (DoE). Traditional methods and recent deep learning-based approaches often assume that the training set includes all possible classes, which is impractical for detecting new waveforms. This limitation gives rise to the problem of open-set recognition (OSR), which involves correctly identifying known classes while detecting and rejecting unknown or unseen classes. To address this limitation, we propose a novel dual-domain complex-valued neural architecture that jointly processes time-domain and frequency-domain signal representations using transformer mechanisms. A transformer model is a deep learning architecture that uses self-attention mechanisms to process and learn relationships in sequential data. Our model employs a cosine similarity loss to extract domain-specific features and incorporates a transformer architecture in the latent space to weigh the importance of different features from the time and frequency domains. The transformer layer includes stacked self-attention and cross-attention modules to learn intra-domain and inter-domain dependencies, creating a more holistic signal representation. An attention-based fusion module intelligently combines the time and frequency-domain features using multi-head attention, enabling the network to learn the optimal feature for each domain in each input signal. Quantitative results demonstrate the impact of these architectural choices on overall performance, showing significant improvement after incorporating self and cross-attention modules and using complex attention fusion over simple weighted fusion. Our ongoing work will focus on addressing the limitations of threshold-based OSR methods by developing a novel generative framework that integrates a conditional diffusion probabilistic model (DPM). DPM is a generative framework that learns to synthesize complex data by reversing a gradual noising process using a neural network trained to denoise step-by-step. Our goal is to leverage the inherent strengths of DPMs for identifying unknown signals more robustly. One primary advantage of using a DPM is its ability to provide a more reliable anomaly score based on the model's reconstruction error, rather than relying solely on classifier confidence. Additionally, the iterative denoising process of DPMs makes this approach naturally resilient to low Signal-to-Noise Ratio (SNR) conditions, where traditional methods often fail. By implementing this generative framework, we aim to enhance the model's capability to accurately detect unknown waveforms and maintain performance in challenging environments.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗
DOE OSTI2025

Active Learning Surrogates for Integrating Electron Microscopy and Computational Insights from Simulations in Autonomous Experiments

Artificial Intelligence (AI) combined with simulations and experiments has great potential to accelerate scientific discovery across technology and pharmaceuticals. However, the gap between simulations and experiments is challenging due to disparities in time and scale, making it difficult to estimate properties like energy and electronic states from experiments, and to provide feedback based on theoretical insights.Our research addresses the challenge by developing unique deep kernel based surrogate models that learns from microscopic images, mapping structural features to energy differences from defect formation. We start with full-training using simulated images to determine optimal settings, establishing a baseline for active learning. Using these settings from the baseline, active learning is trained, and predicts structures along simulation trajectories based on uncertainty and energetic stability, thus reducing data requirements, simulation time and computational costs. The results demonstrate that the model achieves a low average error margin of approximately 0.03 meV, indicating good performance. To enhance feature extraction and reconstruction capabilities, we developed an autoencoder-decoder as additional surrogate to create latent space to capture essential features, enabling precise comparisons between simulations and experiments. The results from this model achieved a reconstruction loss of around 0.2 and accurately reconstructed molecular structures.Overall, this work advances the steering of experiments through computational simulations by employing a surrogate models that actively predicts the trajectories of structural evolution, achieving time-to-solution comparable to experimental measurements.

Saranathan, Gayathri [Hewlett-Packard]↗
DOE OSTI2025

FPGA-accelerated SpeckleNN with SNL for real-time X-ray single-particle imaging

We present the implementation of a specialized version of our previously published unified embedding model, SpeckleNN, for real-time speckle pattern classification in X-ray Single-Particle Imaging (SPI), using the SLAC Neural Network Library (SNL) on an FPGA platform. This hardware realization transitions SpeckleNN from a prototypic model into a practical edge solution, optimized for running inference near the detector in high-throughput X-ray free-electron laser (XFEL) facilities, such as those found at the Linac Coherent Light Source (LCLS). To address the resource constraints inherent in FPGAs, we developed a more specialized version of SpeckleNN. The original model, which was designed for broader classification across multiple biological samples, comprised ~5.6 million parameters. The new implementation, while reducing the parameter count to 64.6K (a 98.8% reduction), focuses on maintaining the model's essential functionality for real-time operation, achieving an accuracy of 90%. Furthermore, we compressed the latent space from 128 to 50 dimensions. This implementation was demonstrated on the KCU1500 FPGA board, utilizing 71% of available DSPs, 75% of LUTs, and 48% of FFs, with an average power consumption of 9.4W according to the Vivado post-implementation report. The FPGA performed inference on a single image with a latency of 45.015 microseconds at a 200 MHz clock rate. In comparison, running the same inference on an NVIDIA A100 GPU resulted in an average power consumption of ~73W and an image processing latency of around 400 microseconds. Our FPGA-accelerated version of SpeckleNN demonstrated significant improvements, achieving an 8.9 × speedup and a 7.8 × reduction in power consumption compared to the GPU implementation. Key advancements include model specialization and dynamic weight loading through SNL, which eliminates the need for time-consuming FPGA design re-synthesis, allowing fast and continuous deployment of models (re)trained online. These innovations enable real-time adaptive classification and efficient vetoing of speckle patterns, making SpeckleNN more suited for deployment in XFEL facilities. This implementation has the potential to significantly accelerate SPI experiments and enhance adaptability to evolving experimental conditions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Reducing model error using optimized galaxy selection: weak lensing cluster mass estimation

Galaxy clusters are one of the most powerful probes to study extensions of General Relativity and the Standard Cosmological Model. Upcoming surveys like the Vera Rubin Observatory’s Legacy Survey of Space and Time are expected to revolutionise the field, by enabling the analysis of cluster samples of unprecedented size and quality. To reach this era of high-precision cluster cosmology, the mitigation of sources of systematic error is crucial. A particularly important challenge is bias in cluster mass measurements induced by the mismodelling of photometric redshift estimates of source galaxies. This work proposes a method to optimise the source sample selection in cluster weak lensing analyses drawn from wide-field survey lensing catalogs to reduce the bias on reconstructed cluster masses. We use a combinatorial optimisation scheme and methods from variational inference to select galaxies in latent space to produce a probabilistic galaxy source sample catalog for highly accurate cluster mass estimation. We show that our method reduces the critical surface mass density Σ crit relative modelling bias on the 60-70% level, while maintaining up to 90% of galaxies. We highlight that our methodology has applications beyond cluster mass estimation as an approach to jointly combine galaxy selection and model inference under sources of systematics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Design and performance assessment of a dual-mode latent thermal storage integrated heat pump across multiple climate zones

The electrification of space cooling and heating systems risks overloading the existing electrical grid during peak hours. Heat pumps integrated with thermal energy storage (HP-TES) offer a promising solution by shifting peak loads to off-peak hours, reducing grid strains. This work presents the design and assessment of a dual-mode HP-TES that uses a single 22°C phase-change material (PCM) as a heat source or sink in heating and cooling modes, respectively. Performance assessment was conducted using Modelica-based transient models, and full-year simulations were conducted across eight US climate zones using typical weather data. Two HP-TES performance metrics: peak energy reduction and recharge energy increases, were defined by comparing HP-TES energy consumption with base HP peak consumptions. Annual heating demand reductions (40–65%) exceeded cooling demand reductions (15–20%) due to the elimination of peak-hour backup heating. Cooling mode recharge energy requirements (0–10%) were reduced in locations with lower summer nighttime temperatures, while heating recharge requirements were lower than the high base-peak energy demand from backup heating. Simplified rapid methods that were 10 7 times faster than annual simulations were developed to predict seasonal cooling and heating energy reductions, and recharge energy requirements, with maximum deviations of ±2.5% and ±3.5% points. These methods enabled rapid parametric studies that identified optimal location-specific PCM temperatures between 22°C and 27°C, highlighting the need to consider both discharge and recharge energy requirements to achieve sustainable, energy-efficient peak load shifting across various climate zones.

25 ENERGY STORAGE↗
DOE OSTI2024

Development of Low-Cost, High-Performance, Easy-To-Apply, Non-Flammable, Inorganic Phase Change Material (PCM) Technology (Project Final Report)

This report describes a 45-months long research program focused on the development of novel, easy-to-apply, non-flammable, and high-performance inorganic phase change materials (PCMs) for building and industrial applications. The University of Massachusetts Lowell (UML) formed a world-class team consisting of researchers form InsolCorp (only N. American manufacturer of inorganic PCM systems for building applications), and a group of industrial advisors, to develop a universal/multipurpose, simple-to-manufacture and cost-effective PCM technology. The project team expects that the results of this work will spur in the future the adoption of thermal storage materials – a key building energy saving technology as identified by DOE BTO – for a variety of building envelope applications. The main goal of this project was to demonstrate a suite of low-cost, multipurpose, and durable inorganic PCM formulations with phase transition temperatures encompassing typical building applications (between +5 o C and +55 o C). The first objective was to design, fabricate, and experimentally validate a performance of inexpensive, durable, highly efficient, non-flammable, and easy to manufacture PCMs. To allow a variety of building applications, the project team focused on formulations that exhibit repeatable phase transitions between +5 o C and +55 o C. To follow the DOE BTO cost efficiency target without compromising thermal performance, our work was based on inorganic compounds (mostly salt hydrates) and their blends, which represent a fraction of the cost of most of organic PCMs with about twice as high density as well as significantly higher thermal conductivity and phase change enthalpy. The second objective was to develop easy-to-manufacture and -install packaging/encapsulation designs that are 1) a superior barrier to current state-of-the-art macro-packaging, which significantly reduces the risk of loss of hydration water and PCM leak, and 2) optimal in enhancing the heat exchange rates with the surroundings and within the PCM core to ensure complete charging/discharging of the entire PCM within the product. Finally, the project’s intend was to scale-up the fabrication process to demonstrate installation on system-scale applications, and to validate the performance under field conditions. This work aimed at developing low-cost, high-energy storage, and reliable latent heat storage technology for building applications. This development was realized by formulating and integrating the following two technology components: 1) inorganic salt hydrate based PCMs that have high latent enthalpies and are low-cost and durable, and 2) PCM encapsulation (packaging) technology that maximizes PCM concentration and enhances heat transport characteristics in the product and with the external environment/materials. High thermal storage capacity, low cost and fire resistance are key to the building market entry for PCM technology. Therefore, the project’s focus was on salt-hydrate-based formulations which satisfy all these criteria. Packaging and/or encapsulation of PCM is a key processing step. The project team recognized that a low-cost and simple-to-manufacture salt hydrate-based PCM technology holds the best chance to be successful in the building construction market, a market which is traditionally extremely sensitive to cost and where commodity thermal insulations are the benchmark for envelope-related energy saving measures. That is why, in this project, the main intention was to minimize the production cost and maximize the product energy storage density without sacrificing the PCM performance. It was achieved through: 1. Minimizing the non-PCM components (plastics, additives, packaging/encapsulation materials, etc.) because they are significantly more expensive than salt hydrates, 2. Using highly thermally conductive and lightweight PCM carrier (packaging material) to facilitate more complete phase cycling, and 3. Optimizing the thickness and minimizing air spaces in product design (such as in pouched PCM). For this purpose, our approach was to enable an easy system design, including selection of the PCM operating temperatures, optimizing the necessary heat storage capacity (by stacking together several layers of PCM products), and if needed, a synchronized usage of PCM products of different temperatures. A specially designed, robust, highly thermally conducting and highly impermeable packaging (to retain salt hydrate water during phase transition cycles) was designed and tested to increase the overall system thermal performance and durability. All PCM products developed during this project were tested in both lab scale and in full scale field conditions. It is expected that, after further developments and commercialization, the developed PCM technologies may be also applied in space conditioning, energy storage technologies, and heat transfer applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Contrastive Machine Learning with Gamma Spectroscopy Data Augmentations for Detecting Shielded Radiological Material Transfers

Data analysis techniques can be powerful tools for rapidly analyzing data and extracting information that can be used in a latent space for categorizing observations between classes of data. Machine learning models that exploit learned data relationships can address a variety of nuclear nonproliferation challenges like the detection and tracking of shielded radiological material transfers. The high resource cost of manually labeling radiation spectra is a hindrance to the rapid analysis of data collected from persistent monitoring and to the adoption of supervised machine learning methods that require large volumes of curated training data. Instead, contrastive self-supervised learning on unlabeled spectra can enhance models that are built on limited labeled radiation datasets. This work demonstrates that contrastive machine learning is an effective technique for leveraging unlabeled data in detecting and characterizing nuclear material transfers demonstrated on radiation measurements collected at an Oak Ridge National Laboratory testbed, where sodium iodide detectors measure gamma radiation emitted by material transfers between the High Flux Isotope Reactor and the Radiochemical Engineering Development Center. Label-invariant data augmentations tailored for gamma radiation detection physics are used on unlabeled spectra to contrastively train an encoder, learning a complex, embedded state space with self-supervision. A linear classifier is then trained on a limited set of labeled data to distinguish transfer spectra between byproducts and tracked nuclear material using representations from the contrastively trained encoder. The optimized hyperparameter model achieves a balanced accuracy score of 80.30%. Any given model—that is, a trained encoder and classifier—shows preferential treatment for specific subclasses of transfer types. Regardless of the classifier complexity, a supervised classifier using contrastively trained representations achieves higher accuracy than using spectra when trained and tested on limited labeled data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗