Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Predicting fault slip via transfer learning

Abstract Data-driven machine-learning for predicting instantaneous and future fault-slip in laboratory experiments has recently progressed markedly, primarily due to large training data sets. In Earth however, earthquake interevent times range from 10’s-100’s of years and geophysical data typically exist for only a portion of an earthquake cycle. Sparse data presents a serious challenge to training machine learning models for predicting fault slip in Earth. Here we describe a transfer learning approach using numerical simulations to train a convolutional encoder-decoder that predicts fault-slip behavior in laboratory experiments. The model learns a mapping between acoustic emission and fault friction histories from numerical simulations, and generalizes to produce accurate predictions of laboratory fault friction. Notably, the predictions improve by further training the model latent space using only a portion of data from a single laboratory earthquake-cycle. The transfer learning results elucidate the potential of using models trained on numerical simulations and fine-tuned with small geophysical data sets for potential applications to faults in Earth.

58 GEOSCIENCES↗
DOE OSTI2026

Integrating Data Centers and Grid Technologies at Scale

This presentation focuses on the challenge of integrating AI-driven data centers with the power grid at scale. It examines the AI data center capacity challenge and the role of new Medium Voltage Direct Current (MVDC) and other grid-enhancing technologies in enabling efficient and reliable power delivery. The session will highlight the National Laboratory of the Rockies' ARIES capabilities and planning tools, along with collaborative examples involving Verrus, Compass, and Schneider through the Agora test bed for grid-friendly data center evaluations, and ON. Energy for UPS evaluation. It will showcase the NLR Stable Grid Platform for studying oscillations caused by large-scale data centers, along with planning tools to assess grid security and reliability. Additionally, the presentation covers reconductoring strategies to increase grid capacity and explores innovative data center architectures, including the Advanced DC Architectures with Power-electronic Transformers (ADAPT) platform, which enables testing of complete DC architectures for data centers.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

A Unified Data Infrastructure for Biological and Environmental Research: A Report from the BER Advisory Committee

The Biological and Environmental Research (BER) program within the U.S. Department of Energy (DOE) Office of Science supports large-scale data generation efforts across its two divisions: Biological Systems Science and Earth and Environmental Systems Sciences. These efforts include user facilities in atmospheric radiation measurements, genomics, metabolomics, proteomics, compute, and imaging. In addition, BER supports the development of plant-based fuels; research in biosystems design, environmental microbiomes, and atmospheric systems; energy flux monitoring; climate-based ecosystem experiments; pathogen biopreparedness; and modeling of climate, urban interfaces, and interactions between people and energy resources. For data access, BER supports community data services at its user facilities, along with specialized data initiatives for Earth and environmental science, climate modeling, genomic and microbial analysis, and multisector dynamics modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Training data selection for event classification in a highly variable environment

A problem of interest for nuclear nonproliferation is monitoring activities at nuclear facilities, where proliferation events may only take place a few times and often under variable conditions. Machine learning has revolutionized data analytics by enabling the use of measurable signatures to generate predictive models of facility operations. However, traditional methods for training these models require large, reliable data sets with labeled observations, a challenge for nonproliferation. Highly variable conditions further complicate this as events from training data may have occurred in conditions quite different from the event of interest. Our hypothesis is that when events occur in a highly variable environment, careful training data selection for each test event could outperform the standard approach of using all available training data. We developed a method to optimize training data selection for the given test event and applied it to predicting the power level of the High Flux Isotope Reactor (HFIR) at Oak Ridge National Laboratory. In this study, the reactor startup exhibits variability between occurrences due to natural variability in environmental conditions and operational procedures. Using a combination of analysis techniques, a similitude assessment was performed on data collected from HFIR to isolate clusters that were optimal for training a predictive model. Concepts such as dynamic time warping and Jaccard similarity were used in conjunction with clustering analysis. In order to validate this approach, the model was trained on every combination of unique training events and the predictive performance was compared to the performance using a subset of the training data selected by isolated clusters found through the similitude assessment.

Iyer, A↗
DOE OSTI2020

Dimensionality Reduction of SDSS Spectra with Variational Autoencoders

High-resolution galaxy spectra contain much information about galactic physics, but the high dimensionality of these spectra makes it difficult to fully utilize the information they contain. We apply variational autoencoders (VAEs), a nonlinear dimensionality reduction technique, to a sample of spectra from the Sloan Digital Sky Survey (SDSS). In contrast to principal component analysis (PCA), a widely used technique, VAEs can capture nonlinear relationships between latent parameters and the data. We find that a VAE can reconstruct the SDSS spectra well with only six latent parameters, outperforming PCA with the same number of components. Different galaxy classes are naturally separated in this latent space, without class labels having been given to the VAE. The VAE latent space is interpretable because the VAE can be used to make synthetic spectra at any point in latent space. For example, making synthetic spectra along tracks in latent space yields sequences of realistic spectra that interpolate between two different types of galaxies. Using the latent space to find outliers may yield interesting spectra: in our small sample, we immediately find unusual data artifacts and stars misclassified as galaxies. In this exploratory work, we show that VAEs create compact, interpretable latent spaces that capture nonlinear features of the data. While a VAE takes substantial time to train (≈1 day for 48,000 spectra), once trained, VAEs can enable the fast exploration of large astronomical data sets.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Learning-Based Real-Time Event Identification Using Rich Real PMU Data

A large-scale deployment of phasor measurement units (PMUs) that reveal the inherent physical laws of power systems from a data perspective enables an enhanced awareness of power system operation. However, the high-granularity and non-stationary nature of PMU data and imperfect data quality could bring great technical challenges for real-time system event identification. To address these challenges, this paper proposes a two-stage learning-based framework. In the first stage, a Markov transition field (MTF) algorithm is exploited to extract the latent data features by encoding temporal dependency and transition statistics of PMU data in graphs. Then, a spatial pyramid pooling (SPP)-aided convolutional neural network (CNN) is established to efficiently and accurately identify power events. The proposed method fully builds on and is also tested on a large real-world dataset from several tens of PMU sources (and the corresponding event logs), located across the U.S., with a time span of two consecutive years. We report the numerical results validate that our method has high identification accuracy while showing good robustness against poor data quality.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Extracting Material Property Measurements from Scientific Literature with Limited Annotations

Extracting material property data from scientific text is pivotal for advancing data-driven research in chemistry and materials science; however, the extensive annotation effort required to produce training data for named entity recognition (NER) models for this task often makes it a barrier to extracting specialized data sets. Here, in this work, we present a comparative study of the conventional, supervised NER methodology to alternative few-shot learning architectures and large language model (LLM)-based approaches that mitigate the need to label large training data sets. We find that the best-performing LLM (GPT-4o) not only excels in directly extracting relevant material properties based on limited examples but also enhances supervised learning through data augmentation. We supplement our findings with error and data quality assessments to provide a nuanced understanding of factors that impact property measurement extraction.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Building Large-Scale U.S. Synthetic Electric Distribution System Models

Rapid increases in distributed energy resources on distribution systems are prompting research efforts to improve and evaluate electric power distribution algorithms; however, there is a shortage of realistic, large-scale, U.S.-style test systems for the evaluation of such advanced algorithms. Some available tools to build large-scale test systems are of European style, and their application to studies focused on the United States might not be desirable given very different characteristics between the European and U.S. distribution designs. Motivated by this need, this paper develops detailed algorithms to build large-scale U.S. distribution systems and incorporates them in a new Reference Network Model, RNM-US. The approach starts with information from street maps and a catalog with electric equipment that includes power lines, transformers, voltage regulators, capacitors, and switching devices. The paper presents the algorithms through an illustrative case study of the different steps that comprise the process of building a synthetic distribution grid. Finally, the paper presents a medium- and a large-scale data set covering 10 million electrical nodes and 120,000 km of power lines, demonstrating the applicability of the proposed method to build very large-scale synthetic distribution systems.

27 ARPA - Advanced Research Projects Agency-Energy↗
DOE OSTI2024

Neutron skins: A perspective from dispersive optical models

An overview of neutron skin predictions obtained using an empirical nonlocal dispersive optical model (DOM) is presented. The DOM links both scattering and bound-state experimental data through a subtracted dispersion relation which allows for fully consistent, data-informed predictions for nuclei where such data exist. Large skins were predicted for both 48 Ca ( R$^{48}_{skin}$ = 0.25 ± 0.023 fm in 2017) and 208 Pb (R$^{208}_{skin}$) = 0.25 ± 0.05 fm in 2020). Whereas the DOM prediction in 208 Pb is within 1σ of the subsequent PREX-2 measurement, the DOM prediction in 48 Ca is over 2σ larger than the thin neutron skin resulting from CREX. From the moment it was revealed, the thin skin in 48 Ca has puzzled the nuclear-physics community as no adequate theories simultaneously predict both a large skin in 208 Pb and a small skin in 48 Ca. The DOM is unique in its ability to treat both structure and reaction data on the same footing, providing a unique perspective on this R skin puzzle. It appears vital that more neutron data be measured in both the scattering and bound-state domain for 48 Ca to clarify the situation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials

In computed TEM tomography, image segmentation represents one of the most basic tasks with implications not only for 3D volume visualization, but more importantly for quantitative 3D analysis. In case of large and complex 3D data sets, segmentation can be an extremely difficult and laborious task, and thus has been one of the biggest hurdles for comprehensive 3D analysis. Heterogeneous catalysts have complex surface and bulk structures, and often sparse distribution of catalytic particles with relatively poor intrinsic contrast, which possess a unique challenge for image segmentation, including the current state-of-the-art deep learning methods. To tackle this problem, we apply a deep learning-based approach for the multi-class semantic segmentation of a γ-Alumina/Pt catalytic material in a class imbalance situation. Specifically, we used the weighted focal loss as a loss function and attached it to the U-Net’s fully convolutional network architecture. We assessed the accuracy of our results using Dice similarity coefficient (DSC), recall, precision, and Hausdorff distance (HD) metrics on the overlap between the ground-truth and predicted segmentations. Our adopted U-Net model with the weighted focal loss function achieved an average DSC score of 0.96 ± 0.003 in the γ-Alumina support material and 0.84 ± 0.03 in the Pt NPs segmentation tasks. We report an average boundary-overlap error of less than 2 nm at the 90th percentile of HD for γ-Alumina and Pt NPs segmentations. The complex surface morphology of γ-Alumina and its relation to the Pt NPs were visualized in 3D by the deep learning-assisted automatic segmentation of a large data set of high-angle annular dark-field (HAADF) scanning transmission electron microscopy (STEM) tomography reconstructions.

36 MATERIALS SCIENCE↗
DOE OSTI2024

DASEventNet: AI‐Based Microseismic Detection on Distributed Acoustic Sensing Data From the Utah FORGE Well 16A (78)‐32 Hydraulic Stimulation

Abstract Distributed acoustic sensing (DAS) has emerged as a promising seismic technology for monitoring microearthquakes (MEQs) with high spatial resolution. Efficient algorithms are needed for processing large DAS data volumes. This study introduces a deep learning (DL) model based on a Residual Convolutional Neural Network (ResNet) for detecting MEQs using DAS data, named as DASEventNet. The test data were collected from the Utah FORGE 16A (78)‐32 hydraulic stimulation experiments conducted in April 2022. The DASEventNet model achieves a remarkable accuracy of 100% when discriminating MEQs from noise in the raw test set of 260 examples. Surprisingly, the model identified weak MEQ signatures that have been manually categorized as noise. The decision‐making process with the model is decoded by the classic activation map, which illuminates learning features of the DASEventNet model. These features provide clear illustrations of weak MEQs and varied noise types. Finally, we apply the trained model to the entire period (∼7 days) of continuous DAS recordings and find that it discovers >5,700 new MEQs, previously unregistered in the public Silixa DAS catalog. The DASEventNet model significantly outperforms the traditional seismic method Short‐Term Average/Long‐Term Average (STA/LTA), which detected only 1,307 MEQs. The DASEventNet detection threshold is M w −1.80 compared to the minimum magnitude of M w −1.14 detected by STA/LTA. The spatiotemporal distribution of the newly identified MEQs defines an extensive stimulation zone and more accurately characterizes fracture geometry. Our results highlight the potential of DL for long‐term, real‐time microseismic monitoring that can improve enhanced geothermal systems and other activities that include subsurface hydraulic fracturing.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

A review of privacy in energy applications

As the distribution system continues to experience an increase in distributed energy resource (DER) and electric vehicle (EV) penetration, so does the need for new solutions that can help grid operators manage and leverage their capabilities. This will undoubtedly lead to new operational schemes and business opportunities that will transform the traditional consumer into a prosumer who will be more actively engaged in grid operations. Although the field is still under active development, many of the potential use cases presented in literature or industry are built upon edge computing, two-way communications, and other innovative computational constructs to attain their goals. However, at their core, many use cases assume a great level of data access to aid with the decision-making process, an assumption that may need to be revised to ensure fair and equitable operational processes are maintained. This may be particularly true as edge resources are predicted to participate in retail-side, many-to-many, or peer-to-peer markets and thus may lead to financial impacts if data access considerations are ignored. The need to revise data access mechanisms can be further justified by the introduction of new participants into the operational process, who do not have the same level of trust, nor the incentives to focus on energy delivery as their primary objective. At the same time, more consumers are becoming aware of their own data, and the potential impacts of its abuse. To help solution developers better understand these risks, this report has been developed to offer an initial introduction to the topic of privacy. This is achieved by 1) Highlighting the need for privacy-aware solutions; 2) Encouraging system designers to be inquisitive about the status quo; 3) Documenting the existing threat space; 4) Presenting and evaluating tools that may be helpful towards enabling better privacy postures; and 5) Making recommendations to encourage the adoption of better practices. From a technical perspective, the report focuses on evaluating two potential techniques by applying them to the Transactive Energy Space. Based on the obtained results, it can be established that differential privacy (DP) methods may have limited applicability when highly correlated, time-series data records need to be protected. However, DP may be a powerful tool when it is used to aggregate and analyze mid-size and large-size data sets in a more traditional statistical environment. The second tool under evaluation is threshold cryptography, which can guarantee complete secrecy (and thus privacy) but requires the establishment of key management procedures and dedicated communication channels for key coordination. Therefore, due to its increased computational overhead, the use of threshold cryptography must be weighted using a cost/benefit analysis on a per-application basis.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Eureka: Enabling Fine-Grained Access and Range Queries on Compressed Scientific Data via Data-Index Co-Compression

Handling large-scale scientific data in high-performance computing (HPC) environments poses significant challenges, including excessive I/O, high storage costs, and slow query performance. Traditional approaches often require full data decompression and scans, making them impractical for real-time or interactive analysis. To address these limitations, we introduce Eureka, a unified data-index co-compression framework that enables fine-grained access and efficient range queries on compressed scientific datasets. Eureka integrates spatial domain decomposition with block-wise error-bounded lossy compression to support selective decompression. It constructs a hierarchical AVL-tree index during compression to capture block-level value ranges, enabling fast pruning during query execution. To reduce metadata overhead, the index itself is also compressed while ensuring recall-preserving results. Experiments on six diverse HPC simulation datasets show that Eureka achieves up to 25x data compression and over 300x index compression, surpassing state-of-the-art compressors such as SZ3 and ZFP in rate-distortion performance. Additionally, Eureka delivers over 30x speedup for low-selectivity range queries, making it a scalable and efficient solution for modern scientific data analysis.

Yan, Ning↗
DOE OSTI2021

COTS Data Analytics Software User Manual: Version 1.0

Large volumes of data are being collected by Sandia National Laboratories as part of an active commercial-off-the-shelf (COTS) part testing and surveillance program. This user manual documents Python-based COTS Data Analytics software that has been developed for standardizing, displaying, visualizing, and analyzing the resulting COTS part testing and surveillance data. It is the objective of these software tools to streamline the analysis of COTS testing and surveillance data and improve the efficiency with which test engineers and data analytics experts can pinpoint possible performance and reliability problems in COTS parts.

42 ENGINEERING↗
DOE OSTI2021

Fast Extraction and Characterization of Fundamental Frequency Events from a Large PMU Dataset using Big Data Analytics

A novel method for fast extraction of fundamental frequency events (FFE) based on measurements of frequency and rate of change of frequency by Phasor Measurement Units (PMU) is introduced. The method is designed to work with exceptionally large historical PMU datasets. Statistical analysis was used to extract the features and train Random Forest and Catboost classifiers. The method is capable of fast extraction of FFE from a historical dataset containing measurements from hundreds of PMUs captured over multiple years. The reported accuracy of the best algorithm for classification expressed as Area Under the receiver operating Characteristic curve reaches 0.98, which was obtained in out-of-sample evaluations on 109 system-wide events over 2 years observed at 43 PMUs. Then Minimum Volume Enclosing Ellipsoid Algorithm was used to further analyze the events. 93.72% events were correctly characterized, where average duration of the event as seen by the PMU was 9.93 sec.

Baembitov, Rashid↗
DOE OSTI2022

A nearsighted force-training approach to systematically generate training data for the machine learning of large atomic structures

A challenge of atomistic machine-learning (ML) methods is ensuring that the training data are suitable for the system being simulated, which is particularly challenging for systems with large numbers of atoms. Most atomistic ML approaches rely on the nearsightedness principle (“all chemistry is local”), using information about the position of an atom’s neighbors to predict a per-atom energy. Here, in this work, we develop a framework that exploits the nearsighted nature of ML models to systematically produce an appropriate training set for large structures. We use a per-atom uncertainty estimate to identify the most uncertain atoms and extract chunks centered around these atoms. It is crucial that these small chunks are both large enough to satisfy the ML’s nearsighted principle (that is, filling the cutoff radius) and are large enough to be converged with respect to the electronic structure calculation. We present data indicating when the electronic structure calculations are converged with respect to the structure size, which fundamentally limits the accuracy of any nearsighted ML calculator. These new atomic chunks are calculated in electronic structures, and crucially, only a single force—that of the central atom—is added to the growing training set, preventing the noisy and irrelevant information from the piece’s boundary from interfering with ML training. The resulting ML potentials are robust, despite requiring single-point calculations on only small reference structures and never seeing large training structures. We demonstrated our approach via structure optimization of a 260-atom structure and extended the approach to clusters with up to 1415 atoms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Accelerating Structure–Property Relationship Discovery with Multimodal Machine Learning and Self-Driving Microscopy

Microscopy combined with local spectroscopy is widely used to correlate nanoscale structure with functional properties in materials, but conventional measurements rely heavily on human-selected sampling locations and predefined targets, limiting data set diversity and the potential for discovery. Here, we present a framework that integrates autonomous microscopy with dual-novelty deep kernel learning (DN-DKL) for adaptive data acquisition and a dual variational autoencoder (VAE) for representation learning. DN-DKL actively guides the microscopy toward structurally and spectroscopically novel regions, enabling efficient collection of large spectral data sets. Dual-VAE embeds local structures and spectroscopic responses into a shared latent manifold that serves as a structure–property relationship map. We applied this framework for the investigation of halide perovskite films by using conductive atomic force microscopy. The results reveal distinct hysteresis behaviors that are linked to specific nanoscale structural motifs, including grain boundary junction points that show hysteresis under different bias conditions and asymmetric grain boundaries that suppress the charge transport. This framework establishes a general strategy that leverages the complementary strengths of self-driving microscopy, machine learning, and human expertise to accelerate scientific discovery in functional materials.

atomic force microscopy↗
DOE OSTI2023

Sister Rod Destructive Examinations (FY23) Appendix F3: Uncertainty and Conservative Bias in the Cyclic Integrated Reversible-Bending Fatigue Test

The development and application of the Cyclic Integrated Reversible-Bending Fatigue Tester (CIRFT) has been documented over the past 10 years. The CIRFT system was developed in collaboration with the U.S. Nuclear Regulatory Commission (NRC) to address the concern that high-burnup fuel rod fatigue performance was degraded compared to cladding-only behavior. Fatigue data measured by CIRFT on high-burnup fuel rods were then used by the NRC to develop a fatigue response curve for pressurized water reactor fuel rods with Zircaloy-4 cladding in NUREG-2224. Subsequently, CIRFT has been used in the sister rod project to test whether reoriented hydrides associated with vacuum drying influenced the fatigue lifetime, to extend the fatigue testing database to include fuel rods with ZIRLO cladding, and to add to the experience on both Zircaloy-4 and M5 cladding. These data will be used as a reference for subsequent characterization of fuel rods removed from the high-burnup demonstration cask to better understand the effects of long-term dry storage, which can be thought of as a multiyear elevated temperature anneal. The objective of fatigue testing is to provide data that can be used to develop a best-estimate fatigue curve and a conservative fatigue design curve that accounts for uncertainty and can be compared with actual transportation conditions. The process for defining a design fatigue curve for commercial used fuel rods was developed by Oak Ridge National Laboratory (ORNL) in FY22. One issue noted in the fatigue evaluation was that the calculated combined uncertainty in the strain amplitude values determined from the measured data was large. It was proposed then to directly measure the uncertainty in the strain amplitude to either validate the calculated combined uncertainty or update the uncertainty based on the measurement data. Thus, the purpose of this appendix is to document the measured uncertainty in the strain amplitude data. In the process, it was determined that there has been a conservative bias in the calculated strain amplitude that is small enough to be within the measurement uncertainty. This observation is discussed herein, and a case is made to continue to include the conservative bias in the data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗