Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Data association algorithm for large-scale multi-object tracking with complex interactions

We present an online multi-object tracking algorithm to track multiple objects across a large number of image frames. Our work is motivated by the need to study evolution of nanoscale objects by transmission electron microscopy. The proposed approach is based on the existing multi-way data association tracking algorithm that is capable of tracking interacting objects with complex behaviors (i.e., merge, split, overlap, and appearance or disappearance). The multi-way data association is an offline algorithm to associate objects across all image frames at one step with a global optimization, which does not scale very well for large number of image frames. The proposed online tracking algorithm processes image frames as they arrive by detecting all objects in the newly arrived image frame and making the associations of the objects to those detected from the previous frame by the multi-way data association. This frameby-frame association scheme can cause fragmented traces of the objects that are occasionally misdetected for some image frames. We overcome this issue by allowing previously unassociated objects to be associated when the objects reappear within a fixed number of future image frames, namely the frame-delayed association. We combine the multi-way data association with the frame-delayed association to be able to track interacting objects with accurate handling of object disappearance events. The proposed method is validated through applications to simulated multi-object tracking problem and a real multi-object tracking problem. Here, the outcome of the proposed method is compared with four state-of-the-art algorithms.

36 MATERIALS SCIENCE↗
DOE OSTI2023

DDStore: Distributed Data Store for Scalable Training of Graph Neural Networks on Large Atomistic Modeling Datasets

Graph neural networks (GNNs) are a class of Deep Learning models used in designing atomistic materials for effective screening of large chemical spaces. To ensure robust prediction, GNN models must be trained on large volumes of atomistic data on leadership class supercomputers. Even with the advent of modern architectures that consist of multiple storage layers that include node-local NVMe devices in addition to device memory for caching large datasets, extreme-scale model training faces I/O challenges at scale.We present DDStore, an in-memory distributed data store designed for GNN training on large-scale graph data. DDStore provides a hierarchical, distributed, data caching technique that combines data chunking, replication, low-latency random access, and high throughput communication. DDStore achieves near-linear scaling for training a GNN model using up to 1000 GPUs on the Summit and Perlmutter supercomputers, and reaches up to a 6.15x reduction in GNN training time compared to state-of-the-art methodologies.

Choi, Jong Youl↗
DOE OSTI2024

Energy dataset of Frontier supercomputer for waste heat recovery

The Hewlett Packard Enterprise–Cray EX Frontier is the world’s first and fastest exascale supercomputer, hosted at the Oak Ridge Leadership Computing Facility in Tennessee, United States. Frontier is a significant electricity consumer, drawing 8–30 MW; this massive energy demand produces significant waste heat, requiring extensive cooling measures. Although harnessing this waste heat for campus heating is a sustainability goal at Oak Ridge National Laboratory (ORNL), the 30 °C–38 °C waste heat temperature poses compatibility issues with standard HVAC systems. Heat pump systems, prevalent in residential settings and some industries, can efficiently upgrade low-quality heat to usable energy for buildings. Thus, heat pump technology powered by renewable electricity offers an efficient, cost-effective solution for substantial waste heat recovery. However, a major challenge is the absence of benchmark data on high-performance computing (HPC) heat generation and waste heat profiles. This paper reports power demand and waste heat measurements from an ORNL HPC data centre, aiming to guide future research on optimizing waste heat recovery in large-scale data centres, especially those of HPC calibre.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Indications for Freeze-Out of Charge Fluctuations in the Quark-Gluon Plasma at the LHC

The D-measure of net-charge fluctuations quantifies the variance of net charge in strongly interacting matter. It was introduced over 20 years ago as a potential signal of quark-gluon plasma (QGP) in heavy-ion collisions, where it is expected to be suppressed due to the fractional electric charges of quarks. Measurements have been performed at RHIC and LHC, but the conclusion has been elusive in the absence of quantitative calculations for both scenarios. We address this issue by employing a recently developed formalism of density correlations and incorporate resonance decays, local charge conservation, and experimental kinematic cuts. We find that the hadron gas scenario is in fair agreement with the ALICE data for $\sqrt{{𝑠}_{\textrm{NN}}}$ = 2.76 TeV Pb–Pb collisions only when a very short rapidity range of local charge conservation is enforced, while the QGP scenario is in excellent agreement with experimental data and largely insensitive to the range of local charge conservation. A Bayesian analysis of the data utilizing different priors yields moderate evidence for the freeze-out of charge fluctuations in the QGP phase relative to hadron gas. The upcoming high-fidelity measurements from LHC Run 2 will serve as a precision test of the two scenarios.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Elliptically-Contoured Tensor-variate Distributions with Application to Image Learning

Statistical analysis of tensor-valued data has largely used the tensor-variate normal (TVN) distribution that may be inadequate for data arising from distributions with heavier or lighter tails. We study a general family of elliptically contoured (EC) TV distributions and derive its characterizations, moments, marginal, and conditional distributions. We describe procedures for maximum likelihood estimation from data that are (1) uncorrelated draws from an EC distribution, (2) from a scale mixture of the TVN distribution, and (3) from an underlying but unknown EC distribution, for which we extend Tyler’s robust estimator. A detailed simulation study highlights the benefits of choosing an EC distribution over the TVN for heavier-tailed data. We develop TV classification rules using discriminant analysis and EC errors and show that they better predict cats and dogs from images in the Animal Faces-HQ dataset than the TVN-based rules. A novel tensor-on-tensor regression and TV analysis of variance (TANOVA) framework under EC errors is also demonstrated to better characterize gender, age, and ethnic origin than the usual TVN-based TANOVA in the celebrated labeled faces of the wild dataset.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Sharing as a Catalyst for Expanding the Energy Frontier

As the energy landscape evolves to include technologies such as geothermal energy, comprehensive data become essential for driving innovation and scalability, particularly with the growing use of tools like machine learning and artificial intelligence. In emerging sectors, the cost of gathering high-quality data across large spatial areas can present a significant barrier. A key solution is leveraging existing data from well-established industries like oil and gas. However, the proprietary nature of data in these industries often hinders collaboration. This paper explores how cultivating a culture of data sharing can act as a catalyst for progress, fueling breakthroughs across both conventional and renewable energy sectors. Practical compromises that protect business interests while enabling data access are proposed, and real-world success stories are highlighted, demonstrating how collaboration has accelerated advancements in geothermal, carbon capture, and other innovative technologies.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

ROBIN (RuciO/BIgData Express/SENSE) A Next-Generation High-Performance Data Service Platform

Increasingly, scientific research discoverydepends on the analysis of extremely large amounts of data. Thisdata is gathered through very large scientific instruments, verylarge numbers of small devices, and mid-scale instruments.These devices generate exponentially increasing volumes of data.Such data is often obtained, analyzed, stored, visualized, andtransferred at and among sites world-wide by largecollaborations of science communities. Managing and movingextremely large volumes of data across today’s networks is aspecial multidimensional challenge. Also, existing datamanagement and movement services and tools often areinadequate to address all specific requirements. This paperdescribes the various components of this challenge and howthose issues can be addressed by ROBIN, a uniquecomprehensive set of integrated services designed specificallyfor managing and moving extremely large amounts of data overlong distances, e.g., thousands of miles around the globe. Thispaper also describes the results of initial experiments using thatset of integrated services.

Wu, Wenji↗
DOE OSTI2024

MITgcm-AD v2: Open source tangent linear and adjoint modeling framework for the oceans and atmosphere enabled by the Automatic Differentiation tool Tapenade

The Massachusetts Institute of Technology General Circulation Model (MITgcm) is widely used by the climate science community to simulate planetary atmosphere and ocean circulations. A defining feature of the MITgcm is that it has been developed to be compatible with an algorithmic differentiation (AD) tool, TAF, enabling the generation of tangent-linear and adjoint models. These provide gradient information which enables dynamics-based sensitivity and attribution studies, state and parameter estimation, and rigorous uncertainty quantification. Importantly, gradient information is essential for computing comprehensive sensitivities and performing efficient large-scale data assimilation, ensuring that observations collected from satellites and in-situ measuring instruments can be effectively used to optimize a large uncertain control space. As a result, the MITgcm forms the dynamical core of a key data assimilation product employed by the physical oceanography research community: Estimating the Circulation and Climate of the Ocean (ECCO) state estimate. Although MITgcm and ECCO are used extensively within the research community, the AD tool TAF is proprietary and hence inaccessible to a large proportion of these users. The new version 2 (MITgcm-AD v2) framework introduced here is based on the source-to-source AD tool Tapenade, which has recently been open-sourced. Another feature of Tapenade is that it stores required variables by default (instead of recomputing them) which simplifies the implementation of efficient, AD-compatible code. The framework has been integrated with the MITgcm model’s main branch and is now freely available.

Adjoints↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same domain, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications, which rely on large quantities of data from multiple sources. To address these challenges, the open-source Time-Series Data Pipelines (Tsdat) Python framework was developed by Pacific Northwest National Laboratory, with strategic guidance and direction provided by the National Renewable Energy Laboratory and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the marine energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat framework and the data standards within which it operates. A beta version of Tsdat has been released and is being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Intelligent Manufacturing Support: Specialized LLMs for Composite Material Processing and Equipment Operation

Engineering educational curriculum and standards cover many material and manufacturing options. However, engineers and designers are often unfamiliar with certain composite materials or manufacturing techniques. Large language models (LLMs) could potentially bridge the gap. Their capacity to store and retrieve data from large databases provides them with a breadth of knowledge across disciplines. However, their generalized knowledge base can lack targeted, industry-specific knowledge. To this end, we present two LLM-based applications based on the GPT-4 architecture: (1) The Composites Guide: a system that provides expert knowledge on composites material and connects users with research and industry professionals who can provide additional support and (2) The Equipment Assistant: a system that provides guidance for manufacturing tool operation and material characterization. By combining the knowledge of general AI models with industry-specific knowledge, both applications are intended to provide more meaningful information for engineers. In this paper, we discuss the development of the applications and evaluate it through a benchmark and two informal user studies. The benchmark analysis uses the Rouge and Bertscore metrics to evaluate our models’ performance against GPT-4o. The results show that GPT-4o and the proposed models perform similarly or better on the ROUGE and BERTScore metrics. The two user studies supplement this quantitative evaluation by asking experts to provide qualitative and open-ended feedback about our model’s performance on a set of domain-specific questions. The results of both studies highlight a potential for more detailed and specific responses with the Composites Guide and the Equipment Assistant.

Kapoor, Gunnika [Oak Ridge National Laboratory (OR↗
DOE OSTI2023

Implications of Large- N c QCD for the NN Interaction

We present a method for ordering two-nucleon interactions based upon their scaling with the number of QCD colors, N c , in the limit that N c becomes large. Available data in the two-nucleon sector show general agreement with this ordering, indicating that the method may be useful in other contexts where data are less readily available. However, several caveats and potential pitfalls can make the large-N c ordering fragile and/or vulnerable to misinterpretation. We discuss the application of the large-N c analysis to two- and three-nucleon interactions, including those originating from weak and BSM (beyond the Standard Model) interactions, as well as two-nucleon external currents. Finally, we discuss some open questions in the field.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Forest expansion dominates China’s land carbon sink since 1980

Carbon budget accounting relies heavily on Food and Agriculture Organization land-use data reported by governments. Here we develop a new land-use and cover-change database for China, finding that differing historical survey methods biased China’s reported data causing large errors in Food and Agriculture Organization databases. Land ecosystem model simulations driven with the new data reveal a strong carbon sink of 8.9 ± 0.8 Pg carbon from 1980 to 2019 in China, which was not captured in Food and Agriculture Organization data-based estimations due to biased land-use and cover-change signals. The land-use and cover-change in China, characterized by a rapid forest expansion from 1980 to 2019, contributed to nearly 44% of the national terrestrial carbon sink. In contrast, climate changes (22.3%), increasing nitrogen deposition (12.9%), and rising carbon dioxide (8.1%) are less important contributors. This indicates that previous studies have greatly underestimated the impact of land-use and cover-change on the terrestrial carbon balance of China. This study underlines the importance of reliable land-use and cover-change databases in global carbon budget accounting.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A Lakehouse Architecture for the Management and Analysis of Heterogeneous Data for Biomedical Research and Mega-biobanks

Data Lakehouse is a new paradigm in data architectures that embodies and integrates already established concepts for the systematic management of disparate, large-scale data – a data lake for heterogeneous data management, use of open standards for high-performance querying, and systematic maintenance of the data "freshness". In addition to being a new concept, the data lakehouse is also still a conceptual construct. Many projects that use the lakehouse require maturing, empirical studies, and specific implementations. In this paper, we present our implementation of the data lakehouse concept in a biomedical research and health data analytics domain, and we discuss the implementation of some unique and novel features such as support for specialized access controls in support of HIPAA regulation and IRB protocols, and support for the FAIR standard.

Begoli, Edmon↗
DOE OSTI2025

Systematic uncertainties in the measurement of the neutron lifetime using the Lunar Prospector neutron spectrometer

The lifetime of free neutrons measured in the laboratory has a longstanding disparity of ≈9 s. A space-based technique has recently been proposed to independently measure the neutron lifetime using interactions between the galactic cosmic rays and a low atmosphere planetary body. This technique has not produced competitive results yet due to constraints of nonoptimized data that contain large systematic errors. We use data from the neutron spectrometer on-board NASA's Lunar Prospector, and study two large systematics in the measurement of neutron lifetime: the lunar subsurface temperature and the lunar surface composition. We use the HeCd and HeSn neutron spectrometer data when the spacecraft was in a highly elliptical orbit during the orbit insertion period. We report the neutron lifetime using four different models that each have different choices of surface temperature and composition. The 5° [Prettyman et al ., J. Geophys. Res.: Planets 111, 2005JE002656 (2006)] and 2° [Wilson et al., Phys. Rev. C 104, 045501 (2021)] rebinned maps result in 777.6±11.7 s and 739.6±10.8 s, respectively. For the 20° map (Prettyman et al., 2006), constant equatorial and a latitude-dependent temperature model result in 738.6±10.8 s and 767.3±11.2 s, respectively. Increasing the complexities of the models accounting for the systematic effects increase the measured lifetime. However, the reported measurements are not competitive with the laboratory results due to large unaccounted systematics resulting from nonoptimized measurements and modeling assumptions. This work serves as a study of systematic uncertainties for future neutron lifetime measurements using the space-based technique. We estimate the effect on the lifetime from the choice of temperature model to be to be 28.7 ±15.5 s, and choice of compositional map (for 20° and 5° maps) to be 10.3 ±12.2 s.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Source term estimation using noble gas and aerosol samples

Algorithms that estimate the location, time, and magnitude of a point-source atmospheric release using remotely sampled air concentrations typically use data for a single chemical or radioactive isotope. Here, a Bayesian algorithm is presented that uses data from multiple radioactive isotopes that are all released in the same short-duration event. Data from noble gas and aerosol samplers can be used simultaneously in the model. Application to a large synthetic data set using four isotopes shows the new algorithm generally gives more accurate location and time estimates than a comparable model using a single isotope.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Bridging the Gap for Powering Data Centers

The rapid expansion of data centers, primarily driven by artificial intelligence, is outpacing the adaptability of the U.S. electric grid. This report, developed by Idaho National Laboratory (INL) , presents a gap analysis of some of the infrastructure challenges associated with large-scale data center deployment. Drawing from a national workshop hosted by INL in October of 2025, the report synthesizes stakeholder insights, survey data, and technical discussions to identify critical barriers and research needs. Key findings highlight the growing preference for behind-the-meter generation, the perceived inadequacy of legacy interconnection processes, and the urgent need for improved coordination between utilities, regulators, and data center developers. Environmental concerns such as water use and noise pollution, as well as economic constraints like equipment lead times and cost allocation, are also explored. The report outlines national lab capabilities in modeling, simulation, and technical assistance, and proposes targeted R&D priorities to support resilient, scalable, and efficient integration of data centers into the grid.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Demonstration of object location, classification, and characterization by developed deep learning dust ablation trail analysis code package using plasma jets

Based on deep learning, a Dust Ablation Trail Analysis (DATA) code package is developed to detect dust ablation trails in tokamaks, which is intended to analyze a large amount data of tokamak dusts. To validate and benchmark the DATA code package, 2440 plasma jet images are exploited for the training and test of the deep learning DATA code package, since plasma jets resemble the shape and size of dust ablation clouds in tokamaks. After being trained by 1920 plasma jet images, the DATA code package is able to locate 100% plasma jets, classify plasma jets with the accuracy of >99.9%, and output image skeleton information for classified plasma jets. The DATA code package trained by the plasma jet images is also used to analyze the dust ablation trails captured in the Experimental Advanced Superconducting (EAST) tokamak with the satisfactory performance, further verifying its applicability in the fusion dust ablation investigation. Based on its excellent performance presented here, it is demonstrated that our DATA code package is able to automatically identify and analyze dust ablation trails in tokamaks, which can be used for further detailed investigations, such as the three-dimensional reconstruction of dusts and their ablation trails.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗