Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Improving Text Classification with Large Language Model-Based Data Augmentation

Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

NanoSIP: NanoSIMS Applications for Microbial Biology

High-resolution imaging with secondary ion mass spectrometry (nanoSIMS) has become a standard method in systems biology and environmental biogeochemistry and is broadly used to decipher ecophysiological traits of environmental microorganisms, metabolic processes in plant and animal tissues, and cross-kingdom symbioses. When combined with stable isotope-labeling—an approach we refer to as nanoSIP—nanoSIMS imaging offers a distinctive means to quantify net assimilation rates and stoichiometry of individual cell-sized particles in both low- and high-complexity environments. While the majority of nanoSIP studies in environmental and microbial biology have focused on nitrogen and carbon metabolism (using 15 N and 13 C tracers), multiple advances have pushed the capabilities of this approach in the past decade. The development of a high-brightness oxygen ion source has enabled high-resolution metal analyses that are easier to perform, allowing quantification of metal distribution in cells and environmental particles. New preparation methods, tools for automated data extraction from large data sets, and analytical approaches that push the limits of sensitivity and spatial resolution have allowed for more robust characterization of populations ranging from marine archaea to fungi and viruses. Further, NanoSIMS studies continue to be enhanced by correlation with orthogonal imaging and ‘omics approaches; when linked to molecular visualization methods, such as in situ hybridization and antibody labeling, these techniques enable in situ function to be linked to microbial identity and gene expression. Here we present an updated description of the primary materials, methods, and calculations used for nanoSIP, with an emphasis on recent advances in nanoSIMS applications, key methodological steps, and potential pitfalls.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2025

Toward equitable environmental exposure modeling through convergence of data, open, and citizen sciences: an example of air pollution exposure modeling amidst increasing wildfire smoke

Exposure modeling is critical in environmental epidemiology and human health but may face challenges (e.g., skewed data, unequal error, context-insensitive validation, and computational demands). Modeling decisions reflect the intended use of the models and the values that modelers prioritize. We aimed to provide a conceptual framework and machine learning (ML) modeling protocols that address these issues. With 500m-gridded hourly PM 2.5 and O 3 levels in Illinois before, during, and after the 2023 Canadian wildfire season as a motivating example, we conducted modeling experiments to evaluate modeling methods, guided by three domains we propose based on theories of science: 1) Data Diversity, leveraging open and citizen science data to enhance inclusivity, parsimony, and representativeness; 2) Equitable Accuracy, ensuring fairly distributed uncertainties across subpopulations; and 3) Sustainable Modeling, balancing accuracy with reducing computational demands to promote accessibility for under-resourced researchers. Here, we found that ML with publicly available data can achieve high accuracy. Depending on methods, performance may vary substantially, even with identical input data. Large but skewed data may reduce performance. Misuse of cross-validation protocols can underestimate prediction error; although we observed R 2 s of ∼98 %, the modeled estimates varied significantly, indicating the need for careful model validation. By using new modeling protocols including representativeness-considered training and validation data and a new loss function, we achieved high agreement between estimates and ground-based measurements (e.g., R 2 = ∼90 % for PM 2.5 ; ∼80 % for O 3 ), equally distributed errors across sociodemographic strata and urban–rural divides, and reduction in computation time—from several weeks or months to a few days.

Exposure assessment↗
DOE OSTI2025

Gaussian Process Regression under Computational and Epistemic Misspecification

Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kernel. This paper investigates the effect of such kernel approximations on the interpolation error. We introduce a unified framework to analyze Gaussian process regression under important classes of computational misspecification: Karhunen-Loève expansions that result in low-rank kernel approximations, multiscale wavelet expansions that induce sparsity in the covariance matrix, and finite element representations that induce sparsity in the precision matrix. Furthermore, our theory also accounts for epistemic misspecification in the choice of kernel parameters.

Gaussian process regression↗
DOE OSTI2023

Advancing the Theory of Nuclear Data Evaluations [Abstract]

We present recent advances in the R-matrix formalism as well as the Bayesian evaluation framework for improved nuclear data evaluations. The advances in the R matrix formalism include: 1) direct processes, 2) doorway, as well as multistep, processes, and 3) various forms of the Reich-Moore approximation for eliminated capture channels. Furthermore, to address unreasonably small posterior uncertainties often encountered in nuclear data evaluations of large data sets using the conventional form of the Bayes’ theorem, we introduce imperfections (of the data or the model) as a formal evaluation tool for taming the evaluated uncertainties in harmony with Bayes’ theorem. These theoretical advances were motivated by the nuclear data evaluations of differential resolved resonance cross section data using the code SAMMY, as well as the integral benchmark experiments using the SCALE code system, being performed at Oak Ridge National Laboratory for the Nuclear Criticality Safety Program. Some pedagogical applications of the new formalism, as well as a snapshot of the SAMMY modernization efforts, will be presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Electromagnetic Transient Modeling of Data Centers

This report serves as a user manual for the accompanying EMT model library developed by the National Laboratory of the Rockies (NLR) for various equipment in large data centers. The EMT model library enables detailed modeling of large data center loads for conducting grid stability studies. The EMT model library for data centers include detailed models of a 5.5 kW power supply unit (PSU), a 2.5 uninterruptible power supply (UPS), a 260 MW gas turbine-generator, a 500 kW motor load, and a 33 kW IT rack. These components represent all major equipment in data centers that need to be modeled for performing grid stability studies for data centers.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
DOE OSTI2023

Progressive Tree-Based Compression of Large-Scale Particle Data

Scientific simulations and observations using particles have been creating large datasets that require effective and efficient data reduction to store, transfer, and analyze. However, current approaches either compress only small data well while being inefficient for large data, or handle large data but with insufficient compression. Toward effective and scalable compression/decompression of particle positions, we introduce new kinds of particle hierarchies and corresponding traversal orders that quickly reduce reconstruction error while being fast and low in memory footprint. Our solution to compression of large-scale particle data is a flexible block-based hierarchy that supports progressive, random-access, and error-driven decoding, where error estimation heuristics can be supplied by the user. For low-level node encoding, we introduce new schemes that effectively compress both uniform and densely structured particle distributions. Our proposed methods thus target all three phases of a tree-based particle compression pipeline, namely tree construction, tree traversal, and node encoding. In conclusion, the improved efficacy and flexibility of these methods over existing compressors are demonstrated through extensive experimentation, using a wide range of scientific particle datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Extreme Compression for Large Scale Data Store

For the last 5 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4.In this contribution, we will present our concepts of “functionally lossless compression”, have a glance at examples and achievements in other communities, present the results and outcome of our current, ongoing R&D, as well as present a high-level view of our plan to move forward with a ROOT implementation that would deliver a basic solution readily integrated into HENP applications. As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Quantifying Atomically Dispersed Catalysts Using Deep Learning Assisted Microscopy

The catalytic performance of atomically dispersed catalysts (ADCs) is greatly influenced by their atomic configurations, such as atom–atom distances, clustering of atoms into dimers and trimers, and their distributions. Scanning transmission electron microscopy (STEM) is a powerful technique for imaging ADCs at the atomic scale; however, most STEM analyses of ADCs thus far have relied on human labeling, making it difficult to analyze large data sets. Here, we introduce a convolutional neural network (CNN)-based algorithm capable of quantifying the spatial arrangement of different adatom configurations. The algorithm was tested on different ADCs with varying support crystallinity and homogeneity. Results show that our algorithm can accurately identify atom positions and effectively analyze large data sets. Here, this work provides a robust method to overcome a major bottleneck in STEM analysis for ADC catalyst research. We highlight the potential of this method to serve as an on-the-fly analysis tool for catalysts in future in situ microscopy experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Online randomized interpolative decomposition with a posteriori error estimator for temporal PDE data reduction

Traditional low-rank approximation is a powerful tool for compressing large data matrices that arise in simulations of partial differential equations (PDEs), but suffers from high computational cost and requires several passes over the PDE data. The compressed data may also lack interpretability thus making it difficult to identify feature patterns from the original data. Here, to address these issues, we present an online randomized algorithm to compute the interpolative decomposition (ID) of large-scale data matrices in situ. Compared to previous randomized IDs that used the QR decomposition to determine the column basis, we adopt a streaming ridge leverage score-based column subset selection algorithm that dynamically selects proper basis columns from the data and thus avoids an extra pass over the data to compute the coefficient matrix of the ID. In particular, we adopt a single-pass error estimator based on the non-adaptive Hutch++ algorithm to provide real-time error approximation for determining the best coefficients. As a result, our approach only needs a single pass over the original data and thus is suitable for large and high-dimensional matrices stored outside of core memory or generated in PDE simulations. A strategy to improve the accuracy of the reconstructed data gradient, when desired, within the ID framework is also presented. We provide numerical experiments on turbulent channel flow and ignition simulations, and on the NSTX Gas Puff Image dataset, comparing our algorithm with the offline ID algorithm to demonstrate its utility in real-world applications.

Column subset selection↗
DOE OSTI2026

Measurement Adequacy for Monitoring Data Center Oscillations

Artificial intelligence (AI) training data centers with periodic load profiles can induce sustained grid oscillations across a wide frequency range, making accurate monitoring essential for reliable power system operation. This report evaluates the adequacy of existing measurement systems for monitoring such oscillations, focusing on phasor measurement units (PMUs) and point-on-wave (POW) measurement systems. The analysis shows that while PMUs are highly effective for monitoring low-frequency electromechanical oscillations, they have inherent limitations in accurately representing higher-frequency oscillations due to constraints imposed by reporting rates and the bandwidth of phasor estimation filters. Even when configured with higher reporting rates, the filtering inherent in the phasor estimation process can significantly attenuate oscillation magnitudes, potentially leading to underestimation of oscillatory behavior. This has important implications for compliance and performance monitoring of large loads. To address the limitations associated with PMU-based monitoring, the report examines the use of high-resolution POW measurements, which can capture oscillations across a broader frequency range. However, continuous POW monitoring introduces practical challenges related to large data volumes, communication bandwidth, and real-time data processing. For this reason, the report also discusses emerging approaches that use POW measurements as a complementary capability alongside PMUs to improve observability of oscillations from large data center loads.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

EVT 16s Data and Large Supplementary Files

Soil microorganisms often interact to carry out decomposition of complex organic carbon and nitrogen compounds, such as chitin, but the high diversity and complexity of the soil microbiome and habitat has posed a challenge to elucidating such interactions between soil microorganisms. Here, we seek to address this challenge through analysis of a model soil consortium (MSC-2) of eight soil bacterial species. Our aim was to elucidate specific roles of the member species during chitin metabolism. Samples were collected from MSC-2 incubated in chitin-enriched soil over three months. Multi-omics was used to understand how the community composition, transcripts, proteins and chitin decomposition shifted over time. The data clearly and consistently revealed a temporal shift during chitin decomposition with defined contributions by individual species. A Streptomyces genus member (sp001905665) was a key player in early steps of chitin decomposition, with other MSC-2 members being central in carrying out later steps. These results illustrate how multi-omics applied to a defined consortium untangles interactions between soil microorganisms.

McClure, Ryan [Pacific Northwest National Laborato↗
DOE OSTI2020

F2 extraction from Inclusive Cross Section Data at Large Bjorken x

While we have made significant progress in probing nuclear structure at low Bjorken x and high Q 2 , we still have gaps in our knowledge regarding the high Bjorken x and intermediate Q 2 kinematic region. This is not an accident, but instead due to the diffculty in assessing the non-perturbative region of nuclear physics. For this reason, the E12-10-002 experiment measured the H(e,e') and D(e,e') inclusive cross section in the resonance region. This will allow us to study both perturbative and non-perturbative physics in a kinematic region that lacks precision measurements. Measurements were made up to a Bjorken x of 0.99 and Q 2 up to 17 GeV 2 . These measurements are made using the independent HMS and SHMS spectrometers at the upgraded beam energy of 11 GeV in Hall C at Jefferson Lab. Further, we extract F 2 structure functions from the measured data and study the effect of their inclusion in a global Parton Distribution Function analysis.

Araiza Gonzalez, Fernando↗
DOE OSTI2021

The Rise of Neural Networks for Materials and Chemical Dynamics

Machine learning (ML) is quickly becoming a premier tool for modeling chemical processes and materials. ML-based force fields, trained on large data sets of high-quality electron structure calculations, are particularly attractive due their unique combination of computational efficiency and physical accuracy. This Perspective summarizes some recent advances in the development of neural network-based interatomic potentials. Designing high-quality training data sets is crucial to overall model accuracy. One strategy is active learning, in which new data are automatically collected for atomic configurations that produce large ML uncertainties. Another strategy is to use the highest levels of quantum theory possible. Transfer learning allows training to a data set of mixed fidelity. A model initially trained to a large data set of density functional theory calculations can be significantly improved by retraining to a relatively small data set of expensive coupled cluster theory calculations. These advances are exemplified by applications to molecules and materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Radiative properties of quantum emitters in boron nitride from excited state calculations and Bayesian analysis

Abstract Point defects in hexagonal boron nitride (hBN) have attracted growing attention as bright single-photon emitters. However, understanding of their atomic structure and radiative properties remains incomplete. Here we study the excited states and radiative lifetimes of over 20 native defects and carbon or oxygen impurities in hBN using ab initio density functional theory and GW plus Bethe-Salpeter equation calculations, generating a large data set of their emission energy, polarization and lifetime. We find a wide variability across quantum emitters, with exciton energies ranging from 0.3 to 4 eV and radiative lifetimes from ns to ms for different defect structures. Through a Bayesian statistical analysis, we identify various high-likelihood charge-neutral defect emitters, among which the native V N N B defect is predicted to possess emission energy and radiative lifetime in agreement with experiments. Our work advances the microscopic understanding of hBN single-photon emitters and introduces a computational framework to characterize and identify quantum emitters in 2D materials.

Chemistry↗
DOE OSTI2025

The System for Classification of Low-Pressure Systems (SyCLoPS): An All-In-One Objective Framework for Large-Scale Data Sets

We propose the first unified objective framework (SyCLoPS) for detecting and classifying all types of low-pressure systems (LPSs) in a given data set. We use the state-of-the-art automated feature tracking software TempestExtremes (TE) to detect and track LPS features globally in ERA5 and compute 16 parameters from commonly found atmospheric variables for classification. A Python classifier is implemented to classify all LPSs at once. The framework assigns 16 different labels (classes) to each LPS data point and designates four different types of high-impact LPS tracks, including tracks of tropical cyclone (TC), monsoonal system, subtropical storm and polar low. The classification process involves disentangling high-altitude and drier LPSs, differentiating tropical and non-tropical LPSs using novel criteria, and optimizing for the detection of the four types of high-impact LPS. A comparison of our labels with those in the International Best Track Archive for Climate Stewardship (IBTrACS) revealed an overall accuracy of 95% in distinguishing between tropical systems, extratropical cyclones, and disturbances. SyCLoPS produces a better TC detection skill compared to the previous algorithms, highlighted by an approximately 6% reduction in the false alarm rate compared to the previous TE algorithm. The vertical cross section composite of the four types of high-impact LPS we detect each shows distinct structural characteristics. Finally, we demonstrate that SyCLoPS is valuable for investigating various aspects of LPSs in climate data, such as the evolution of a single LPS track, patterns of LPS frequencies, and precipitation or wind influence associated with a particular LPS class.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

CoreCruncher : Fast and Robust Construction of Core Genomes in Large Prokaryotic Data Sets

The core genome represents the set of genes shared by all, or nearly all, strains of a given population or species of prokaryotes. Inferring the core genome is integral to many genomic analyses, however, most methods rely on the comparison of all the pairs of genomes; a step that is becoming increasingly difficult given the massive accumulation of genomic data. Here, we present CoreCruncher; a program that robustly and rapidly constructs core genomes across hundreds or thousands of genomes. CoreCruncher does not compute all pairwise genome comparisons and uses a heuristic based on the distributions of identity scores to classify sequences as orthologs or paralogs/xenologs. Although it is much faster than current methods, our results indicate that our approach is more conservative than other tools and less sensitive to the presence of paralogs and xenologs. CoreCruncher is freely available from: https://github.com/lbobay/CoreCruncher. CoreCruncher is written in Python 3.7 and can also run on Python 2.7 without modification. It requires the python library Numpy and either Usearch or Blast. Certain options require the programs muscle or mafft.

59 BASIC BIOLOGICAL SCIENCES↗