Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A New Measurement of the Extragalactic Background Light Using 15 yr of Fermi-large Area Telescope Data

The extragalactic background light (EBL) from ultraviolet to infrared comprises the emission from all stars, galaxies, and actively accreting black holes in the observable Universe. A precise measurement of the EBL is critically important to probe models of star formation and galaxy evolution. The EBL can be measured via the absorption imprint left on the spectra of γ-ray blazars. In this work, we rely on 15 yr of Fermi-LAT data and 1576 blazars to measure the EBL optical depth in the 0 < z < 4.3 range. We detect the EBL attenuation with ≅23σ significance and measure the optical depth in 19 redshift bins, extending the coverage and improving on our previous results. This allows us to reconstruct the evolution of the EBL and find general consistency with recent EBL models. These results represent the most precise determination of the EBL with GeV γ rays to date.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Ice Phase Classification Made Easy with Score-Based Denoising

Accurate identification of ice phases is essential for understanding various physicochemical phenomena. However, such classification for structures simulated with molecular dynamics is complicated by the complex symmetries of ice polymorphs and thermal fluctuations. For this purpose, both traditional order parameters and data-driven machine learning approaches have been employed, but they often rely on expert intuition, specific geometric information, or large training data sets. In this work, we present an unsupervised phase classification framework that combines a score-based denoiser model with a subsequent model-free classification method to accurately identify ice phases. Further, the denoiser model is trained on perturbed synthetic data of ideal reference structures, eliminating the need for large data sets and labeling efforts. The classification step utilizes the smooth overlap of atomic position (SOAP) descriptors as the atomic fingerprint, ensuring Euclidean symmetries and transferability to various structural systems. Our approach achieves a remarkable 100% accuracy in distinguishing ice phases of test trajectories using only seven ideal reference structures of ice phases as model inputs. This demonstrates the generalizability of the score-based denoiser model in facilitating phase identification for complex molecular systems. The proposed classification strategy can be broadly applied to investigate structural evolution and phase identification for a wide range of materials, offering new insights into the fundamental understanding of water and other complex systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗
DOE OSTI2021

An open-access database and analysis tool for perovskite solar cells based on the FAIR data principles

Large datasets are now ubiquitous as technology enables higher-throughput experiments, but rarely can a research field truly benefit from the research data generated due to inconsistent formatting, undocumented storage or improper dissemination. Here we extract all the meaningful device data from peer-reviewed papers on metal-halide perovskite solar cells published so far and make them available in a database. We collect data from over 42,400 photovoltaic devices with up to 100 parameters per device. We then develop open-source and accessible procedures to analyse the data, providing examples of insights that can be gleaned from the analysis of a large dataset. The database, graphics and analysis tools are made available to the community and will continue to evolve as an open-source initiative. This approach of extensively capturing the progress of an entire field, including sorting, interactive exploration and graphical representation of the data, will be applicable to many fields in materials science, engineering and biosciences.

14 SOLAR ENERGY↗
DOE OSTI2020

Machine learning prediction of incidence of Alzheimer’s disease using large-scale administrative health data

Nationwide population-based cohort provides a new opportunity to build an automated risk prediction model based on individuals’ history of health and healthcare beyond existing risk prediction models. We tested the possibility of machine learning models to predict future incidence of Alzheimer’s disease (AD) using large-scale administrative health data. From the Korean National Health Insurance Service database between 2002 and 2010, we obtained de-identified health data in elders above 65 years (N = 40,736) containing 4,894 unique clinical features including ICD-10 codes, medication codes, laboratory values, history of personal and family illness and socio-demographics. To define incident AD we considered two operational definitions: “definite AD” with diagnostic codes and dementia medication (n = 614) and “probable AD” with only diagnosis (n = 2026). We trained and validated random forest, support vector machine and logistic regression to predict incident AD in 1, 2, 3, and 4 subsequent years. For predicting future incidence of AD in balanced samples (bootstrapping), the machine learning models showed reasonable performance in 1-year prediction with AUC of 0.775 and 0.759, based on “definite AD” and “probable AD” outcomes, respectively; in 2-year, 0.730 and 0.693; in 3-year, 0.677 and 0.644; in 4-year, 0.725 and 0.683. The results were similar when the entire (unbalanced) samples were used. Important clinical features selected in logistic regression included hemoglobin level, age and urine protein level. This study may shed a light on the utility of the data-driven machine learning model based on large-scale administrative health data in AD risk prediction, which may enable better selection of individuals at risk for AD in clinical trials or early detection in clinical settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Development and performance evaluation of a novel γ-γ coincidence analysis software

Processing of large amounts of data from a γ-γ coincidence system is a practical challenge in trace-level radionuclide measurements. Given the benefits and growing number of multidimensional gamma spectrometers worldwide, this is a major restriction in wide-spread applicability of coincidence techniques. This study demonstrates a novel γ-γ coincidence analysis software developed at Pacific Northwest National Laboratory, USA. The software utilizes advanced search techniques and Python libraries to readily identify coincidence signatures in the large data. Further, using both synthetic and experimental data, the study uncovers various strategies/libraries employed during software’s development. This work aims to provide a reference document in handling large data routinely encountered in trace-level measurements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

L-PBF High-Throughput Data Pipeline Approach for Multi-modal Integration

Abstract Metal-based additive manufacturing requires active monitoring solutions for assessing part quality. Multiple sensors and data streams, however, generate large heterogeneous data sets that are impractical for manual assessment and characterization. In this work, an automated pipeline is developed that enables feature extraction from high-speed camera video and multi-modal data analysis. The framework removes the need for manual assessment through the utilization of deep learning techniques and training models in a weakly supervised paradigm. We demonstrate this pipeline’s capability over 700,000 high-speed camera frames. The pipeline successfully extracts melt pool and spatter geometries and links them to corresponding pyrometry, radiography, and processparameter information. 715 individual prints are examined to reveal melt pool areas that exceeds 0.07 mm 2 and pyrometry signal over a threshold (375 pyrometry units) were more likely to have defects. These automated processes enable massive throughput of characterization techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Nuclear Science for the Manhattan Project and Comparison to Today’s ENDF Data

Nuclear physics advances in the United States and Britain from 1939 to 1945 are described. The Manhattan Project’s work led to an explosion in our knowledge of nuclear science. A conference in April 1943 at Los Alamos provided a simple formula used to compute critical masses and laid out the research program needed to determine the key nuclear constants. In short order, four university accelerators were disassembled and reassembled at Los Alamos, and methods were established to make measurements on extremely small samples owing to the initial lack of availability of enriched 235U and plutonium. I trace the program that measured fission cross sections, fission-emitted neutron multiplicities and their energy spectra, and transport cross sections, comparing the measurements with our best understanding today as embodied in the Evaluated Nuclear Data File ENDF/B-VIII.0. The large nuclear data uncertainties at the beginning of the project, which often exceeded 25% to 50%, were reduced by 1945 often to less than 5% to 10%. Uranium-235 and plutonium-239 fission cross-section assessments in the fast mega-electron-volt range were reduced following more accurate measurements, and the neutron multiplicity $\overline{v}$ increased. By a lucky coincidence of canceling errors, the initial critical mass estimates were close to the final estimated masses. Some images from historical documents from our Los Alamos archives are shown. Many of the original measurements from these early years have not previously been widely available. Through this work, these data have now been archived in the international experimental nuclear reaction data library (EXFOR) in a collaboration with the International Atomic Energy Agency and Brookhaven National Laboratory.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Data for KETCHUP: Parameterizing of Large-Scale Kinetic Models Using Multiple Datasets with Different Reference States

Repository for Kinetic Estimation Tool Capturing Heterogeneous Datasets Using Pyomo (KETCHUP), a flexible parameter estimation tool that leverages a primal-dual interior-point algorithm to solve a nonlinear programming (NLP) problem that identifies a set of parameters capable of recapitulating the steady-state fluxes and concentrations in wild-type and perturbed metabolic networks. KETCHUP can use K-FIT [2] input files. Example K-FIT input files are located in the K-FIT repository at https://github.com/maranasgroup/K-FIT.

Metabolomics↗
DOE OSTI2025

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗
DOE OSTI2021

A Self-Evolution Data Fusion Platform for Large-Scale Water Models

Focal Area: Data acquisition and assimilation enabled by machine learning (ML), artificial intelligence (AI), and advanced methods including experimental/network design/optimization, unsupervised learning (deep learning), leveraging advanced hardware (e.g., edge computing). This development is to enable solving the science questions regarding the human and climatic factors that interact with and drive global water scarcity.

54 ENVIRONMENTAL SCIENCES↗