SEARCH · Search NASA

Results for “data-intensive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2025

dCache: The Storage System of Choice for Data-Intensive Applications

The ever-increasing volumes of data produced by modern scientific facilities like EuXFEL and LHC put significant stress on data management infrastructure operated by laboratories and research centers. The challenges to be addressed span the entire data life cycle, from ingest and efficient data analysis to long-term preservation, typically involving large tape libraries. dCache, a storage system developed in collaboration between the Deutsches Elektronen-Synchrotron (DESY), Fermi National Accelerator Laboratory, and Nordic e-Infrastructure Collaboration (NeIC), is designed to manage a large number of disk servers and to facilitate transparent data migration to and from archival storage. Its multifaceted approach offers a unified method to support a variety of scientific use cases with the same storage infrastructure, including high-throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and long-term data preservation on tertiary storage. Initially developed for high energy physics (HEP) experiments, dCache is now used by various scientific communities, including astrophysics, biomedical research, and life sciences, each having specific requirements. This paper presents architecture, deployment strategies, performance and scalability enhancements, and recent advancements in dCache addressing the needs of scientific communities. Finally, we touch on the development and release process, ensuring the software’s high quality.

DCache↗

DOE OSTI2026

Data Summarization and Inference at Scale

This is the final report for the DOE ASCR grant SC-0022260, Data Summarization and Inference at Scale, PI: Alex Pothen, Purdue University. The goal of the project was to solve data-intensive and compute-intensive problems in the physical sciences, engineering, information science, data science, etc. by designing and implementing new algorithms that could work with a subset of the data. The four subgoals were: (a) The solution of problems where the data is too large to be stored in the memory of a computer. In this streaming model of computation, the data arrives as a stream of elements to the computer, each element is processed as it arrives, and a decision is made to discard the data or to store it; only a small subset of the data proportional to the size of the output solution is stored, and when all the data has been streamed, a solution to the problem is computed from the stored subset. (b) The use of machine learning methods to compute solutions to data-intensive problems. The use of GPUs is critical to obtain high performance on machine learning tasks, but their memory sizes are smaller relative to that of CPUs. For large-scale problems, the data is sampled many times, and small samples are used with repetition, for robustness, to compute solutions to inference tasks. This sampling reduces the memory required to solve the problem, but attention is needed to avoid slow convergence to the solutions, and reduced accuracy of inference. We propose submodular optimization, Large Language Models, and physics-informed neural networks to enable GPU computations here. (c) Modeling and visualization of high-dimensional data using interpretable features. Clinical proteomic data sets from immunology for the detection of cancer and other diseases are temporal and high-dimensional, and algorithms for visualizing these data sets using clinically interpretable features are lacking. We propose methods that compute distances based on the optimal transportation problem and graph edit distances to address this problem. We also propose the use of optimal transport-based distances, spatial statistics, and network structure to classify image data sets, We apply these algorithms to electron micrographs of the peripheral nervous system in the digestive tract. (d) The design of data-intensive algorithms on emerging architectures, specifically, noisy, intermediate-scale quantum (NISQ) devices. Quantum computers offer the possibility of exploring large solution spaces due to the principle of superposition, but current quantum computers are limited by few qubits, short coherence times due to noise, poor interconections among the qubits, etc. We propose the use of the divide and conquer paradigm to solve large-scale problems, wherein collections of small subproblems are solved on the quantum devices, and the solutions to the subproblems are integrated into a solution for the original problem on a classical computer.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Machine learning for domain transfer between simulated and experimental 2D X-ray diffraction patterns using generative adversarial networks

X-ray diffraction (XRD) is a well-established technique for analyzing materials at an atomic level. Dynamic compression experiments (DCE), in which materials are subject to extreme pressures, can provide fundamental understanding to pressure-induced phase transitions and compression of the crystal lattice. The analysis of XRD patterns from highly compressed samples is non-trivial given the sparsity of data, high experimental costs, and the fact that the data is often marred with X-ray background and other artifacts. While accurate computational frameworks exist, they solve the forward problem—from structures and orientations to XRD patterns. Solving the inverse problem for 2D experimental diffraction patterns is currently a complex manual process of matching and comparing experimentally observed patterns to computationally generated ones. Machine learning is a promising tool for automating the matching process but often requires data-intensive architectures. Here, in this study, we use a CycleGAN to translate the domain of limited experimental data to a domain in which there is readily available simulated data. This domain shift allows data-intensive machine learning models that have only been trained on simulated XRD patterns to be used in the analysis of experiments.

Brozak, Samantha Jean [Sandia National Laboratorie↗

DOE OSTI2026

A family portrait of lanmodulin selectivity for enhanced rare-earth separations

Proteins offer a molecular design space to create bespoke ligands for the separation of critical metals like rare earth elements (REs). However, data-intensive approaches to tune metalloprotein selectivity are constrained by the low-throughput nature of existing characterization methods. Here we invented an assay called ‘SpyTag-Catcher Immobilization of Lanmodulin for Assaying Metal-Binding Selectivity’ (SpyCI-LAMBS) to measure metalloprotein selectivity en masse. This 96-format workflow was used to study the selectivity of 621 lanmodulin (LanM) orthologs for 15 REs, revealing eight distinct selectivity profiles based on sequence-to-function analyses. We discovered >200 LanMs with stronger selectivity against low-value LaIII relative to the prototypical LanM. This includes a LanM that can perform a challenging one-stage separation of PrIII from LaIII with up to >99.9 mol% purity and 83% yield. SpyCI-LAMBS is a powerful tool that can rapidly collect high-fidelity selectivity data to inform metal ion separations and machine-learning-assisted metalloprotein design.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

A rapid embodied carbon assessment tool for priority materials

Embodied carbon limits within building materials are a driving factor in global trade, generating new research and analysis tools in industry. These product assessments, which require utilizing life-cycle assessment (LCA) across broad supply chains, can be expensive, time and data-intensive, and subject to significant variations. Existing methods and tools, such as specific environmental product declarations, typically do not capture these variations and dynamics in supply and manufacturing. Moreover, models and tools must enable stakeholders to assess customized supply chains and future scenarios. In this study, we present the Rapid Embodied Carbon Assessment and Target-setting for Emissions-intensive Materials (REDuCE) tool for building materials. We developed a tool that allows users to select production technologies, transportation mode and distances, concrete carbonation, fuel sources, and regional electricity mixes to supply customization for cement and concrete produced and consumed in California. We generated and integrated a material demand model using residential building stock projections. We provide the user with a wide range of mitigation alternatives through low-carbon production pathways, material use efficiency, transportation modes, and projected electricity grid mixes. In a case study application through four mitigation scenarios, we find emission savings up to 80% by maximizing user mitigation alternatives, primarily driven by reductions in material use intensities. This work represents a foundation for expanding LCA and embodied carbon tools to better enable stakeholders to rapidly and accurately assess customized supply chains while meeting trade requirements.

building materials↗

DOE OSTI2025

Unified architecture for quantum lookup tables

Quantum access to arbitrary classical data encoded in unitary black-box oracles underlies interesting data-intensive quantum algorithms, such as machine learning or electronic structure simulation. The feasibility of these applications depends crucially on gate-efficient implementations of these oracles, which are commonly some reversible versions of the Boolean circuit for a classical lookup table. Here, we present a general parametrized architecture for quantum circuits implementing a lookup table that encompasses all prior work in realizing a continuum of optimal trade-offs between qubits, non-Clifford gates, and error resilience, up to logarithmic factors. Our architecture assumes only local 2D connectivity, yet recovers results, with the appropriate parameters, polylogarithmic error scaling. We also identify regimes, such as simultaneous sublinear scaling, in all parameters. These results enable tailoring implementations of the commonly used lookup table primitive to any given quantum device with constrained resources.

quantum circuits↗

DOE OSTI2026

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗

DOE OSTI2025

Single-Cell Universal Logic-in-Memory Using 2T-nC FeRAM: An Area and Energy-Efficient Approach for Bulk Bitwise Computation

This work presents a novel approach to configure 2T-nC ferroelectric RAM (FeRAM) for performing single cell logic-in-memory operations, highlighting its advantages in energy-efficient computation over conventional DRAM-based approaches. Unlike conventional 1T-1C dynamic RAM (DRAM), which incurs refresh overhead, 2T-nC FeRAM offers a promising alternative as a non-volatile memory solution with low energy consumption. Our key findings include the potential of quasi-nondestructive readout (QNRO) sensing in 2T-nC FeRAM for logic-in-memory (LiM) applications, demonstrating its inherent capability to perform inverting logic without requiring external modifications, a feature absent in traditional 1T-1C DRAM. We successfully implement the MINORITY function within a single cell of 2T-nC FeRAM, enabling universal NAND and NOR logic, validated through SPICE simulations and experimental data. Additionally, the research investigates the feasibility of 3D integration with 2T-nC FeRAM, showing substantial improvements in storage and computational density, facilitating bulk-bitwise computation. Our evaluation of eight real-world, data-intensive applications reveals that 2T-nC FeRAM achieves 2× higher performance and 2.5× lower energy consumption compared to DRAM. Furthermore, the thermal stability of stacked 2T-nC FeRAM is validated, confirming its reliable operation when integrated on a compute die. These findings emphasize the advantages of 2T-nC FeRAM for LiM, offering superior performance and energy efficiency over conventional DRAM.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Pushing the limits of NAND technology scaling with ferroelectrics

Artificial intelligence (AI) continues to drive transformative advancements across various industries. The data-intensive nature of AI training (and inferencing) has resulted in the generation of unprecedented volumes of data with machine-generated content surpassing human-generated data by more than 100-fold in 2025. Efficiently managing this data influx necessitates advanced digital storage technologies. However, traditional NAND flash memory, which is critical for supporting data flows in AI systems—alongside high-bandwidth memory, for AI training—faces fundamental scaling limitations as it approaches the 1000-layer milestone, encompassing more than 40 trillion transistors. This article delves into the potential of hafnia-based ferroelectric materials as a breakthrough solution to these challenges. Recent advancements indicate that the intrinsic limitations of ferroelectric field-effect transistors (FEFETs) can be mitigated through material and device-level engineering. These advancements enable FEFETs to meet the stringent density, reliability, and scalability requirements of future three-dimensional NAND technology. The role of ferroelectrics in addressing NAND scaling challenges and expanding storage capabilities presents a promising avenue for meeting the storage demands of the AI-driven era.

3D NAND↗

DOE OSTI2025

2020 Budget Request for the DOE Computational Science Graduate Fellowship (CSGF) Grant

The Department of Energy Computational Science Graduate Fellowship (DOE CSGF) is essential for addressing the increasingly complex national workforce demands stemming from the growth of computational science and engineering challenges. Computational science and engineering (CSE) takes a multidisciplinary approach that utilizes scientific computing to tackle practical problems and provide technical tools across the spectrum of scientific discovery. The DOE CSGF specifically highlights high-performance computing (HPC) as a critical enabling technology in CSE, driving advancements in science and engineering that are vital to both the DOE and the broader economy. Over the past half-century, HPC has been an essential tool for DOE’s success. During this period, important missions, such as nuclear stockpile stewardship, have turned to HPC as an essential technology. Entire science disciplines have been transformed through the augmentation of scientific observation via HPC. At government laboratories, academic institutions, and in industry, DOE CSGF alumni are helping push traditional HPC boundaries while contributing to discoveries in high-energy physics, quantum information systems, fusion-reactor design, machine learning, additive manufacturing, nano materials for next-generation batteries and transistors, and advanced nuclear reactor modeling. In addition, HPC is used to address national health needs that will eventually point to cures both by helping cancer researchers manage and analyze huge troves of data, by simulating biological mechanisms, and by accelerating drug development. A 2023 report from the ASCAC Subcommittee on American Competitiveness and Innovation to the ASCR office, “Can the United States Maintain Its Leadership in High-Performance Computing?” says of the Program, “The CSGF program provides a barometer for disciplines that will be of interest to future DOE computing. Computational biology, machine learning, and quantum computing are among the subjects that began to swell in the ranks of CSGF applicants before the labs were hiring as high a percentage of employees in these categories.” The explosion of scientific and technological data has heightened the demand for advanced high-performance computing (HPC) to transform these data into meaningful scientific insights. As access to vast amounts of data increases, the fields of Machine Learning and Artificial Intelligence are experiencing a resurgence, enhancing the established practices of computational modeling and simulation. In its September 2020 subcommittee report on "AI/ML, Data Intensive Science, and High-Performance Computing," the DOE Advanced Scientific Computing Advisory Committee (ASCAC) specifically called for a fellowship program to train computational and data scientists to address exascale and data-intensive computing challenges. This integration of empirical and theoretical modeling will increasingly guide federal policymakers in making decisions that impact American society and future generations. It demands a workforce of highly skilled and intellectually agile computational scientists capable of navigating the rapid advancements in scientific computing within the DOE National Laboratory research environment. The DOE CSGF program has consistently addressed this critical need.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Data Cards for Standardized Metadata Across DOE-Aligned Data Initiatives: Toward Transparent, Interoperable, and Governed Dataset Documentation

As data-intensive research, advanced computing, and artificial intelligence become increasingly central to scientific and operational workflows, the need for consistent, transparent, and machine-actionable documentation has grown correspondingly. Multiple DOE-aligned communities—including Office of Science, Genesis Mission, American Science Cloud (AmSC), National Nuclear Security Administration (NNSA) stewardship and governance, and related cross-laboratory collaborations—have independently developed metadata practices to support discovery, access, reuse, repository deposit, and compliance.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

Carothers, Chris [Rensselaer Poly.]↗

DOE OSTI2026

Genesis Mission Data cards

As data-intensive research and artificial intelligence become central to DOE mission science, the need for machine-actionable dataset documentation has grown accordingly. However, many DOE-aligned communities, including the Office of Science, NNSA, and cross-laboratory collaborations, have developed independent metadata practices. This fragmentation creates friction for discovery, federation, and reuse across programs. To address these challenges, this talk introduces the Genesis Data Card: a shared metadata artifact developed in collaboration with a broad DOE community (Jefferson Lab and the National Lab of the Rockies, Oak Ridge, Sandia, Idaho, Berkeley, and Los Alamos). The Genesis Data Card aims to standardize dataset documentation across DOE-aligned initiatives while remaining extensible to discipline-specific needs. This talk will describe the data card template and the supporting code to validate completed data cards, using a companion LinkML schema. I'll walk through the design decisions behind the template, its alignment with existing standards, its treatment of sensitivity and governance metadata, and the phased roadmap toward lifecycle-integrated "xCards" that support autonomous discovery and reuse. The talk closes with current gaps, ongoing work, and how others can contribute datasets and feedback to the shared repository.

McSpadden, Helen [Thomas Jefferson National Accele↗

DOE OSTI2026

Report of the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing: Harnessing Community, Software, and AI for Cross-Disciplinary Team Science

Scientific computing is undergoing rapid transformation as advances in artificial intelligence, heterogeneous computing, automation, and data-intensive research reshape not only computational tools but also the institutions, workforce models, and collaborative practices that support scientific discovery. This report synthesizes insights from the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing, the second in a three-year series focused on strengthening scientific computing ecosystems through socio-technical co-design. Workshop discussions identified four interdependent strategic themes: software ecosystems for AI-enabled scientific discovery; trust, validation, and traceability; human-AI teaming and paradigm shifts; and workforce, pedagogy, and governance. The report translates these themes into eight priorities for community action spanning shared research infrastructure, trust and traceability, user experience, human-AI teaming, workforce development, cross-sector coordination, stewardship and sustainability, and evaluation of scientific value. Together, these priorities outline directions for building scientific computing ecosystems that remain trustworthy, sustainable, innovative, and resilient as AI assumes a growing role in scientific work.

AI↗

DOE OSTI2026

photoD with Rubin ’s Data Preview 1: First stellar photometric distances and faint blue star deficits

Aims. We investigate the utility of Rubin’s Data Preview 1 (DP1) for estimating stellar number density profiles across the Milky Way halo. Methods. We used stellar broad-band near-UV to near-IR ugrizy photometry released in Rubin’s DP1 to estimate distance and metallicity for blue main sequence stars brighter than r = 24 in three ~1.1 sq. deg. fields at southern Galactic latitudes. Results. Compared to TRILEGAL simulations of the Galaxy’s stellar content, we found a likely deficit of blue main sequence turn-off stars with 22 < r < 24. We interpreted this discrepancy as a signature of a steeper halo number density profile at galactocentric distances 10–50 kpc than the canonical ~1/r 3 profile assumed in TRILEGAL simulations. Conclusions. This interpretation is consistent with earlier suggestions based on observations of more luminous, but much less numerous, evolved stellar populations, along with a few pencil beam surveys of blue main sequence stars in the northern sky. These results bode well for the future Galactic halo exploration with Rubin’s Legacy Survey of Space and Time (LSST).

Galaxy: fundamental parameters↗

DOE OSTI2026

The Vera C. Rubin Observatory Data Preview 1

We present Rubin Data Preview 1 (DP1), the first data from the National Science Foundation–Department of Energy Vera C. Rubin Observatory, comprising raw and calibrated single-epoch images, coadds, difference images, detection catalogs, and ancillary data products. DP1 is based on 1792 optical–near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera (LSSTComCam) on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile in late 2024. DP1 covers ∼15 deg 2 distributed across seven roughly equal-sized noncontiguous fields, each independently observed in six broad photometric bands, ugrizy. The median FWHM of the point-spread function across all bands is approximately 1"14, with the sharpest images reaching about 0." 58. The 5σ point-source depths for coadded images in the deepest field, the Extended Chandra Deep Field South, are u = 24.55, g = 26.18, r = 25.96, i = 25.71, z = 25.07, and y = 23.1. Other fields are no more than 2.2 mag shallower in any band, where they have nonzero coverage. DP1 contains approximately 2.3 million distinct astrophysical objects, of which 1.6 million are extended in at least one band in coadds, and 431 solar system objects, of which 93 are new discoveries. DP1 is approximately 3.5 TB in size and is available to Vera C. Rubin Observatory data rights holders via the Rubin Science Platform, a cloud-based environment for the analysis of petascale astronomical data. While small compared to future LSST releases, its high quality and diversity of data support a broad range of early science investigations ahead of full operations in 2026.

Ground-based astronomy↗

DOE OSTI2025

RTN-045: Guidelines for User Tutorials

This document defines the guidelines, principles, and formats for user-facing tutorials that demonstrate how to use the Rubin Science Platform (RSP) to analyze data from the Legacy Survey of Space and Time (LSST). All Rubin staff and the broader science community should use these guidelines when contributing to the sets of Jupyter Notebook or documentation-based tutorials maintained by the Rubin Community Science team (CST).

79 ASTRONOMY AND ASTROPHYSICS↗