Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

SlimIO: Lightweight I/O Path Design for Write Isolation in FDP-backed In-Memory Databases

In-Memory Databases (IMDBs) are widely used with HPC applications to manage transient data, often using snapshot-based persistence for backups. Redis, a representative IMDB, employs both snapshot and Write-Ahead Log (WAL) mechanisms, storing data on persistent devices via the traditional kernel I/O path. This method incurs syscall overhead, I/O contention between processes, and SSD garbage collection (GC) delays. To address these issues, we propose SlimIO, which adopts I/O passthru to minimize syscall overhead and inter-process I/O interference. Additionally, it leverages Flexible Data Placement (FDP) SSDs as backup storage to avoid performance degradation from SSD GC. Experimental results show that SlimIO reduces snapshot time by up to 25%, increases query throughput by up to 30% during non-snapshot periods, and lowers 99.9%-ile latency by up to 50%. Furthermore, it achieves a write amplification factor (WAF) of 1.00, indicating no redundant internal writes, thus extending SSD lifespan.

Lee, Sangyun [Sogang University]↗
DOE OSTI2025

Redesign of the Timeline Generator at Fermilab using a web-based Flutter application, GraphQL API and an IOC

The control system at Fermilab is undergoing an evolution with a shift towards web-based applications with connections to the EPICS infrastructure. The Timeline Generator (TLG) is an application that serves to coordinate events across the lab using different timing links. These links include the Tevatron clock (TCLK), a 10 MHz serial link with events encoded at 20Hz and Machine Data (MDAT), a communication link with states encoded at 720Hz. This paper covers the redesign of the major components of the TLG. This includes a web-based Flutter application for building timelines. A placement service is in use that has a GraphQL interface and uses a timeline input to compute a schedule of events and states. The Flutter application sends this computed schedule to the TLG IOC via a GraphQL interface to the Data Pool Manager (DPM). The TLG IOC runs on an Arria FPGA, the Accelerator Clock Generator (ACLK-GEN), which is responsible for writing the events and states on to the different timing links.

Carmichael, Linden [Fermilab]↗
DOE OSTI2024

Efficient Dimension Reduction of Complex Three-dimensional CO2 Saturation using Deep Learning Models

In the domain of deep learning (DL), dimension reduction is crucial for enhancing training efficiency and mitigating overfitting, particularly when managing complex data such as three-dimensional (3D) saturation data. The 3D saturation data in the context of geological carbon storage (GCS) presents unique challenges due to its inherent sparsity and the abrupt transitions at plume boundaries, known as shock fronts. To address the challenges, we proposed a novel DL framework that integrates dimension reduction with advanced 3D reconstruction techniques. Our model leveraged latent variables derived from 2D average saturation data, offering a robust and efficient solution tailored to the intricate dynamics of 3D saturation fields. The proposed framework can extract the critical features of the high-dimensional data while reducing the variable numbers, which is more tractable for DL models and enhances the model robustness and accuracy. Therefore, it provides a novel approach for modeling and analyses in complex geological scenarios, which finds great potential applications in environmental monitoring and energy storage.

Wang, Hongsheng↗
DOE OSTI2025

Collaborative: in situ visual analytics technologies for extreme scale combustion simulations

This project aims to drastically enhance the usability of in situ analysis and visualization for extreme-scale scientific simulations. Current exascale computing capabilities promise to offer greater predictive ability of simulations and to further push the frontiers of science and technology. However, to validate the simulation output at extreme scale, examine the modeled phenomena, and discover previously unknowns from the output data, the output must be reduced or transformed in situ as it is being generated during the simulation such that the amount of data to examine and store is kept to a minimum. Such in situ approaches allow us to process and analyze the data and any embedded geometry to an extent that would be prohibitively expensive, if not impossible, to perform as a post hoc task. While in situ processing has been demonstrated to be a feasible and promising approach, its full potential has not yet been leveraged. In this project, we have developed comprehensive enhancements to in situ technology based on probability distributions in data. Our research focuses on jointly developing new ways of interacting with massive statistical samples while creatively utilizing new state-of-the-art computational resources to push the boundaries of in situ exploration. Moreover, we have developed new time-dependent techniques to enable previously unattainable capabilities in areas such as intelligent simulation steering and precise feature identification. We have experimentally studied our design and implementation at NERSC and OLCF, and are able to leverage existing in situ infrastructures whenever possible. While the exemplar in this project is combustion, many other fields for which turbulent transport is important, e.g., fusion, climate, astrophysics among others, encounter similar issues as simulations scale up to the exascale. This project shows its potential to generate high impact on DOE missions since the resulting technology promises to improve scientists’ ability to rapidly and correctly interpret and tune extreme-scale simulations, leading to new scientific understanding and advancements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Materials Characterization, Prediction and Control Project: Summary Report on Data Analytics Framework

This report summarizes the activities performed under the data analytics Vertex in the Materials Characterization, Prediction and Control Project funded under laboratory directed research and development at Pacific Northwest National Laboratory. The data analytics Vertex developed models for associating global or local process parameters, microstructural features, and performance properties of friction-stir-processed 316L stainless steel plates. Statistical, machine learning, and deep learning models, as well as generative artificial intelligence approaches, were used to develop the associations between the process-structure-property data streams. These associations formed the basis for predicting global properties of parts manufactured under different process envelopes, providing a basis for predicting performance using data driven as well as physics-informed and physics-constrained approaches. Additionally, the associations were used to predict local process parameters and microstructural features of the product, predictive relationships that have the potential to form the basis of a control framework that could eventually modulate a friction-stir process to maintain product quality.

316L stainless steel↗
DOE OSTI2025

Adoption of AI in the Utility T&D Sector: Use Cases, Consequence, Assessment and Benefits

Digital transformation and utilization of artificial intelligence (AI) in the electric grid are fundamentally changing the industry’s approach to common problems and enabling a broader paradigm shift in grid planning and operations. The change in approach is circularly both enabling and driving modernization, with load growth and reliable management of data center and AI infrastructure shifting away from planning approaches with relatively predictable behaviors and toward a mix of consumer and industrial choices that surpass human cognitive abilities to process. This movement has potential to condition humans to not understand the system on which the AI depends, while requiring it for development of the necessary infrastructure. Approaches which would address most likely grid conditions and events, such as faults, aging of equipment, and weather, now must also account for large loads which shift not based upon weather or time of day, but the computational load. Quantifying computational load is independent of the traditional grid forecasting variables, where a data center’s aggregate load is determined by user and AI system behavior and decoupled from normal grid planning and operations. AI is both the cause and solution for these challenges, with new grid planning tools integrating massive amounts of decisions into frameworks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Redesign of the Timeline Generator at Fermilab using a web-based Flutter Application, GraphQL API and an IOC

Redesign of the Timeline Generator at Fermilab using a web-based Flutter application, GraphQL API and an IOC ABSTRACT = The control system at Fermilab is undergoing an evolution with a shift towards web-based applications with connections to the EPICS infrastructure. The Timeline Generator (TLG) is an application that serves to coordinate events across the lab using different timing links. These links include the Tevatron clock (TCLK), a 10 MHz serial link with events encoded at 20Hz and Ma-chine Data (MDAT), a communication link with states encoded at 720Hz. This paper covers the redesign of the major components of the TLG. This includes a web-based Flutter application for building timelines. A placement service is in use that has a GraphQL interface and uses a timeline input to compute a schedule of events and states. The Flutter application sends this computed schedule to the TLG IOC via a GraphQL interface to the Data Pool Manager (DPM). The TLG IOC runs on an Arria FPGA, the Accelerator Clock Generator (ACLK-GEN), which is responsible for writing the events and states on to the different timing links.

Carmichael, Linden [Fermilab]↗
DOE OSTI2026

FAIR Data Meets FAIR Software

Modern scientific research is increasingly defined by the interplay between data, software, and the workflows that connect them. Yet while the FAIR (Findable, Accessible, Interoperable, Reusable) principles have become foundational for scientific data stewardship, the same level of structure and expectation has only recently begun to extend to research software. This talk covers why and how FAIR principles are being applied to data and software to support data reuse. It outlines the gaps in current sharing norms, the growing federal emphasis on persistent identifiers and public access, and the opportunities created when datasets, computational workflows, code, and models are linked through rich, standardized metadata. Practical implementation pathways for the EIC and JLab communities are described, including datacards for structured dataset documentation and provenance-aware workflows. By aligning data lifecycle management with FAIR-aligned software practices, the scientific community can advance toward autonomous knowledge graphs, generative workflows, and high-quality, AI-ready scientific datasets.

McSpadden, Diana [Thomas Jefferson National Accele↗
DOE OSTI2026

Requirements and Overview of the LAMP PDM Vault

The Los Alamos Neutron Science Center (LANSCE) is located at Technical Area 53 (TA-53) at Los Alamos National Laboratory in Los Alamos, New Mexico. LANSCE is driven by an 800 mega electron volt (MeV) proton accelerator that delivered first beam in 1972. The LANSCE accelerator is unique in that it accelerates both H- (to full energy of 800 MeV) and H+ ions (up to 100 MeV currently but has accelerated high-power H+ beam to 800 MeV in the past) and supports five separate experimental areas that operate simultaneously with each having different timing and beam current requirements.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

A galactic approach to neutron scattering science

Neutron scattering science is leading to significant advances in our understanding of materials and will be key to solving many of the challenges that society is facing today. Improvements in scientific instruments are actually making it more difficult to analyze and interpret the results of experiments due to the vast increases in the volume and complexity of data being produced and the associated computational requirements for processing that data. New approaches to enable scientists to leverage computational resources are required, and Oak Ridge National Laboratory (ORNL) has been at the forefront of developing these technologies. We recently completed the design and initial implementation of a neutrons data interpretation platform that allows seamless access to the computational resources provided by ORNL. For the first time, we have demonstrated that this platform can be used for advanced data analysis of correlated quantum materials by utilizing the world's most powerful computer system, Frontier. In particular, we have shown the end-to-end execution of the DCA++ code to determine the dynamic magnetic spin susceptibility χ(q, ω) for a single-band Hubbard model with Coulomb repulsion U/t = 8 in units of the nearest-neighbor hopping amplitude t and an electron density of n = 0.65. The following work describes the architecture, design, and implementation of the platform and how we constructed a correlated quantum materials analysis workflow to demonstrate the viability of this system to produce scientific results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

NDMAS Portal Updates

Overview of current ART-GCR Data, including fuel fabrication, irradiation monitoring, fuel and graphite, post-irradiation examination, graphite characterization, high temperature metals mechanical tests, methods validation data (including: HTTR, NSTF, and HTTF), addition of AGR fuel data, other GCR and related data, updated high temperature metals, ongoing and upcoming work, users guide, and the link to those interested in access to NDMAS.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

NDMAS

Overview of Current ART-GCR Data: Fuel Fabrication, Irradiation Monitoring (Fuel & Graphite – near real-time for HDG-1), Post-Irradiation Examination (Fuel & Graphite), Graphite Characterization (Baseline and Irradiated), High Temperature Metals Mechanical Tests, Design, Methods, and Validation Data, Japan Atomic Energy Agency’s High Temperature Test Reactor (HTTR), Argonne National Laboratory’s Natural convection Shutdown heat removal Test Facility (NSTF), Oregon State University’s High Temperature Test Facility (HTTF), Generation IV International VHTR Materials Handbook, Additional related data, and Advanced Test Reactor operations (near real-time).

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Towards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures

Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load—derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.

kilic, Ozgur Ozan [Brookhaven National Laboratory ↗
DOE OSTI2025

Data for Identifying the best high-biomass sorghum hybrids based on biomass yield potential and feedstock quality affected by nitrogen fertility management under various environments

Data were collected from agronomy fields in Urbana and Ewing, IL, during the 2022 and 2023 growing seasons. The dataset includes dry biomass yield, nitrogen, phosphorus, and potassium concentrations and removals, and chemical composition elements (cellulose, hemicellulose, lignin, and soluble fractions) for 13 high-biomass sorghum hybrids. data_sharing.xlsx contains 20 columns and 104 rows. Below is the explanation of all variables in the file: Year: 2022; 2023 Location: Urbana, IL; Ewing, IL N rate (kg-N/ha): 0; 112 Hybrid #: H1-H13 Pedigree: Pedigree for 13 hybrids Dry biomass yield (Mg/ha): Aboveground dry biomass yield N (g/kg): Nitrogen concentration in plant tissue P (g/kg): Phosphorus concentration in plant tissue K (g/kg): Potassium concentration in plant tissue N (kg/ha): Nitrogen removal by aboveground biomass P (kg/ha): Phosphorus removal by aboveground biomass K (kg/ha): Potassium removal by aboveground biomass Cellulose (g/kg): Cellulose concentration in plant tissue Hemicellulose (g/kg): Hemicellulose concentration in plant tissue Lignin (g/kg): Lignin concentration in plant tissue Soluble (g/kg): Soluble concentration in plant tissue Cellulose (Mg/ha): Cellulose content in aboveground biomass Hemicellulose (Mg/ha): Hemicellulose content in aboveground biomass Lignin (Mg/ha): Lignin content in aboveground biomass Soluble (Mg/ha): Soluble content in aboveground biomass

environmental adaptability↗