Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Stores”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Continuous snow depth and temperature measurements from dense network of above-ground distributed temperature profiling systems from 2021-09-23 to 2024-08-23, Seward Peninsula, Alaska

The dataset contains temperature measurements from distributed temperature profiling (DTP) systems (Dafflon et al., 2022; Wielandt et al., 2022; Wang et al., 2024a; Fiolleau et al., 2024) deployed vertically above the ground surface at a large number of locations from 2021 to 2024. The research is designed to improve understanding of the local heterogeneity in snow depth and snow thermal insulation dynamics, as well as their interactions in a discontinuous permafrost region (Wang et al., 2025). The DTP systems were deployed at 96 locations in a watershed along the Nome-Teller road at mile marker 27 (T27) and at 54 locations on a hillslope along the Kougarok road at mile marker 64 (K64) in the Seward Peninsula, Alaska. The probe location information is stored in Probe_locations_T27.csv and Probe_locations_K64.csv. Temperature measurements were recorded at 15-minute intervals using high-precision digital sensors (accuracy: ±0.1°C, resolution: 0.0078°C). The temperature probes, either 1.4 m or 1.6 m long, contain sensors spaced every 5 cm or 10 cm along their length. The temperature data are stored in compressed files following the format: DTP_snow_air_temperature_(site)_(start)_(end).zip, where site is either T27 or K64, and start and end represent the time series period. Within each ZIP file, individual CSV files are named by probe ID and contain temperature records at different heights above the ground surface.This dataset also includes derived snow depth time series over three snow seasons, estimated from temperature measurements. Snow depth was estimated by identifying the consecutive sensor pair that exhibited the largest drop in high-frequency temperature fluctuations (detailed in the methods). These data are stored in: Snow_depths_flags_(site)_(start)_(end).csv, which includes snow depth time series and corresponding quality flags (defined in the methods) from different probes. Additionally, the dataset includes derived metrics and supporting measurements at selected locations over two snow seasons, contributing to the manuscript of Wang et al., 2025. These locations were chosen based on the availability of high-quality snow depth time series during both seasons. The additional data include: (1) Air temperature proxies measured from the top sensors on the pole when they were not buried by snow, stored in Air_temperature_proxies_(site)_(start)_(end).csv (2) Ground interface temperature, recorded at 3 cm above the ground, stored in Ground_interface_temperature_(site)_(start)_(end).csv (3) Site characteristics, including vegetation height, elevation, and the topographic position index (TPI) within a 50 m radius, stored in Selected_probe_locations_gps_vegheight_tpi_elevation_(site).csv. These metrics were derived from 1 m resolution summer LiDAR-based digital elevation models and digital surface models from Singhania et al., 2023, DOI:10.5440/1832016. Metadata files include data descriptions (_dd.csv) for tabular data. All included files are listed and described in xxxx_flmd.csv.This dataset is an updated version of a previous archive (Wang et al., 2024b, DOI: 10.15485/2475020), incorporating multiple seasons and improved snow depth estimation. Please note that due to large amount of information present in this dataset, many specificities associated with the acquisition of snow temperature, air temperature proxy and estimation of snow depth, and the future archiving of additional datasets on the soil temperature, thaw depth and soil characteristics at these locations, the author would welcome being contacted by people planning to use this dataset.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Myna: Connecting powder bed fusion build data to simulation tools for digital twin applications

Additive manufacturing (AM), as a digital process, can generate a detailed digital thread linking a part’s design and manufacturing to its operational performance. As AM systems advance, an increasing amount of process data is stored in manufacturing databases. In principle, this data can be utilized by simulation-based digital twin approaches, such as real-time process control and asynchronous post-processing guidance. However, few tools currently exist for systematically integrating digital thread data with computational tools. Here, in this study, we propose a software package, called Myna, for connecting data from powder bed fusion processes to simulation tools. The utility of such a platform is demonstrated using build data from the Oak Ridge National Laboratory Manufacturing Demonstration Facility “Peregrine v2023-10” public dataset to automatically configure and run 54 semi-analytical 3DThesis melt pool simulations, 78 numerical Additive FOAM melt pool simulations, and 3 ExaCA microstructure simulations. The simulated, spatially registered microstructures are then compared directly with electron backscatter diffraction characterization of the corresponding as-built part locations. The resulting simulated microstructure showed variation as a function of process parameters, particularly stripe width; however, the experimental data had little variation between the microstructure texture and grain size resulting from different processing conditions. Analysis of the discrepancies suggest that it is possible a two-phase ferritic-austenitic solidification model is needed to accurately predict grain size and texture for certain stainless steel 316L feedstock compositions under powder bed fusion conditions, providing direction for future research. As illustrated here, due to the number and complexity of the simulations involved in AM process-structure–property predictions, automated methods to connect process data and simulations will remain necessary tools for testing hypotheses and implementing digital twin applications.

Knapp, Gerald L. [Oak Ridge National Laboratory (O↗
DOE OSTI2025

Real-time data processing for serial crystallography experiments

We report the use of streaming data interfaces to perform fully online data processing for serial crystallography experiments, without storing intermediate data on disk. The system produces Bragg reflection intensity measurements suitable for scaling and merging, with a latency of less than 1 s per frame. Our system uses the CrystFEL software in combination with the ASAP::O data framework. In a series of user experiments at PETRA III, frames from a 16 megapixel Dectris EIGER2 X detector were searched for peaks, indexed and integrated at the maximum full-frame readout speed of 133 frames per second. The computational resources required depend on various factors, most significantly the fraction of non-blank frames ('hits'). The average single-thread processing time per frame was 242 ms for blank frames and 455 ms for hits, meaning that a single 96-core computing node was sufficient to keep up with the data, with ample headroom for unexpected throughput reductions. Further significant improvements are expected, for example by binning pixel intensities together to reduce the pixel count. We discuss the implications of real-time data processing on the `data deluge' problem from recent and future photon-science experiments, in particular on calibration requirements, computing access patterns and the need for the preservation of raw data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

An Algorithmic and Software Pipeline for Very Large Scale Scientific Data Compression with Error Guarantees

Efficient data compression is becoming increasingly critical for storing scientific data because many scientific applications produce vast amounts of data. This paper presents an end-to-end algorithmic and software pipeline for data compression that guarantees both error bounds on primary data (PD) and derived data, known as Quantities of Interest (QoI).We demonstrate the effectiveness of the pipeline by compressing fusion data generated by a large-scale fusion code, XGC, which produces tens of petabytes of data in a single day. We demonstrate that the compression is conducted by setting aside computational resources known as staging nodes, and does not impact the simulation performance. For efficient parallel I/O, the pipeline uses ADIOS2, which many codes such as XGC already use for their parallel I/O. We show that our approach can compress the data by two orders of magnitude while guaranteeing high accuracy on both the PD and the QoIs. Further, the amount of resources required by compression is a few percent of the resources required by simulation while ensuring that the compression time for each stage is less than the corresponding simulation time.This pipeline consists of three main steps. The first step decomposes the data using domain decomposition into small subdomains. Each subdomain is then compressed independently to achieve a high level of parallelism. The second step uses existing techniques that guarantee error bounds on the primary data for each subdomain. The third step uses a post-processing optimization technique based on Lagrange multipliers to reduce the QoI errors for data corresponding to each subdomain. The Lagrange multipliers generated can be further quantized or truncated to increase the compression level. All of the above characteristics of our approach make it highly practical to apply on-the-fly compression while guaranteeing errors on QoIs that are critical to the scientists.

Banerjee, Tania↗
DOE OSTI2021

AI-Batt (Autonomous Identification of Battery Life Models) [SWR 21-36]

Autonomous Identification of Battery Life Models (AI-Batt) AI-Batt is a MATLAB code base for developing lifetime models for batteries from accelerated aging data. The code base provides many functions for processing, visualizing, and modeling battery aging data, making the data processing, exploration, and modeling workflow substantially faster. These tools are tailored for working with battery aging data sets, which usually consist of many separate time-series for each cell, with many test conditions and possible replicates at each condition, which makes it difficult to simply process or visualize the data set. Complex modeling tasks, such as cross-validation, sensitivity analysis, and uncertainty quantification have been implemented to enable thorough statistical investigation of model predictions. Additionally, several machine-learning algorithms are implemented to autonomously identify suitable models via symbolic regression. Data processing functions automatically cast data from the struct data type, which is commonly used to store experimental data, but is not an acceptable input for most algorithms, to the table data type, which can be easily used as input to any optimization algorithm. Also, the data can be separated into time-invariant and time-variant data tables, which is helpful for exploring the data set as well as developing separate models for time-variant and time-invariant aging mechanisms. For example, in aging tests with constant temperature, temperature is a time-invariant experimental condition. Visualization tools enable plotting of data, model fits, and model simulations possible with single-line function calls, empowering data exploration of complex data sets with both time-varying and time-invariant trends. Plots can be automatically generated for the whole data set, or separated by data group (groups of test replicates) or individual data series. Data points or data series can be automatically colored by the value of a variable with a variety of color maps, and model predictions can also be colored by the value of a fit statistic. Comparisons between data sets and the predictions/simulations of different models on the same data set can be easily plotted as well. Distributions of parameter values from bootstrap resampling can be plotted to visualize the reliability of parameter estimation, or determine any correlations between parameters. Modeling tools handle the complex task of creating and parsing symbolic equations for modeling battery lifetime. Equations are parsed to grab relevant data variables, parameter values, or specified sub-models for input into optimization, evaluation, or simulation functions. Models can be optimized locally (one set of parameters for each data series), bi-level (some parameters shared across the data set), or globally (single set of parameters for all data). Functions implementing symbolic regression algorithms help users to discover effective model equations, even in poorly sampled, high-dimensional data.

Smith, Kandler↗
DOE OSTI2026

Cyote Insights

CyOTE Insights leverages React, Vite, Typescript, Tailwind, and Daisy UI for the Graphical User Interface. It was designed in a particular style with a dark mode and a light mode. All code is broken down into components and reusable wrapper components for efficiency. All data is stored in Deep Lynx as a central data repository using an ontology based schema. The application serves as a main endpoint for the data in the COREII and CyOTE programs. The main purpose of the application is to display historical attack data in the Operational Technology space. At the time of this writing, it supports 27 historical attack reports compiled from OSINT sources. All of the data is publicly available, but what this application offers is the ability to see many years worth of publications in a detailed dashboard. It will also support future reports that are written using the other applications in the COREII program.

Pluth, AdamJ [Idaho National Laboratory (INL), Ida↗
DOE OSTI2021

Method and apparatus for using compression to improve performance of low voltage caches

A method of operating a cache in a computing device includes, in response to receiving a memory access request at the cache, determining compressibility of data specified by the request, selecting in the cache a destination portion for storing the data based on the compressibility of the data and a persistent fault history of the destination portion, and storing a compressed copy of the data in a non-faulted subportion of the destination portion, wherein the persistent fault history indicates that the non-faulted subportion excludes any persistent faults.

Kalamatianos, John↗
DOE OSTI2020

Distributed heterogeneous compute infrastructure for the study of additive manufacturing systems

We present the current status of a large-scale computing framework to address the need of the multidisciplinary effort to study chemical dynamics. Specifically, we are enabling scientists to process and store experimental data, run large-scale computationally expensive high-fidelity physical simulation, and analyze these results using the state-of-the-art data analytics tools, machine learning, and uncertainty quantification methods using heterogeneous computing resources, such as CPU and GPU cluster. The framework can integrate or abstract out multiple domains based on roles. In order to develop this framework, we have leveraged an existing framework coupled with in-house heterogeneous computing resources. We present the results of using this framework on a single metadata triggered workflow to accelerate an additive manufacturing use case.

heterogeneous computing, hpc, workflows, reproduci↗
DOE OSTI2024

Structuring and storing signals with their metadata: practical considerations

This chapter aims to provide a comprehensive overview on structuring signal data and their metadata, highlighting key considerations for optimal management and storage. Specifically, it focuses on: (a) the relevance of data organization; (b) what data to store and what to keep; and (c) data management methods. Thus, this chapter answers where and how to store metadata efficiently. Chapter 3 explains what is considered metadata. Chapters 5 and 6 present how to collect certain metadata through dedicated sensor validation tests (Chapter 5) or algorithmic analysis (Chapter 6).

Nicolaï, Niels↗
DOE OSTI2022

Promoting prefetched data from a cache memory to registers in a processor

An electronic device includes a processor having a cache memory, a plurality of physical registers, and a promotion logic functional block. The promotion logic functional block promotes prefetched data from a portion of a cache block in the cache memory into a given physical register, the promoting including storing the prefetched data in the given physical register. Upon encountering a load micro-operation that loads data from the portion of the cache block into a destination physical register, the promotion logic functional block sets the processor so that the prefetched data stored in the given physical register is provided to micro-operations that depend on the load micro-operation.

Kotra, Jagadish↗
DOE OSTI2023

Thrifty Array Format (TAF) file specifications

Thrifty Array Foram (TAF) files store numeric data in a binary format, minimizing storage requirements while preserving quick read access. Real data of any size and dimensionality can be stored in this format at varying degrees of numeric precision. Implicit array are associated with each dimension, eliminating the need to explicitly store uniformly-spaced grid vectors. Unlimited text comments may be included with the array for user documentation, and every file begins with a text synopsis of the binary structure. The format is deliberately designed for memory mapping, where portions of the array can be read without loading the entire file at once.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath) SciDAC Institute

As computational models scale to larger computers, the rate at which they produce data has far outstripped the same computers ability to write that data and further the file systems ability to store that data. Almost all of the SciDAC applications, but especially those related to fusion solve very large scale PDEs whose scientific output his impacted by this problem. To gain access to dynamics in an exascale simulation that are not identifiable a priori and to make that dynamical data available to machine learning requires fundamental research in the area of in situ data data analytics. Here data analytics includes compression, visualization, uncertainty quantification, and machine learning. This in situ data analytics will enable on-the-fly spatial and temporal compression of solution dynamics, expose that space-time compressed field to machine learning algorithms that have been specialized to work with dynamically evolving data (existing machine learning algorithms treat data sets as static), greatly improving the opportunity for machine learning to provide feedback to the compression, all within an ongoing simulation, without the need to write data to files. The same concepts are also being applied to uncertainty quantification and multi-fidelity modeling which have similar needs for spatial and temporal compression of the ongoing exascale simulation to perform either without the typical, unacceptable writing of data to files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Python Library For Reading Endf Files

Evaluated Nuclear Data Files (ENDFs) are the most common way to store nuclear data. However, they are a hard-to-read format that few people or programs can read. OpenMC, an open source project, developed a way to read Evaluated Nuclear Data Files (ENDF) into a useful form for python. This library will remove all unnecessary code from OpenMC to focus solely on reading nuclear data. In addition, it will handle parsing Grouped cross-section data from an ENDF file, which OpenMC does not currently handle.

Gale, MicahD.↗
DOE OSTI2025

Wind Turbine Sound Setbacks and Supply Curves: Ordinances and Extrapolated Trends, 110 Hub Height, 130 Rotor Diameter

This dataset provides a comprehensive set of wind turbine sound setbacks from every residential structure in the contiguous United States (CONUS). A sound setback is defined as the minimum required distance between a residential structure and a hypothetical turbine installation site to ensure that modeled sound levels received at the residence do not exceed local sound ordinances, which are commonly expressed in A-weighted decibels (dBA). Therefore, sound setbacks are a local spatial assessment combining multiple factors, including the sound pressure curve as a function of the observer location (distance and direction) relative to the turbine, local sound regulations, and the geographical distribution of residential structures. The dataset is organized into multiple scenario-based products, detailed as follows: 1. Existing and extrapolated sound setbacks. An existing scenario characterizes sound setbacks only in states or counties that have implemented sound regulations as of 2022. The extrapolated scenarios extend a constant sound threshold to counties that lack explicit sound regulations, with thresholds ranging from 35 to 60 dBA, in 5-dBA increments reflecting the variation observed in current sound ordinances. 2. Sound setbacks in directional and worst scenarios. The directional scenario accounts for the distance and orientation of residential structures relative to a hypothetical turbine location, utilizing the turbine's sound emissions in that specific direction. In contrast, the worst scenario takes loudest sound level at each distance step from the turbine, irrespective of directional considerations, which aligns with current industry practice. 3. Supply curves for Open and Reference Access scenarios. This dataset includes supply curves generated by the reV model, which integrates each of the above sound setbacks into both Open and Reference siting scenarios. In addition, two Open and Reference baselines scenarios were included which do not consider sound setbacks for comparative analysis. All sound setback data are stored in TIF files, with partial maps of the data provided in PNG format. The values in the sound setback raster range from 0 to 1, representing the fraction of developable land within a 90 meter by 90 meter pixel due to sound ordinances. A value of 0 indicates areas where wind energy development is prohibited, while a value of 1 signifies areas fully permissible. The wind turbine parameters used in the sound modeling are based on the land-based turbine from International Energy Agency (IEA), featuring a rated electrical power of 3.4 MW, a rotor diameter of 130 meters, and a hub height of 110 meters. The atmospheric conditions, including wind speed/direction, turbulence, air temperature, relative humidity, and air pressure, that drive the sound generation are obtained from the WIND Toolkit dataset.

Array↗
DOE OSTI2020

Redirecting data to improve page locality in a scalable data fabric

A data processing system includes a host processor, a local memory coupled to the host processor, a plurality of remote memory media, and a scalable data fabric coupled to the host processor and to the plurality of remote memory media. The scalable data fabric includes a filter for storing information indicating a location of data that is stored by the data processing system. The host processor includes a hardware sequencer coupled to the filter for selectively moving data stored by the filter to the local memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

NXREFINE

NXRefine implements a complete workflow for both data acquisition and reduction of single crystal diffuse x-ray scattering collected as a series of area detector frames during continuous sample rotation. The data are stored in HDF5 files, which conform to the NeXus data format standard. These files contain a comprehensive set of metadata, including the incident wavelength, powder calibration scans used to define detector distances, beam centers, and yaw, pitch, and roll corrections. Automated peak searches of the three-dimensional data arrays define a set of Bragg peaks that are used to define an orientation matrix, defined according to Busing and Levy, which is then used to transform the data from angular coordinates to reciprocal space coordinates using the CCTW program. Multiple rotations at different detector translations and offset rotation axes are combined to ensure that there are no gaps in the reconstructed data, and allow the construction of masks to eliminate detector artifacts due to Compton scattering within the sensor layers. Finally, the data can be transformed into 3D-ΔPDF maps using the punch-and-fill method after symmetrization.

OSBORN, RAYMOND↗
DOE OSTI2023

4. Geometry Data Storage

3D-printing begins with the design of an object using computer aided design (CAD) software. The geometry of the object must be exported and saved in a data file format that can be used in the slicing process to generate machine instructions for printing. The standard method of saving the data is to tessellate the object as a triangulated mesh stored as a .stl file. Due to the flat triangular faces used to store the data, this mesh is a low-resolution representation of the high-fidelity object designed in CAD. The STL has its limitations with respect to geometrical accuracy, material information, and instances of invalid mesh data, but can still be used for the 3D-printing process. New file variants, such as OBJ, AMF, and 3MF, are being developed to fix some of these issues and increase the capabilities of geometry data storage for 3D-printing.

Roschli, Alex↗