Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Instantiation of the Damara Tern Platform for Advanced Materials and Manufacturing Technologies (AMMT) Program Collaborative Data Management

This work package focused on deploying an instance of the Damara Tern platform to support AMMT collaborative research activities. The objectives were to provide selected AMMT collaborators with access to a shared environment for capturing operations, trackables, and associated metadata, and to implement data entry functionalities that reflect site-specific procedures. Key activities included creating configurable, schema-driven entry forms and validating the data collection process. The report details the deployment process, the platform infrastructure, and the implemented data entry workflows, providing a reference for end users and establishing a foundation for future production-scale deployments.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Development of Data Reporting Standards for High-Temperature Gas-cooled Reactor (HTGR) Nuclear Energy University Program (NEUP) Thermal-Fluid Experiments

Since 2009, the U.S. Department of Energy (DOE) Office of Nuclear Energy's Nuclear Energy University Program (NEUP) has been at the forefront of nuclear research, specifically concentrating on advancing high-temperature gas-cooled reactor (HTGR) technologies. By Fiscal Year 2023, NEUP has authorized 35 projects dedicated to HTGR research, each contributing significantly to the enhancement of our understanding of this technology. The outcomes of these diverse projects have been disseminated through final NEUP reports, peer-reviewed journal articles, and presentations at academic conferences, forming a comprehensive tapestry of knowledge. Despite the substantial value of these findings, their dissemination has been fragmented, posing challenges for accessibility to researchers and policymakers and leading to underutilization of DOE investments. Recognizing this critical gap and its potential consequences for the future of nuclear research, the Advanced Reactor Technologies (ART) Gas-Cooled Reactor (GCR) program conducted an extensive survey of completed and ongoing HTGR NEUP projects. This survey enabled the compilation of crucial data, resulting in the development of a specialized public-access database tailored for computational fluid dynamics and system code validation, specifically designed for HTGR applications. However, the data collection process revealed a significant challenge in central data organization due to individual researchers from different institutes employing varying logics and preferences for recording and documenting experimental data. Consequently, an urgent need has been identified to establish a standardized reporting format for HTGR experimental projects. Addressing this issue is essential for enhancing collaboration, maximizing the impact of DOE investments, and ensuring the seamless advancement of HTGR technologies in nuclear research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

HTGR Validation: NEUP Survey and Database - Data Reporting Standard for HTGR Thermal-Fluid Experiments

Since 2009, the U.S. Department of Energy (DOE) Office of Nuclear Energy's Nuclear Energy University Program (NEUP) has been at the forefront of nuclear research, specifically concentrating on advancing high-temperature gas-cooled reactor (HTGR) technologies. By Fiscal Year 2024, NEUP has authorized 36 projects dedicated to HTGR research, each contributing significantly to the enhancement of our understanding of this technology. The outcomes of these diverse projects have been disseminated through final NEUP reports, peer-reviewed journal articles, and presentations at academic conferences, forming a comprehensive tapestry of knowledge. Despite the substantial value of these findings, their dissemination has been fragmented, posing challenges for accessibility to researchers and policymakers and leading to underutilization of DOE investments. Recognizing this critical gap and its potential consequences for the future of nuclear research, the Advanced Reactor Technologies (ART) Gas-Cooled Reactor (GCR) program conducted an extensive survey of completed and ongoing HTGR NEUP projects. This survey enabled the compilation of crucial data, resulting in the development of a specialized public-access database tailored for computational fluid dynamics and system code validation, specifically designed for HTGR applications. However, the data collection process revealed a significant challenge in central data organization due to individual researchers from different institutes employing varying logics and preferences for recording and documenting experimental data. Consequently, an urgent need has been identified to establish a standardized reporting format for HTGR experimental projects. Addressing this issue is essential for enhancing collaboration, maximizing the impact of DOE investments, and ensuring the seamless advancement of HTGR technologies in nuclear research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Automated Framework for Groundwater Monitoring Using DWT with LSTM and Transformers

Environmental monitoring is critical for safeguarding public health and ecological well-being. Traditional data structuring and workflow monitoring methods consume significant time and effort, hindering timely insights and effective decision-making. Our study addresses this challenge by presenting an AI framework that automates data cleaning, structuring, and modeling processes, specifically targeting applications in groundwater monitoring. By leveraging automation for data processing and model training, our framework establishes a novel and efficient paradigm for environmental monitoring, with its potential application to the vast network of over a hundred Department of Energy Environmental Management (DoE-EM) cleanup sites across the country. It analyzes data streams from a network of groundwater Internet-of-Things (IoT) sensors deployed at the Savannah River Site (SRS) for prediction modeling. This allows human experts to focus on analysis and decision-making, ultimately leading to better environmental outcomes.The framework employs multivariate time-series forecasting methods to study and model the behavior of varying chemical analytes. The continuous learning process is enabled by utilizing deep learning techniques. It allows the framework to become more nuanced in its analysis over time, adapting to the specific characteristics of the environmental site and the evolving nature of contaminant behavior. Deep learning models known for sequence modeling, LSTM, and Transformers are employed for time series forecasting. Data processing and structuring are essential components significantly impacting the final model's performance. This hypothesis was proven by presenting a comparative analysis of model performance with processed and unprocessed data. The feature engineering approach utilized was the Discrete Wavelet Transform, which works well with time series data.

Discrete Wavelet Transform (DWT)↗
DOE OSTI2024

Organic Matter Concentration and Composition in November 2021 and April 2022 from 12 Streams Impacted by the 2020 Holiday Farm Fire (v2)

This dataset represents results from a field study aiming to understand storm induced transport of pyrogenic materials to streams impacted by varying degrees of burn severity. Time series samples were collected at 5 sites within the McKenzie River Watershed (Oregon, USA) whose catchment were each completely engulfed by the 2020 Holiday Farm Fire. An additional 7 sites were sampled once during the storm. The samples were collected during storm events in November 2020, January 2021, November 2021, and April 2022. Samples were characterized for benezenepolycarboxylic acids (BPCA), ultra-high resolution mass spectrometry, dissolved organic carbon and optics (absorbance and fluorescence). Fourier-transform ion cyclotron resonance mass spectrometry (FTICR) and dissolved organic carbon data from the November 2020 (referred to as “EWEB_2020”) sampling can be found in a separate data package (doi: 10.15485/1869708). NOTE: The 2020 samples were run on FTICR-MS in two unique instances. The first run can be found in the previous data package (EWEB_2020). The second run is included in this data package. These samples were run for a second time so that the data were more directly interoperable with the other samples in this data package. We have not done any investigation into the differences/similarities between these datasets and the previously ran/published data in the other data package. This data package was originally published in November 2024. It was updated in April 2025 (v2; new and modified files). See the change history section below for more details. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. This dataset contains (1) file-level metadata; (2) data dictionary; (3) data package readme; (4) metadata; (5) methods information; (6) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data; (7) excitation emission matrix (EEM) methods; and (8) a sub-folder with processed EEM data (9) benzene polycarboxylic acid (BPCA) concentration data; (10) Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) methods; and (11) folder of high-resolution characterization of organic matter via 12 Tesla FTICR-MS generated through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory). The EEMs sub-folder contains two additional folders; the Absorbance and Fluorescence folders which contain the processed EEMs absorbance and fluorescence data respectively. This package contains the following file types: csv, xml, pdf.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Classification of events from α -induced reactions in the MUSIC detector via statistical and ML methods

The Multi-Sampling Ionization Chamber (MUSIC) detector is typically used to measure nuclear reaction cross sections relevant for nuclear astrophysics, fusion studies, and other applications. From the MUSIC data produced in one experiment scientists carefully extract an order of 10 3 events of interest from about 10 9 total events, where each event can be represented by an 18-dimensional vector. However, the standard data classification process is based on expert driven, manually intensive data analysis techniques that require several months to identify patterns and classify the relevant events from the collected data. Here, to address this issue, we present a method for the classification of events originating from specific α-induced reactions by combining statistical and machine learning methods that require significantly less input from the domain scientist, relative to the standard technique. Here, we applied the new method to two experimental data sets and compared our results with those obtained using traditional methods. With few exceptions, the number of events classified by our method agrees within ±20% with the results obtained using traditional methods. With the present method, which is the first of its kind for the MUSIC data, we have established the foundation for the automated extraction of physical events of interest from experiments using the MUSIC detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Desmearing Bonse–Hart USANS data using Bayesian Gaussian process regression

Ultra-small-angle neutron scattering (USANS) enables access to micrometer-scale structures but is intrinsically affected by strong, anisotropic resolution smearing arising from slit-geometry optics. As a result, recovery of the intrinsic scattering intensity constitutes an ill-posed inverse problem, and commonly used iterative desmearing methods lack rigorous uncertainty quantification. We present a Bayesian desmearing framework for slit-geometry USANS based on Gaussian process regression. In this approach, the scattering intensity is modeled as a smooth random function, and the instrumental point spread function is incorporated explicitly as a forward operator. The resulting formulation yields a closed-form maximum a posteriori solution with well-defined credibility intervals. Computational benchmarks and experimental validation using combined USANS and small-angle neutron scattering (SANS) measurements demonstrate that the framework enables stable desmearing, suppresses experimental noise, and preserves physically meaningful structural features under realistic conditions.

Tung, Chi-Huan [Oak Ridge National Laboratory (ORN↗
DOE OSTI2025

SetBERT: the deep learning platform for contextualized embeddings and explainable predictions from high-throughput sequencing

MOTIVATION: High-throughput sequencing (HTS) is a modern sequencing technology used to profile microbiomes by sequencing thousands of short genomic fragments from the microorganisms within a given sample. This technology presents a unique opportunity for artificial intelligence to comprehend the underlying functional relationships of microbial communities. However, due to the unstructured nature of HTS data, nearly all computational models are limited to processing DNA sequences individually. This limitation causes them to miss out on key interactions between microorganisms, significantly hindering our understanding of how these interactions influence the microbial communities as a whole. Furthermore, most computational methods rely on post-processing of samples which could inadvertently introduce unintentional protocol-specific bias. RESULTS: Addressing these concerns, we present SetBERT, a robust pre-training methodology for creating generalized deep learning models for processing HTS data to produce contextualized embeddings and be fine-tuned for downstream tasks with explainable predictions. By leveraging sequence interactions, we show that SetBERT significantly outperforms other models in taxonomic classification with genus-level classification accuracy of 95%. Furthermore, we demonstrate that SetBERT is able to accurately explain its predictions autonomously by confirming the biological-relevance of taxa identified by the model. AVAILABILITY AND IMPLEMENTATION: All source code is available at https://github.com/DLii-Research/setbert. SetBERT may be used through the q2-deepdna QIIME 2 plugin whose source code is available at https://github.com/DLii-Research/q2-deepdna.

Ludwig, David W↗
DOE OSTI2025

Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- and GPU-based algorithms generate 3D reconstructed volumes of experimental samples. We employ two data movement and reconstruction models: real-time processing as data streams directly to NERSC compute nodes, and automated file transfer to NERSC and ALCF file systems. The streaming pipeline provides users with feedback in under ten seconds, while the file-based workflow produces high-quality reconstructions suitable for deeper analysis in 20-30 minutes. This infrastructure enables users to utilize HPC resources without direct access to backend systems. We plan to extend this architecture to more endstations, supporting our beamline scientists and users.

Abramov, David↗
DOE OSTI2024

Utah FORGE: Direct Shear Test Data for Investigating Seismic Precursors to Shear Failure of Fractures

This dataset includes results of direct shear tests to investigate the mechanical and geophysical response of dry and saturated fractures in Indiana limestone and Sierra White granite. Direct shear tests were performed on tensile-induced fractures in Indiana limestone and Sierra White granite in a custom water-pressurized chamber. The provided Excel files include the representative seismic wave signals and the normalized wave amplitudes of ultrasonic wave transducers. A link to the published journal article presenting the data and describing the experiment in detail is provided as well.

15 GEOTHERMAL ENERGY↗
DOE OSTI2026

FAIRmaterials: Ontology Tools with Data FAIRification in Development

The bilingual FAIRmaterials package simplifies the creation and visualization of materials and data science ontologies. FAIRmaterials, available in the Python and R languages, addresses the complexities associated with traditional ontology editors based on manual user input such as Protege with an intuitive workflow and easy-to-use templates, making it accessible to users both experienced and inexperienced with ontologies. The FAIRmaterials package is its ability to programatically convert simple and structured CSV inputs into rich, well-defined ontologies. This capability is designed to support the findability, accessibility, interoperability, and reusability (FAIR) of research data and serve as a tool in the process of data FAIRification. Its additional features, such as automated ontology merging, static visualizations, and comprehensive documentation for outputs extend its utility, making it a valuable tool for any researcher engaged in knowledge management.

Bradley, Alexander Harding [Case Western Reserve U↗
DOE OSTI2025

A Performant, Scalable Processing Pipeline for High‐Quality and FAIR Environmental Sensor Data

High-resolution environmental monitoring is necessary to record, understand, and predict biogeochemical and ecological changes particularly in coastal systems but brings significant challenges in processing and making rapidly available the resulting data. The COMPASS-FME project established a network of coastal observational sites across the Chesapeake Bay and western Lake Erie regions extensively instrumented with soil, vegetation, and weather sensors logging data every 15 min. Our data processing framework, written in R and completely open source, prioritizes rapid model-experiment iteration and makes biogeochemical data rapidly available for quality assurance/quality control, analysis, and model ingestion. This pipeline is distinguished by a standardized and modular approach to data curation, extensive metadata and documentation, and its high performance. These attributes combine to make biogeochemical data rapidly accessible across COMPASS-FME and the broader community. Flexible, powerful, and reproducible approaches to handling high-volume environmental data are crucial for accelerating biogeosciences research.

Pennington, Stephanie C. [Pacific Northwest Nation↗
DOE OSTI2025

Fast quantum ghost imaging with a single-photon-sensitive time-stamping camera

Quantum ghost imaging (QGI) leverages correlations between entangled photon pairs to reconstruct an image using light that has never physically interacted with an object. Despite extensive research interest, this technique has long been hindered by slow acquisition speeds, due to the use of raster-scanned detectors or the slow response of intensified cameras. Here, we utilize a single-photon-sensitive time-stamping camera to perform QGI at ultra-low-light levels with rapid data acquisition and processing times, achieving high-resolution and high-contrast images in under 1 min. Our work addresses the trade-off between image quality, optical power, data acquisition time, and data processing time in QGI, paving the way for practical applications in biomedical and quantum-secured imaging.

Mavian, Alex (ORCID:0000000279448830)↗
DOE OSTI2024

epicsuite(EAS)

Software, Documentation, Tutorials, Testing data, Example data for processing, analysis, filtering, querying, visualization and otherwise transforming genomic data for scientific analysis and discovery.

Rogers, David H.↗
DOE OSTI2024

A galactic approach to neutron scattering science

Neutron scattering science is leading to significant advances in our understanding of materials and will be key to solving many of the challenges that society is facing today. Improvements in scientific instruments are actually making it more difficult to analyze and interpret the results of experiments due to the vast increases in the volume and complexity of data being produced and the associated computational requirements for processing that data. New approaches to enable scientists to leverage computational resources are required, and Oak Ridge National Laboratory (ORNL) has been at the forefront of developing these technologies. We recently completed the design and initial implementation of a neutrons data interpretation platform that allows seamless access to the computational resources provided by ORNL. For the first time, we have demonstrated that this platform can be used for advanced data analysis of correlated quantum materials by utilizing the world's most powerful computer system, Frontier. In particular, we have shown the end-to-end execution of the DCA++ code to determine the dynamic magnetic spin susceptibility χ(q, ω) for a single-band Hubbard model with Coulomb repulsion U/t = 8 in units of the nearest-neighbor hopping amplitude t and an electron density of n = 0.65. The following work describes the architecture, design, and implementation of the platform and how we constructed a correlated quantum materials analysis workflow to demonstrate the viability of this system to produce scientific results.

97 MATHEMATICS AND COMPUTING↗