Search NASA⌕ Search

SEARCH · Search NASA

Results for “process data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Cloud-Tracking Data Set for the CSAPR2 Adaptive Scanning during TRACER

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) User Facility (Mather and Voyles 2013) deployed the first ARM Mobile Facility (AMF1; Miller et al. 2016) near LaPorte, Texas to support the Tracking Aerosol Convection Interactions Experiment (TRACER) (Jensen et al. 2025) near Houston, Texas. From October 2021 to September 2022, AMF1 was deployed to 29.67° N, 95.06° W near LaPorte, Texas and the 2nd Generation C-band Scanning ARM Precipitation Radar (CSAPR2) was deployed to a supplementary site at 29.53° N, 95.28° W (Figure 1). During an intensive operational period (IOP) from 1 June to 30 September 2022, the CSAPR2 sampled precipitation echoes in an adaptive scanning mode following the Multisensor Agile Adaptive Scanning (MAAS) framework (Kollias et al. 2020). MAAS helped optimize the CSAPR2 scan strategy to perform frequent plan position indicator (PPI) and range height indicator (RHI) scans (Lamer et al. 2023). Details of the CSAPR2 scanning, data processing, and calibration procedures used by the principal investigator (PI), and the PI data files are described by Oue et al. (2023). Details of the CSAPR2 operational performance, ARM data processing and correction procedures, and data quality masks are described by Feng et al. (2024a).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

16S and ITS Amplicon Sequencing Fastq files and metadata from PARCHED Panama Tropical Forest soils, 2019-2020,

Model projections predict tropical forests will experience longer periods of drought and more intense precipitation cycles under a changing climate. Such transitions have implications for structure-function relationships within microbial communities. We examine how chronic drying might reshape prokaryotic and fungal communities across four lowland forests in Panama with a wide variation in mean annual precipitation and soil fertility. Four sites were established across a 1000 mm span in mean annual precipitation (2335 to 3300 mm). We expected microbial communities at sites with lower MAP to be less sensitive to chronic drying than sites with higher MAP; while fungal communities to be more resistant to disturbance than prokaryotes. At each location, partial throughfall exclusion structures were established over 10 x 10 m plots to reduce direct precipitation input. Raw demultiplexed sequences (bacteria, archaea, fungal) from soil samples taken from PARCHED Panama Tropical Forest throughfall exclusion experiments. Files that contain 16S are sequences from prokaryotes, ITS indicates sequences from fungi. Compressed fastq files are contained in Field_PARCHED_ITS_fastq_2020.zip, Field_PARCHED_ITS_fastq_2019.zip, Field_PARCHED_16S_fastq_2020.zip, Field_PARCHED_16S_fastq_2019.zip.There are four sites across the isthmus of Panama, each with 4 control and 4 exclusion plots. Throughfall exclusion shelters were built to intercept 50% of throughfall precipitation that hits the soil. Samples were taken on May 2019 and Jan 2020 from 0-10 cm and 10-20 cm depth approximately 9 and 18 months after shelter installation. Metadata and sample IDs for fastq files for 2019 sampling are within Field_PARCHED_Metadata_2019_ITS.csv and Field_PARCHED_Metadata_2019_16S.csv. The metadata for both 16S and ITS fastq files for January 2020 is included in Field_PARCHED_Metadata_2020.csv. No data processing or QA/QC was done on the raw data. Data processing example provided in R notebook file.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Framework for custom event sample augmentations for ATLAS analysis data

For HEP event processing, data is typically stored in column-wise synchronized containers, such as most prominently ROOT’s TTree, which have been used for several decades to store by now over 1 exabyte. These containers can combine row-wise association capabilities needed by most HEP event processing frameworks (e.g. Athena for ATLAS) with column-wise storage, which typically results in better compression and more efficient support for many analysis use-cases. One disadvantage is that these containers, TTree in the HEP use-case, require to contain the same attributes for each entry/row (representing events), which can make extending the list of attributes very costly in storage, even if those are only required for a small subsample of events. Since the initial design, the ATLAS software framework features powerful navigational infrastructure to allow storing custom data extensions for subsamples of events in separate, but synchronized containers. This allows adding event augmentations to ATLAS standard data products (such as DAOD-PHYS or PHYSLITE) avoiding duplication of those core data products, while limiting their size increase. For this functionality, the framework does not rely on any associations made by the I/O technology (i.e. ROOT), however it supports TTree friends and builds the associated index to allow for analysis outside of the ATLAS framework. A prototype based on the Long-Lived Particle search is implemented and preliminary results with this prototype will be presented. At this point, augmented data are stored within the same file as the core data. Storing them in separate files will be investigated in future, as this could provide more flexibility, e.g. certain sites may only want a subset of several augmentations or augmentations can be archived to tape once their analysis is complete.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Plan Position Indicator Hydrometeor Field Statistics (PPIHYD) Evaluation Data Product Version 1.0

The PPIHYD evaluation data product provides distinct hydrometeor field statistics calculated from U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility scanning radar plan position indicator (PPI) scans. These statistics include the equivalent reflectivity factor and Doppler spectral width percentiles, min/max values, and first four moments (mean, standard deviation, skewness, and kurtosis) of distinct hydrometeor features (clustered hydrometeor fields). Statistics also include morphological properties, water content and precipitation rate parameterization-based estimates, and thermodynamic properties interpolated using the Interpolated Sonde value-added product (INTERPSONDE VAP). The data set is organized in tabular form and is accompanied by mask arrays with corresponding indices. This straightforward file structure simplifies scanning radar data processing and renders this data set useful for process understanding and model evaluation studies. This report describes the data set and its processing algorithm and provides some examples.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Crossing the Finish Line: Integration of Data-Driven Process Control for Maximization of Energy and Resource Efficiency in Advanced Water Resource Recovery Facilities

Improvements in process monitoring and control at water resource recovery facilities (WRRFs) could result in reductions in electricity consumption, chemical inputs, and greenhouse gas emissions, as well as improved energy recovery. Many current WRRF data collection, monitoring, and control approaches use 20th century process monitoring and control systems, which require large design safety factors to ensure reliability in the absence of more advanced, precise controls. Implementation of more modern data-driven control tools could lead to more efficient operations that provide intrinsic reliability with better overall process performance at full-scale. This project (1) developed and demonstrated data-driven process controls at full-scale facilities for five promising WRRF process technologies that provide whole-plant approaches and offer substantial energy and resource recovery benefits, and (2) created a Machine Learning (ML) Toolkit and an implementation guide of new process control approaches that walks users through each step of the ML workflow and illustrates the steps through case study examples.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2017

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 22 June - 17 September, 2017. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic Vegetation Warming Experiment data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2021

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June - 17 September, 2021. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2018

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 16 June - 24 September, 2018. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2019

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June – 25 September, 2019. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017–2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

WFIP3 Sentinel Buoy 1 / Raw Data

This dataset contains processed data collected by sentinel buoy 1 during the WFIP3 project period at the WFIP3 location. It was deployed on 6/4/2024 16:35Z and is still currently deployed.

17 WIND ENERGY↗
DOE OSTI2024

WFIP3 Sentinel Buoy 1 / Raw Data

This dataset contains processed data collected by sentinel buoy 1 during the WFIP3 project period at the WFIP3 location. It was deployed on 6/4/2024 16:35Z and is still currently deployed.

17 WIND ENERGY↗
DOE OSTI2025

Permanent Source - Distributed Acoustic Sensing (PESO-DAS) v1

The SOV/DAS software program focuses on seismic monitoring and data processing of permanent seismic sources Surface Orbital Vibrators (SOVs) and fiber-optics sensing Distributed Acoustic Sensing (DAS). Its key features include automated data processing of the continuous seismic monitoring data acquired with DAS, and output of processed shot gathers and QC plots. The software supports timelapse seismic for long-term reservoir monitoring, and is used for monitoring of geological carbon storage sites, geothermal reservoirs, and oil and gas, and in general subsurface resource management.

dos Santos Maia Correa, Julia [Lawrence Berkeley N↗
DOE OSTI2020

Use of the Tool to Support Renewable Energy Auctions Processes (RE Data Explorer)

Renewable energy auctions are now a common competitive approach to procure low-cost renewable power around the world. Ensuring a successful auction process increasingly depends on the capabilities of auction designers and participants to identify actionable and defensible insights from large data sets (on renewable energy resources and complementary data) to both attract potential investors and address stakeholder concerns. The Renewable Energy (RE) Data Explorer is a user-friendly geospatial analysis tool for analyzing renewable energy potential and informing decisions. Developed by the National Renewable Energy Laboratory (NREL) and supported by the U.S. Agency for International Development (USAID), RE Data Explorer performs visualization and analysis of renewable energy potential that can be customized for different scenarios. RE Data Explorer can support prospecting, integrated planning, policymaking, and other decision-making activities to accelerate renewable energy deployment. The broader RE Explorer website provides guidance and information to link the RE Data Explorer geospatial analysis tool to key decision areas. This document provides information on how the RE Data Explorer can be used to support renewable energy auction processes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Feature Analysis, Tracking, and Data Reduction: An Application to Multiphase Reactor Simulation MFiX-Exa for In-Situ Use Case

As we enter the exascale computing regime, powerful supercomputers continue to produce much higher amounts of data than what can be stored for offline data processing. To utilize such high compute capabilities on these machines, much of the data processing needs to happen in situ, when the full high-resolution data is available at the supercomputer memory. In this article, we discuss our MFiX-Exa simulation, which models multiphase flow by tracking a very large number of particles through the simulation domain. In one of the use cases, the carbon particles interact with air to produce carbon dioxide bubbles from the reactor. These bubbles are of primary interest to the domain experts for these simulations. For this particle-based simulation, we propose a streaming technique that can be deployed in situ to efficiently identify the bubbles, track them over time, and use them to down-sample the data with minimal loss in these features.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Focused Ion Beam Tomography of Alloy 617 Corroded in Molten Chloride Salt

Materials qualification of reactor structural materials is a critical step in rapid implementation of advanced nuclear reactor technologies, particularly to assess the corrosion performance in these designs. Accelerated qualification of reactor structural materials requires incorporating powerful computational toolsets, such as phase field modelling in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, to predict the evolution of structural materials due to corrosion. Accordingly, computational toolsets will require experimental data generated at appropriate length scales to validate accuracy. Focused ion beam (FIB) provides a high degree of control over manipulation of materials for analytical purposes, including capturing data on the evolution in the microstructure and elemental composition of materials at the mesoscale, an appropriate length scale for phase field modelling of intergranular diffusion phenomena using the MOOSE framework. For instance, the FEI Helios G4 UX dual beam plasma FIB microscope at the Irradiated Materials Characterization Laboratory (IMCL) is capable of backscatter diffraction (EBSD) and energy-dispersive x-ray spectroscopy (EDS) documenting the evolution in the microstructure and elemental composition, respectively. The Helios can perform EDS and EBSD three-dimensionally (3D) using tomography, which is then combined using different software packages to visualize 3D volumes correlating elemental composition to microstructural data. The purpose of this investigation was to develop a streamlined characterization and data processing workflow for 3D tomography studies on the FEI Helios G4 plasma FIB. The investigation is segmented into three parts: 1) Optimizing the data collection workflow, 2) identifying appropriate data processing and visualization software (i.e. DREAM.3D, MIPAR, and VGStudioMax), and 3) establishing an infrastructure for public release. The optimization of the data collection workflow is in collaboration with members of the U220 department to setup formal training on the tomography operation of the G4, through ThermoFisher Scientific, and exploring DREAM.3D, MIPAR, and VGStudioMax data processing/visualization software packages. VGStudioMax currently demonstrates the most promise for future use. Optimization of the data collection and processing workflow is still ongoing. A collaboration with INL High Performance Computing (HPC) established an open-source license for expediting the public release of FIB tomography datasets through HPC. FIB tomography data generated by the G4 will provide comprehensive data for validating 3D phase field mesoscale modelling tools within the MOOSE framework for accelerated qualification of reactor structural materials.

Copeland-Johnson, Trishelle↗