Search NASA⌕ Search

SEARCH · Search NASA

Results for “process data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Development of a framework for sequential Bayesian design of experiments: Application to a pilot-scale solvent-based CO 2 capture process

In this paper, a methodology is developed for sequential design of experiments (SDoE) for process systems and applied to a solvent-based CO 2 capture system. In this approach, the prior knowledge of the system is used to prioritize process data collection at specific operating conditions. These data are then incorporated into a Bayesian inference methodology for updating a stochastic model by refining estimations of its underlying parameters, and the updated model is then used to generate the next set of test runs. Thus, the new knowledge obtained from the data is used to guide subsequent iterations of the experimental runs, ensuring that the overall data collection is maximally informative given that most experimental campaigns, especially at pilot or higher-scale plants, are costly, time-consuming, and resource-limited. The test run objective for this work was to minimize the maximum model prediction uncertainty for key output variables, but the methodology is generic and can be readily applied to other test run objectives. This methodology is applied to an aqueous monoethanolamine (MEA) pilot plant campaign at the National Carbon Capture Center (NCCC) in Wilsonville, Alabama, USA. The SDoE framework was utilized for two iterations, while collecting 18 sets of data representing different process conditions, and this resulted in an overall average reduction in uncertainty of approximately 50% in the prediction of CO 2 capture percentage. Moreover, 11 additional data sets were obtained with variation of absorber packing height for further model validation. This work shows the capability of the SDoE framework to maximize learning given limited resources, allowing for the reduction of model uncertainty, which is of great importance for many applications including reduction of technical risk associated with scale-up and economic analysis.

20 FOSSIL-FUELED POWER PLANTS↗

Characterization of Soil and Rock Magnetic Properties along Multiple Hillslope Transects at Teller Road Site, Seward Peninsula, Alaska, 2018 and 2023

The magnetometer data was collected in multiple directions across the watershed hillslope at the NGEE Arctic Teller Road site at mile marker 27 (TL_MM27) on the Seward Peninsula, Alaska over multiple years in March 2018 and April 2023. The magnetic data were collected using a Geometrics Inc. G-858 gradiometer and G-857 base station in 2018 and the G-864 gradiometer and G857 base station in 2023. The data was collected (in all instances) by towing the gradiometer behind a snow machine around the watershed with the two sensors in a vertical profile with constant spacing during the continuous survey in that specific year. Magnetic total field measurements were collected by gradiometer and base station, and the data processing was performed in Geometrics MagMap2000 software. The processing steps were limited to removal of data spikes (despiking), reading dropouts, and correction/removal of bad GPS points. All offsets between sensors and GPS are stated within the data files and metadata, alongwith the processed and raw data. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).In this data submission there are two sets of raw magnetic data (.bin and .stn for 2018 and base for 2023; raw rover mag for 2023 is in .csv) inside two .zip files that identify the year the mag data was collected. The data are proprietary format to Geometrics and can be opened and processed with MagMap2000 which can be downloaded for free at Geometrics website. There are also two processed data files *.csv for each year and two metadata files *.csv.

54 ENVIRONMENTAL SCIENCES↗

Surface Water Disinfection Byproducts and Organic Matter Characterization Data Associated with: “Disinfection byproducts formed during drinking water treatment reveal an export control point for dissolved organic matter in a subalpine headwater stream”

This dataset is associated with the publication “Disinfection byproducts formed during drinking water treatment reveal an export control point for dissolved organic matter in a subalpine headwater stream” published in Water Research X (Leonard et al. 2022; https://doi.org/10.1016/j.wroa.2022.100144). The associated study analyzed temporal trends from the Town of Crested Butte water treatment facility and synoptic sampling at Coal Creek in Crested Butte, Colorado, US. This work demonstrates how drinking water quality archives combined with synoptic sampling and targeted analyses can be used to identify and understand export control points for dissolved organic matter. This dataset is comprised of one main data folder containing (1) file-level metadata; (2) data dictionary; (3) metadata and international geo-sample number (IGSN) mapping file; (4) dissolved organic carbon (DOC), ultraviolet absorbance at 254 nanometers (UV254), total nitrogen (TN), and specific ultraviolet absorbance (SUVA) data; (5) disinfection bioproduct formation potential (DBP-FP) data; (6) readme; (7) methods codes; (8) water collection protocol; (9) folder of high resolution characterization of organic matter via 12 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory); and (10) folder of excitation emissions matrix (EEM) spectra. The FTICR folder contains a file of DOC (measured as non-purgeable organic carbon; NPOC) used for FTICR sample preparation. The FTICR folder also contains three subfolders: one subfolder containing the raw .xml data files, one containing the processed data, and the other containing instructions for using Formularity (https://omics.pnl.gov/software/formularity) and an R script to process the data based on the user's specific needs. All files are .csv, .pdf, .dat, .R, .ref, or .xml

54 ENVIRONMENTAL SCIENCES↗

Hod Carrier

SAND2023-06720O Hod Carrier is a simple proof-of-concept library for moving data from host to NVIDIA BlueField device memory using remote direct memory access. The Hod Carrier software also facilitates research activities related to the potential uses of NVIDIA BlueField data processing unit devices. Using well-established technologies (e.g., RDMA over Infiniband), it will move data from the host to the data processing unit’s memory. This software is a middleware utility for moving data without regard to its semantic meaning. Its operation is fundamentally unaffected by storage allocation. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗

Grid Edge Waveform Analytics Framework for Event Detection and Classification

This paper provides a grid edge waveform analytics framework for power system event detection and classification in the local as well as in the wide area. This framework overviews data excellence for event detection and classification. The data excellence describes the data acquisition process and requirements, data processing, data quality, and data integrity. Power system event detection in the local area based on different features such as energy-based, cyclostationary approach, template matching, and wavelet transform are also discussed. Furthermore, local area event detection and classification using approaches such as statistical, signal processing, artificial intelligence, and hybrid are also discussed. Moreover, an overview of wide-area event detection and classification along with several other aspects such as wide-area events, wide-area event detection approaches, event location and system performance, event pattern recognition, inter-area oscillation, and wide-area frequency response under variable deployment of inverter-based resources are also provided. The proposed framework is the first step toward the goal of developing appropriate tools and methodologies to detect and classify local as well as wide-area events using waveform analytics. The appropriate event detection and classification framework development is especially important now as more and more grid edge devices with communication capabilities are being deployed in the modern power grid than ever before.

Bhusal, Narayan↗

Machine learning on FPGA for event selection

Real-time data processing is a frontier field in experimental particle physics. The application of FPGAs at the trigger level is used by many current and planned experiments (CMS, LHCb, Belle2, PANDA). Usually they use conventional processing algorithms. LHCb has implemented Machine Learning (ML) elements for real-time data processing with a triggered readout system that runs most of the ML algorithms on a computer farm. The work described in this article aims to test the ML-FPGA algorithms for streaming data acquisition. Herein, there are many experiments working in this area and they have a lot in common, but there are many specific solutions for detector and accelerator parameters that are worth exploring further. This report describes the purpose of the work and progress in evaluating the ML-FPGA application.

47 OTHER INSTRUMENTATION↗

SKA shakes hands with Summit

Recently, a full-scale data processing workflow (Fig. 1) of the Square Kilometre Array (SKA) Phase 1 was successfully executed on the world’s fastest supercomputer Summit, proving that scientists have the expertise, software tools and computing resources to process the SKA data.

79 ASTRONOMY AND ASTROPHYSICS↗

Author Correction: US oil and gas system emissions from nearly one million aerial site measurements

Correction to: Naturehttps://doi.org/10.1038/s41586-024-07117-5 Published online 13 March 2024 In the version of the article initially published, several errors were present and have been corrected in the HTML and PDF versions of the article and Supplementary Information. The main results, conclusions, and our interpretations of the data remain unchanged. See the new Supplementary Information Section S15 for a more detailed description of the errors corrected and the resulting effects on the analysis. Data processing and methods corrections Overflight count correction: We previously used pre-computed source coverage data for some Carbon Mapper campaigns that was computed differently than was required for our analysis. We have re-computed Carbon Mapper source coverage based on flightline polygons and source coordinates. Transition point computation, well sites: The updated version now correctly compares the cumulative emissions distribution of simulated well site emissions with that of aerially detected sources (rather than plumes) when computing the transition point. Transition point computation, midstream: Additionally, the transition point calculation has been corrected to exclude aerially detected midstream emissions below the transition point, which was previously leading to double counting of these emissions. This error was not present for upstream (well site) emissions. Calculation errors Unit error: We corrected a specific unit conversion error affecting well site emissions in the Kairos Fort Worth dataset. Across all datasets, we also correct the conversion factor for converting from standard volume to mass for midstream emissions. Sorting error: We correct code that was applying incorrect sorting when computing correction factors to account for partial detection at well sites. Small typographical corrections were made in Fig. 1b and SI Section S4.1. Data processing and methods corrections Overflight count correction: We previously used pre-computed source coverage data for some Carbon Mapper campaigns that was computed differently than was required for our analysis. We have re-computed Carbon Mapper source coverage based on flightline polygons and source coordinates. Transition point computation, well sites: The updated version now correctly compares the cumulative emissions distribution of simulated well site emissions with that of aerially detected sources (rather than plumes) when computing the transition point. Transition point computation, midstream: Additionally, the transition point calculation has been corrected to exclude aerially detected midstream emissions below the transition point, which was previously leading to double counting of these emissions. This error was not present for upstream (well site) emissions. Calculation errors Unit error: We corrected a specific unit conversion error affecting well site emissions in the Kairos Fort Worth dataset. Across all datasets, we also correct the conversion factor for converting from standard volume to mass for midstream emissions. Sorting error: We correct code that was applying incorrect sorting when computing correction factors to account for partial detection at well sites. Small typographical corrections were made in Fig. 1b and SI Section S4.1. The following practices may help researchers conducting similar analyses avoid making similar errors: 1, Clear, accessible documentation explaining the interpretation of all columns in data input tables and all internal variables within the model, 2, Simple cross-check calculations computed before and after unit conversions.

Sherwin, Evan D↗

Planck 2018 results - IV. Diffuse component separation

We present full-sky maps of the cosmic microwave background (CMB) and polarized synchrotron and thermal dust emission, derived from the third set of Planck frequency maps. These products have significantly lower contamination from instrumental systematic effects than previous versions. The methodologies used to derive these maps follow closely those described in earlier papers, adopting four methods (Commander, NILC, SEVEM, and SMICA) to extract the CMB component, as well as three methods (Commander, GNILC, and SMICA) to extract astrophysical components. Our revised CMB temperature maps agree with corresponding products in the Planck 2015 delivery, whereas the polarization maps exhibit significantly lower large-scale power, reflecting the improved data processing described in companion papers; however, the noise properties of the resulting data products are complicated, and the best available end-to-end simulations exhibit relative biases with respect to the data at the few percent level. Using these maps, we are for the first time able to fit the spectral index of thermal dust independently over 3° regions. We derive a conservative estimate of the mean spectral index of polarized thermal dust emission of β d = 1.55 ± 0.05, where the uncertainty marginalizes both over all known systematic uncertainties and different estimation techniques. For polarized synchrotron emission, we find a mean spectral index of β s = -3.1 ± 0.1, consistent with previously reported measurements. We note that the current data processing does not allow for construction of unbiased single-bolometer maps, and this limits our ability to extract CO emission and correlated components. The foreground results for intensity derived in this paper therefore do not supersede corresponding Planck 2015 products. For polarization the new results supersede the corresponding 2015 products in all respects.

79 ASTRONOMY AND ASTROPHYSICS↗

Euclid I. Overview of the Euclid mission

The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015–2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14 000 deg 2 of extragalactic sky. In addition to accurate weak lensing and clustering measurements that probe structure formation over half of the age of the Universe, its primary probes for cosmology, these exquisite data will enable a wide range of science. This paper provides a high-level overview of the mission, summarising the survey characteristics, the various data-processing steps, and data products. We also highlight the main science objectives and expected performance.

cosmology↗

DUNE DAQ R&D integration in ProtoDUNE Single-Phase at CERN

The DAQ system of ProtoDUNE-SP successfully proved its design principles and met the requirements of the beam run of 2018. The technical design of the DAQ system for the DUNE experiment has major differences compared to the prototype due to different requirements placed on the detector, as well as a radically different location of operation. The single-phase prototype at CERN is a major integration facility for R&D aspects of the DUNE DAQ system. The facility allows for the exploration of additional data processing capabilities and optimization of the FELIX system, which is the chosen TPC readout solution for the DUNE single-phase detectors. One of the fundamental differences from the prototype is that the DUNE DAQ relies on self-triggering. Therefore, real-time processing of the data stream for hit and trigger primitive finding is essential for the requirement of continuous readout. The supernova burst trigger requires a large and fast buffering technique, where 3D XPoint persistent memory solutions are evaluated and integrated. In order to maximize resource utilization of the FELIX hosting servers, the elimination of the 100 Gb network communication stack is desired. This implies the design and development of a single-host application layer, which is a fundamental element of the self-triggering chain. This paper discusses the evaluation and integration of these developments for the DUNE DAQ, in the ProtoDUNE environment.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Towards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures

Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load—derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.

kilic, Ozgur Ozan [Brookhaven National Laboratory ↗

DOE JGI Metagenome Workflow

The DOE Joint Genome Institute (JGI) Metagenome Workflow performs metagenome data processing, including assembly; structural, functional, and taxonomic annotation; and binning of metagenomic data sets that are subsequently included into the Integrated Microbial Genomes and Microbiomes (IMG/M) (I.-M. A. Chen, K. Chu, K. Palaniappan, A. Ratner, et al., Nucleic Acids Res, 49:D751–D763, 2021, https://doi.org/10.1093/nar/gkaa939) comparative analysis system and provided for download via the JGI data portal (https://genome.jgi.doe.gov/portal/). This workflow scales to run on thousands of metagenome samples per year, which can vary by the complexity of microbial communities and sequencing depth. Here, we describe the different tools, databases, and parameters used at different steps of the workflow to help with the interpretation of metagenome data available in IMG and to enable researchers to apply this workflow to their own data. We use 20 publicly available sediment metagenomes to illustrate the computing requirements for the different steps and highlight the typical results of data processing. The workflow modules for read filtering and metagenome assembly are available as a workflow description language (WDL) file (https://code.jgi.doe.gov/BFoster/jgi_meta_wdl). The workflow modules for annotation and binning are provided as a service to the user community at https://img.jgi.doe.gov/submit and require filling out the project and associated metadata descriptions in the Genomes OnLine Database (GOLD) (S. Mukherjee, D. Stamatis, J. Bertsch, G. Ovchinnikova, et al., Nucleic Acids Res, 49:D723–D733, 2021, https://doi.org/10.1093/nar/gkaa983).

59 BASIC BIOLOGICAL SCIENCES↗

ampworks: Battery analysis tools in Python [SWR-25-39]

Ampworks is a collection of tools designed to process experimental battery data with a focus on model-relevant analyses. It currently provides functions for incremental capacity analysis and GITT data processing, helping extract key properties for life and physics-based models (e.g., SPM and P2D). Some tools, like the incremental capacity analysis module, also include graphical user interfaces for ease of use. https://github.com/NREL/ampworks/ https://pypi.org/project/ampworks/

Randall, Corey [National Renewable Energy Laborato↗

Homomorphic data compression for real time photon correlation analysis

The construction of highly coherent X-ray sources, combined with next-generation detectors that are larger and faster, has enabled new research opportunities across the scientific landscape. Among the techniques that benefit most from these advancements is X-ray photon correlation spectroscopy (XPCS), where faster acquisition unlocks the ability to study faster dynamics within samples. However, faster acquisition on larger detectors also introduces unprecedented challenges for online data processing and offline data storage. Such challenges are particularly prominent for XPCS, where real time analyses require simultaneous calculation of all the previously acquired data in the time series. We present a homomorphic compression scheme to effectively reduce the computational time and memory space required for XPCS analysis. Leveraging similarities in the mathematical expression between a matrix-based compression algorithm and the correlation calculation, our approach allows direct operation on the compressed data without their decompression. The offline compression scheme extends storage capacity by a factor of 40 while preserving key features in the lossy compressed data. Meanwhile, the online compression scheme reduces the computational time to below 1 ms, enabling real time calculation of the correlation functions at kHz framerate. Our demonstration of a homomorphic compression of scientific data provides an effective solution to the big data challenge at coherent light sources. Beyond the example shown in this work, the framework can be extended to facilitate real-time operations directly on a compressed data stream for other techniques.

36 MATERIALS SCIENCE↗

HERO WEC V1 Upgrade - 2023 Laboratory Testing (Raw Data)

This submission contains the original, unprocessed data from the 2023 Large Amplitude Motion Platform (LAMP) testing of NREL's Hydraulic and Electric Reverse Osmosis Wave Energy Converter (HERO WEC). This data serves as a companion to MHKDR #520. Data was collected using NREL's Modular Ocean Data AcQuisition (MODAQ) system in TDMS format. Specifications of TDMS files can be found on the NI website. The TDMS files have been separated into zip files corresponding to either Drivetrain, Hydraulic, or Electric configuration runs representing the respective test cases that were run. The drivetrain runs were used to characterize the drivetrain only (no pump or generator). The Hydraulic runs represent the configuration when the seawater pump is installed, and the Electric runs represents the configuration when the generator is installed. The following sub-categories of data are included for each type: - DW - Deep water (monochromatic sine wave) profile (not run in drivetrain configuration) - Heave - Heave only (monochromatic sine wave) profile - Heave_NoRO (hydraulic configuration only) - Heave_ACC (hydraulic configuration only) - IR - Surge and heave irregular wave profile (not run in drivetrain configuration) - RW - Heave only profile created from real world encoder data (not run in drivetrain configuration) Reference documents: - "HERO WEC Lamp Test Run Log.xlsx": contains specifications for each test run - "Lamp Data Description.docx": provides detailed information about data types and processing methods For those interested in the processed data the authors have created a separate submission, MHKDR #520, linked below. This data set has been developed by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Water Power Technologies Office.

16 TIDAL AND WAVE POWER↗

Hestia SW-IFL Onroad Fossil Fuel Carbon Dioxide (FFCO2) product: Road segment-level annual FFCO2 emissions across Arizona (2017-2022), version 1.1

The SW-IFL onroad fossil fuel carbon dioxide (FFCO2) emissions data product represents CO2 emissions from the combustion of fossil fuels by motor vehicles (e.g., passenger cars, trucks, buses, motorcycles) traveling on designated roadways. The emissions are represented geographically on each road segment within the state of Arizona spanning the 2017 to 2022 time period. This data product was developed as part of the Southwest Urban Corridor Integrated Field Laboratory (SW-IFL) project, which aims to provide new knowledge and tools that address urban environmental issues by integrating high-resolution observations, modeling, and civic engagement. The emissions data are provided in CSV (input data, ONR_FFCO2_AZ_county.csv) and GeoPackage form (output polyline objects - about 786,000 road segments, XXXX_AZ_v1.1.gpkg) designated by road class (interstates, arterials, collectors, local). The metadata file (Metadata_SW-IFL_Onroad_annualFFCO2_v1.1.docx) provides details about attributes and data formats. The GeoPackage emissions data are provided separately for local roads and nonlocal roads (interstates, arterials, collectors). The method file (Methods_SW-IFL_Onroad_annualFFCO2_v1.1.docx) describes the data processing flow and data sources. Update on 2024-04-17: Updates were made to both the input emission data file (.csv) and output segment-level emission file (.gpkg). There was an update in county-level emission input data (ONR_FFCO2_AZ_county.csv) and the entire road segments were reprocessed to reflect this update.Update on 2024-04-29: Update was made to one output segment-level emission file (Nonlocal_AZ_v1.0.gpkg). There was an error in the AADT values and the data were reprocessed to reflect this update.Update on 2024-10-22: Temporal coverage was extended to include 2022. VMT values were recalculated using new AADT data and the entire road segments were reprocessed to reflect these updates.

54 ENVIRONMENTAL SCIENCES↗

X-ray Absorption Spectroscopy and Neutron Scattering Data, Mineral Incubation Experiment, Walker Branch Watershed, TN (2020 - 2021)

Iron (Fe) (oxyhydr)oxides are well-recognized contributors to soil carbon (C) storage, but the effects of manganese (Mn) oxides on carbon storage and transformation are relatively unexplored. Here, the relative capacities of Fe and Mn oxides to bind and stabilize soil organic C were directly compared using an in-situ incubation experiment. Quartz sands coated with either poorly crystalline Mn(III/IV) oxides or Fe(III) oxides, or left uncoated, were buried in a temperate forest soil for up to one year. This data package contains processed data outputs of carbon near edge x-ray absorption fine structure spectroscopy (C NEXAFS), iron and manganese x-ray absorption near edge structure spectroscopy (XANES), and small angle neutron scattering (SANS) data collected for initial oxide-coated and uncoated quartz sands and for those buried in the temperate forest soil for 365 days.

Energy (eV)↗