Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Cosmic Shear Analysis in the DECam Local Volume Exploration Survey

We forecast cosmological constraints and develop a cosmic shear analysis pipeline for the DECam Local Volume Exploration Survey (DELVE). We test the effects of two different intrinsic alignment frameworks (TATT and NLA) on simulated data vectors. In addition, we examine the impact of baryon contamination and apply scale cuts to reduce its weight. We find the forecast results to be as constraining as the DES Y3 cosmological parameter measurements.

79 ASTRONOMY AND ASTROPHYSICS↗

Techno Economic Analysis for Production of Renewable Natural Gas and Value-Added Chemicals from Forest Biomass Residues (CRADA Final Report)

West Biofuels - in collaboration with the University of California San Diego, the University of California Davis, the National Laboratory of the Rockies (NLR), the Colorado School of Mines’ Center for Hydrate Research, Placer County Air Pollution Control District, the Sierra Business Council, and the Southern California Gas Company (SoCalGas)-will demonstrate an innovative pathway to convert forest biomass to renewable gas (RG). The project, Production of Pipeline Grade Renewable Natural Gas and Value-Added Chemicals from Forest Biomass Residues, will utilize an existing pilot-scale gasification system and data and lessons learned from a lab-scale catalytic reactor design, develop, and demonstrate an integrated pilot-scale RG process with a scaled-up catalytic reactor and hydrate-based gas separation process. This robust and simple process uses a commercial proven gasifier, a commercially available catalyst that can tolerate gas contaminants, and a gas separation process that simplifies the downstream processing. Current testing has shown that the process can yield a large amount of methane rich renewable gas in addition to a mixture of propanol, ethanol and other higher alcohols and the relative amounts can be controlled by shifting the process conditions. This process is novel and groundbreaking because the product mixture of RG and valuable alcohols byproducts makes RG production economically feasible at $\$$12 per million British thermal units (MMBtu) or less using forest biomass from high hazard zones.

09 BIOMASS FUELS↗

ARES. II. Characterizing the Hot Jupiters WASP-127 b, WASP-79 b, and WASP-62b with the Hubble Space Telescope

This paper presents the atmospheric characterization of three large, gaseous planets: WASP-127 b, WASP-79 b, and WASP-62 b. We analyzed spectroscopic data obtained with the G141 grism (1.088–1.68 μm) of the Wide Field Camera 3 on board the Hubble Space Telescope using the Iraclis pipeline and the TauREx3 retrieval code, both of which are publicly available. For WASP-127 b, which is the least dense planet discovered so far and is located in the short-period Neptune desert, our retrieval results found strong water absorption corresponding to an abundance of log(H{sub 2}O) = −2.71{sub −1.05}{sup +0.78} and absorption compatible with an iron hydride abundance of log(FeH) = −5.25{sub −1.10}{sup +0.88}, with an extended cloudy atmosphere. We also detected water vapor in the atmospheres of WASP-79 b and WASP-62 b, with best-fit models indicating the presence of iron hydride, too. We used the Atmospheric Detectability Index as well as Bayesian log evidence to quantify the strength of the detection and compared our results to the hot Jupiter population study by Tsiaras et al. While all the planets studied here are suitable targets for characterization with upcoming facilities such as the James Webb Space Telescope and Ariel, WASP-127 b is of particular interest due to its low density, and a thorough atmospheric study would develop our understanding of planet formation and migration.

79 ASTRONOMY AND ASTROPHYSICS↗

Cosmic Shear Analysis of the DECam Local Volume Exploration Survey

We forecast cosmological constraints and develop a cosmic shear analysis pipeline for the DECam Local Volume Exploration Survey (DELVE). We test the effects of two different intrinsic alignment frameworks (TATT and NLA) on synthetic data vectors. In addition, we examine the impact of baryon contamination and determine the necessary scale cuts to reduce its influence. We find the forecast results to be as constraining as the DES Y3 cosmological parameter measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗

Roughrider Carbon Storage Hub (Final Report)

The Roughrider Carbon Storage Hub was a 2-year project (October 2023 – September 2025) conducted by the Energy & Environmental Research Center (EERC) focused on advancing the feasibility of a commercial-scale carbon dioxide (CO 2 ) geologic storage hub in McKenzie County, North Dakota. The project’s objective was to investigate the potential that stacked storage complexes (multiple deep saline formations) can safely and economically store at least 50 million tonnes of CO 2 within 30 years. The captured CO 2 would be sourced from industrial emitters including project partner ONEOK, Inc.’s gas-processing plants and a planned gas-to-liquids facility. Drilling of the Roughrider 1 stratigraphic test well (14,979-ft total depth) was completed in November 2024. The wellbore intersected four candidate storage formations: Inyan Kara, Broom Creek, Mission Canyon, and Black Island–Deadwood. Operational challenges, including a stuck drill string, were resolved without long-term impact. A comprehensive logging and coring program was conducted, followed by successful well abandonment and site reclamation. Over 660 ft of 4-in. whole core was retrieved. Core plug samples were processed and analyzed for petrophysical and geochemical properties. Results confirmed promising porosity and permeability in the Inyan Kara and Broom Creek Formations and removal of the Mission Canyon and Black Island–Deadwood horizons from further investigation. Data derived from the logging and coring program were used to improve initial geologic models built from legacy data. CO 2 injection simulations showed that the Inyan Kara alone can feasibly store the target mass of CO 2 . Because of subtle differences in geologic structure and porosity trends between the formations, a stacked storage scenario using the Broom Creek and Inyan Kara Formations resulted in a larger overall plume area than using the Inyan Kara alone. Preliminary CO 2 pipeline routes from the industrial sources were mapped utilizing existing rights of way and evaluated for capacity and cost using U.S. Department of Energy Office of Fossil Energy and Carbon Management/National Energy Technology Laboratory models and U.S. Environmental Protection Agency emissions data. Integrating capture, transport, and storage cost estimates with policy incentives (e.g., 45Q credits) provided a total cost-per-ton analysis. Results indicate that the small scale of the volumes to be transported over the cumulative large distances does not support the project’s financial viability. However, the groundwork laid during this project from geological, regulatory, and social perspectives positions the Roughrider hub site as a promising candidate for commercial carbon storage in North Dakota, especially if the economy of scale is introduced for CO 2 transportation to the hub site.

01 COAL, LIGNITE, AND PEAT↗

A Centralized AI Lakehouse Framework for Brain Tumor MRI Classification and Segmentation, University KPI Forecasting, and Water Potability Prediction

In many university and healthcare projects, models are built for very different data types such as tables, institutional time series, and medical images, but they are deployed as separate applications. In this work, that separation made testing and maintenance difficult because each module had its own pipeline and runtime requirements. This paper presents an integrated AI lakehouse-style implementation that runs three model pipelines inside one containerized backend. For medical imaging, we used MRI datasets from IEEE DataPort: a four-class classification set with 7012 images (5708 train/1304 test) and a segmentation set with 3063 image–mask pairs. The classification model (ResNet50 transfer learning) is evaluated using a proper train–validation–test protocol across multiple splits (80/10/10, 70/10/20, 60/10/30, and 10/30/60), achieving a test accuracy of 99.00% under the standard 80/10/10 split. Additionally, a patient-level evaluation is conducted using an external glioma dataset to provide a more realistic assessment without data leakage. The segmentation model (DeepLabV3-ResNet50) achieved 83.09% validation mIoU and 88.79% Dice score. For university KPI forecasting, we used annual IPEDS and NSF HERD data from 2010 to 2023 for three universities (BSU, EOU, and UAB). To examine the effect of preprocessing on forecasting performance, two case studies are conducted. In the first case, linear interpolation is applied to generate semester-level data. In the second case, the original annual data is used directly without interpolation. Random Forest regression and ARIMA models are evaluated using MAE, RMSE, MAPE, and R 2 . The results showed that interpolation improved apparent forecasting performance due to smoothing, while evaluation on the original annual data provided a more realistic assessment of model behavior. To further validate the framework on a larger dataset, an additional case study is conducted using a student dropout dataset. For water potability, we trained and compared multiple tabular classifiers on a large dataset (1,048,575 samples). A Random Forest model (100 trees, max depth 10) achieved 85.86% test accuracy and high recall for unsafe samples (0.8447). All modules are served via FastAPI and deployed together using Docker, with workflow automation routing requests to the correct endpoint. System-level benchmarking indicates that the backend maintains stable throughput and latency under concurrent requests.

97 MATHEMATICS AND COMPUTING↗

SynBio QC-Library QC (LibQC) v2.0.0

This software is a pipeline used to identify the distribution of user-specified sequences in oligo pools that have been sequenced on the MiSeq platform for quality control purposes. The software counts instances of user-specified sequences (inserts or spacers) identified in sequencing data, returning the distribution of sequences in each pool and the length distribution of all the sequences. Among other types similar types of projects, this can be used to assess the quality of a guideRNA library once it has been inserted into the vector of choice to ensure that there is an even distribution of all the individual protospacers in the pool. This pipeline uses many freely available open source libraries, the value added is that in our application the steps of the pipeline are defined in Workflow Description Language (WDL) and run through the Cromwell workflow engine in Docker containers, for easy distribution and set up.

Simirenko, Lisa↗

Dark Energy Survey Year 3 results: Simulation-based cosmological inference with wavelet harmonics, scattering transforms, and moments of weak lensing mass maps. Validation on simulations

Beyond-two-point statistics contain additional information on cosmological as well as astrophysical and observational (systematics) parameters. In this methodology paper we provide an end-to-end simulationbased analysis of a set of Gaussian and non-Gaussian weak lensing statistics using detailed mock catalogs of the Dark Energy Survey (DES). Here, we implement: 1) second and third moments; 2) wavelet phase harmonics (WPH); 3) the scattering transform (ST). Our analysis is fully based on simulations, it spans a space of seven $νw$CDM cosmological parameters, and it forward models the most relevant sources of systematics of the data (masks, noise variations, clustering of the sources, intrinsic alignments, and shear and redshift calibration). We implement a neural network compression of the summary statistics, and we estimate the parameter posteriors using a likelihood-free-inference approach. We validate the pipeline extensively, and we find that WPH exhibits the strongest performance when combined with second moments, followed by ST, and then by third moments. The combination of all the different statistics further enhances constraints with respect to second moments, up to 25 percent, 15 percent, and 90 percent for S 8 , Ω m , and the figure-of-merit FoM S8;Ωm , respectively. We further find that non-Gaussian statistics improve constraints on w and on the amplitude of intrinsic alignment with respect to second moments constraints. The methodological advances presented here are suitable for application to Stage IV surveys from Euclid, Rubin-LSST, and Roman with additional validation on mock catalogs for each survey. In a companion paper we present an application to DES Year 3 data.

79 ASTRONOMY AND ASTROPHYSICS↗

Meta2DB: Curated Shotgun Metagenomic Feature Sets and Metadata for Health State Prediction

Meta2DB is a curated metagenomic and metadata database that provides structurally consistent microbiome taxonomy feature count tables for 13 897 samples across 84 studies, 23 disease states, and 34 geographical locations. All samples were uniformly processed using a streamlined metagenomic classification pipeline that employs a unique and comprehensive reference database indexed to contain all sequences across all kingdoms of life that were present in the NCBI Nucleotide (nt) database retrieved on 4 January 2023. This pipeline leverages high-performance computing (HPC) resources at Lawrence Livermore National Laboratory and was used to process 50TB of publicly available raw metagenomic sequence data. Extensive metadata curation was carried out through a combination of manual curation and automated parsing, producing a consistent inter-study metadata table specifically structured to facilitate training of ML models for prediction of human health.

Kok, C [Lawrence Livermore National Laboratory (LL↗

Galba: genome annotation with miniprot and AUGUSTUS

The Earth Biogenome Project has rapidly increased the number of available eukaryotic genomes, but most released genomes continue to lack annotation of protein-coding genes. In addition, no transcriptome data is available for some genomes. Various gene annotation tools have been developed but each has its limitations. Here, we introduce GALBA, a fully automated pipeline that utilizes miniprot, a rapid protein-to-genome aligner, in combination with AUGUSTUS to predict genes with high accuracy. Accuracy results indicate that GALBA is particularly strong in the annotation of large vertebrate genomes. We also present use cases in insects, vertebrates, and a land plant. GALBA is fully open source and available as a docker image for easy execution with Singularity in high-performance computing environments. Our pipeline addresses the critical need for accurate gene annotation in newly sequenced genomes, and we believe that GALBA will greatly facilitate genome annotation for diverse organisms.

59 BASIC BIOLOGICAL SCIENCES↗

Scalable All-pairs Shortest Paths for Huge Graphs on Multi-GPU Clusters

We present an optimized Floyd-Warshall (Floyd-Warshall) algorithm that computes the All-pairs shortest path (APSP) for GPU accelerated clusters. The Floyd-Warshall algorithm due to its structural similarities to matrix-multiplication is well suited for highly parallel GPU architectures. To achieve high parallel efficiency, we address two key algorithmic challenges: reducing high communication overhead and addressing limited GPU memory. To reduce high communication costs, we redesign the parallel (a) to expose more parallelism, (b) aggressively overlap communication and computation with pipelined and asynchronous scheduling of operations, and (c) tailored MPI-collective. To cope with limited GPU memory, we employ an offload model, where the data resides on the host and is transferred to GPU on-demand. The proposed optimizations are supported with detailed performance models for tuning. Our optimized parallel Floyd-Warshall implementation is up to 5x faster than a strong baseline and achieves 8.1 PetaFLOPS/sec on 256~nodes of the Summit supercomputer at Oak Ridge National Laboratory. This performance represents 70% of the theoretical peak and 80% parallel efficiency. The offload algorithm can handle 2.5x larger graphs with a 20% increase in overall running time.

Sao, Piyush↗

Offshore Wind Technology Data Update (2019) [Slides]

The 2019 Offshore Wind Technology Data Update compiles information from peer-reviewed literature, market reports, press releases, industry news reports, manufacturer specification sheets, and offshore wind project announcements to provide a comprehensive snapshot of the state of the global wind industry in 2019. These data, including capacity projections, project characteristics, technology trends, and cost estimates, are categorized, tabulated, and plotted to provide easily recognizable and understandable summaries. A particular focus is given to the United States offshore wind project pipeline, including project announcements, state procurements, deployment timelines, infrastructure investments, grid interconnections, and other related developments. The update is intended to be used as a reference document by researchers, analysts, industry practitioners, and government officials.

17 WIND ENERGY↗

Validation of the EDGES Low-band Antenna Beam Model

The response of the antenna is a source of uncertainty in measurements with the Experiment to Detect the Global Epoch of Reionization Signature (EDGES). We aim to validate the electromagnetic beam model of the low-band (50–100 MHz) dipole antenna with comparisons between models and against data. We find that simulations of a simplified model of the antenna over an infinite perfectly conducting ground plane are, with one exception, robust to changes in the numerical electromagnetic solver code or algorithm. For simulations of the antenna with the actual finite ground plane and realistic soil properties, we find that two out of three numerical solvers agree well. Applying our analysis pipeline to a simulated drift-scan observation from an early EDGES low-band instrument that had a 10 m × 10 m ground plane, we find residual levels after fitting and removing a five-term foreground model from the simulated data binned in local sidereal time (LST) average about 250 mK with ±40 mK variation between numerical solvers. A similar analysis of the primary 30 m × 30 m sawtooth ground plane reduced the LST-averaged residuals to about 90 mK with ±10 mK between the two viable solvers. More broadly we show that larger ground planes generally perform better than smaller ground planes. Simulated data have a power that is within 4% of real observations, a limitation of net accuracy of the sky and beam models. We observe that residual spectral structures after foreground model fits match qualitatively between simulated data and observations, suggesting that the frequency dependence of the beam is reasonably represented by the models. We find that a soil conductivity of 0.02 S m{sup -1} and relative permittivity of 3.5 yield good agreement between simulated spectra and observations. This is consistent with the soil properties reported by Sutinjo et al. for the Murchison Radio-astronomy Observatory, where EDGES is located.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Validation and Calibration of Energy Models with Real Vehicle Data from Chassis Dynamometer Experiments

Accurate estimation of vehicle fuel consumption typically requires detailed modeling of complex internal powertrain dynamics, often resulting in computationally intensive simulations. However, many transportation applications-such as traffic flow modeling, optimization, and control-require simplified models that are fast, interpretable, and easy to implement, while still maintaining fidelity to physical energy behavior. This work builds upon a recently developed model reduction pipeline that derives physics-like energy models from high-fidelity Autonomie vehicle simulations. These reduced models preserve essential vehicle dynamics, enabling realistic fuel consumption estimation with minimal computational overhead. While the reduced models have demonstrated strong agreement with their Autonomie counterparts, previous validation efforts have been confined to simulation environments. This study extends the validation by comparing the reduced energy model's outputs against real-world vehicle data. Focusing on the MidSUV category, we tune the baseline Autonomie model to closely replicate the characteristics of a Toyota RAV4. We then assess the accuracy of the resulting reduced model in estimating fuel consumption under actual drive conditions. Our findings suggest that, when the reference Autonomie model is properly calibrated, the simplified model produced by the reduction pipeline can provide reliable, semi-principled fuel rate estimates suitable for large-scale transportation applications.

42 ENGINEERING↗

Dark Energy Survey Year 3 results: cosmology with moments of weak lensing mass maps – validation on simulations

ABSTRACT We present a simulated cosmology analysis using the second and third moments of the weak lensing mass (convergence) maps. The second moment, or variances, of the convergence as a function of smoothing scale contains information similar to standard shear two-point statistics. The third moment, or the skewness, contains additional non-Gaussian information. The analysis is geared towards the third year (Y3) data from the Dark Energy Survey (DES), but the methodology can be applied to other weak lensing data sets. We present the formalism for obtaining the convergence maps from the measured shear and for obtaining the second and third moments of these maps given partial sky coverage. We estimate the covariance matrix from a large suite of numerical simulations. We test our pipeline through a simulated likelihood analyses varying 5 cosmological parameters and 10 nuisance parameters and identify the scales where systematic or modelling uncertainties are not expected to affect the cosmological analysis. Our simulated likelihood analysis shows that the combination of second and third moments provides a 1.5 per cent constraint on S8 ≡ σ8(Ωm/0.3)0.5 for DES Year 3 data. This is 20 per cent better than an analysis using a simulated DES Y3 shear two-point statistics, owing to the non-Gaussian information captured by the inclusion of higher order statistics. This paper validates our methodology for constraining cosmology with DES Year 3 data, which will be presented in a subsequent paper.

79 ASTRONOMY AND ASTROPHYSICS↗

A Parameter-masked Mock Data Challenge for Beyond-two-point Galaxy Clustering Statistics

The past few years have seen the emergence of a wide array of novel techniques for analyzing high-precision data from upcoming galaxy surveys, which aim to extend the statistical analysis of galaxy clustering data beyond the linear regime and the canonical two-point (2pt) statistics. We test and benchmark some of these new techniques in a community data challenge named “Beyond-2pt,” initiated during the Aspen 2022 Summer Program “Large-Scale Structure Cosmology beyond 2-Point Statistics,” whose first round of results we present here. The challenge data set consists of high-precision mock galaxy catalogs for clustering in real space, in redshift space, and on a light cone. Participants in the challenge have developed end-to-end pipelines to analyze mock catalogs and extract unknown (“masked”) cosmological parameters of the underlying ΛCDM models with their methods. The methods represented are density-split clustering, nearest neighbor statistics, BACCO power spectrum emulator, void statistics, LEFTfield field-level inference using effective field theory (EFT), and joint power spectrum and bispectrum analyses using both EFT and simulation-based inference. In this work, we review the results of the challenge, focusing on problems solved, lessons learned, and future research needed to perfect the emerging beyond-2pt approaches. The unbiased parameter recovery demonstrated in this challenge by multiple statistics and the associated modeling and inference frameworks supports the credibility of cosmology constraints from these methods. The challenge data set is publicly available, and we welcome future submissions from methods that are not yet represented.

Krause, Elisabeth [Univ. of Arizona, Tucson, AZ (U↗

Novel Membrane-Based Corrosion Sensors For Operating In Natural Gas Pipelines

Membrane-based electrochemical sensors (MBES) provide an attractive option for real-time continuous humidity and corrosion rate measurements inside of natural gas pipelines. The MBES, can use electrochemical techniques such as linear polarization resistance and electrochemical impedance spectroscopy even when a conductive aqueous phase is intermittent or unavailable. We now present field-test performance data of our MBES operated inside a multiphase water and natural gas flow facility. The MBES measured internal corrosion and conductivity at the top, bottom, and side of the pipe spool. Tests were carried out with flow rates of 4.6, 9.1, and 18.3 m/s, temperatures of 21, 29, and 35 °C, and a pressure of 1000 psia. All sensors followed similar trends in open circuit potential, corrosion rate, and conductivity over time. Neither flow rate, temperature, nor build-up of metal-based solids on the sensor surfaces impacted the quality of the electrochemical measurements. The field-test results confirm that the membrane-based electrochemical sensors are suitable for monitoring internal corrosion of operational natural-gas pipelines.

Duffy, Timothy↗