Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The DESI DR1 Peculiar Velocity Survey: Fundamental Plane Catalogue

Measurements of peculiar velocities in the local Universe are a powerful tool to study the nature of dark energy at low ($z < 0.1$) redshifts. Here we present the largest single set of $z<0.1$ peculiar velocity measurements to date, obtained using the Fundamental Plane (FP) of galaxies in the first data release (DR1) of the Dark Energy Spectroscopic Instrument (DESI). We describe the photometric and spectroscopic selection criteria used to define the sample, as well as extensive quality control checks on the photometry and velocity dispersion measurements. Additionally, we perform detailed systematics checks for the many analysis parameters in our pipeline. Our DESI DR1 catalogue contains FP-based distances and peculiar velocities for $98,292$ unique early-type galaxies, increasing the total number of $z < 0.1$ FP distances ever measured by a factor of $\sim2$. We achieve a precision of $26\%$ random error in our distance measurements which is comparable to previous surveys. A series of companion DESI papers use the distances and peculiar velocities presented in this paper to measure cosmological parameters.

Ross, C. E. [Queensland U.]↗
DOE OSTI2022

Primordial non-Gaussianity from the completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey II: measurements in Fourier space with optimal weights

ABSTRACT We present measurements of the local primordial non-Gaussianity parameter $f_{\mathrm{NL}}^\mathrm{local} $from the clustering of 343 708 quasars with redshifts 0.8 < z < 2.2 distributed over 4808 deg2 from the final data release (DR16) of the extended Baryon Acoustic Oscillation Spectroscopic Survey (eBOSS), the largest volume spectroscopic survey up to date. Our analysis is performed in Fourier space, using the power spectrum monopole at very large scales to constrain the scale-dependent halo bias. We demonstrate the robustness of our analysis pipeline with EZ-mock catalogues that simulate the eBOSS DR16 target selection. We carefully assess the impact of systematics on our measurement, exploiting a novel neural network scheme for cleaning the DR16 sample as well as test multiple contamination removal methods. We estimate the bias due to the overcorrection of the systematic removal to be sub-dominant compared to the statistical uncertainty. We find fNL = −12 ± 21 (68 per cent confidence) for the main clustering sample including quasars with redshifts between 0.8 and 2.2, after applying redshift weighting techniques, designed for non-Gaussianity measurement from large scales structure, to optimize our analysis, which improve our results by 37 per cent.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

BLADE: An Automated Framework for Classifying Light Curves from the Center for Near-Earth Object Studies Fireball Database

Fireballs (bolides) are high-energy luminous phenomena produced when meteoroids and small asteroids enter Earth’s atmosphere at hypersonic speeds, often resulting in fragmentation or complete disintegration accompanied by significant energy release. The resulting bolide light curves capture temporal brightness variations as these objects traverse increasingly dense atmospheric layers, providing essential information on meteoroid entry dynamics, fragmentation behavior, and atmospheric energy deposition processes. The Center for Near-Earth Object Studies’ (CNEOS) continuously expanding fireball database offers a globally comprehensive archive of bolide events, including light curves and associated metadata. Events associated with infrasound detections allow direct correlations between acoustic signatures and light curve features, therefore enabling detailed analyses of fragmentation dynamics and energy deposition. Here, we introduce Bolide Light-curve Analysis and Discrimination Explorer (BLADE), a robust and high-fidelity framework specifically designed to analyze bolide light curves for objects detected from space. BLADE incorporates a processing pipeline integrating Savitzky–Golay filtering, prominence-based peak detection, and gradient analysis, enabling systematic identification and classification of fragmentation events and their associated energy release characteristics. Preliminary results demonstrate that BLADE reliably distinguishes distinct bolide behaviors, providing an objective, scalable methodology for characterization and analysis of large bolide light curve data sets. This foundational work establishes a novel pathway for advanced bolide research, with promising applications in planetary defense and global atmospheric monitoring. Future research should adopt an integrative approach combining CNEOS optical data with complementary infrasound measurements, further clarifying relationships between bolide energy deposition and acoustic signatures, thus refining our understanding of meteoroid and asteroid atmospheric entry processes.

Asteroids↗
DOE OSTI2025

SITCOMTN-162: Testing the implementation of Metadetection and Cell-Based Coadds on Abell 360 LSSTComCam data

The purpose of this technote is to test the technical quality of LSSTComCam commissioning data, specifically the Rubin_SV_38_7 field, by utilizing cell-based coadds and Metadetection by measuring the tangential and cross weak lensing shear profiles of the massive cluster Abell 360 (called A360 throughout the technote). The process entails generating the cell-based coadds for Metadetection to run on, identifying and removing cluster member galaxies, applying quality cuts, calibrating the shear measurements, and validation. Cell-based coadds and Metadetection are both currently in the process of being implemented within the LSST Science Pipelines at the time of this technote. There is substantial technical value in attempting a difficult measurement prior to full implementation. Measuring the tangential shear around A360 will showcase the current abilities of these algorithms, as well as highlight where work is still needed. As seen from the resulting shear profile of A360, the cell-based coadds and Metadetection are able to work in tandem to produce a shear catalog and resulting reduced shear profile. This technote is one part of a series studying A360 in order to both stress test the commissioning camera and demonstrate the technical capabilities of the Vera Rubin Observatory. We study the quality of the PSF modeling and impact it can have on cluster WL in [Combet et al., 2025], implementation of cell-based coadds and subsequent use for Metadetect [Sheldon et al., 2023] in this technote, photometric calibration in (in prep), source selection and photometric redshifts in [Adari et al., 2025], use of Anacal [Li et al., 2024] to produce a cluster shear profile in [Li et al., 2025], and background subtraction in this field and Fornax in [Zhou et al., 2025].

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Advancing the State of Technology thru DOE funded algae cultivation R&D at the Arizona Center for Algae Technology and Innovation

The Department of Energy's Bioenergy Technologies Office (BETO) has a long and successful history of funding an algae R&D portfolio driving progress towards affordable and renewable energy and co-products from algae biomass. A core capability established by BETO was the support of testbed facilities including those established under the Algae Testbed Public Private Partnership (ATP3) led by Arizona State University (ASU) and its Arizona Center for Algae Technology and Innovation (AzCATI). ATP3 established the experimental and operational framework to carry out long term cultivation trials to assess the State of Technology (SOT) for algal biomass productivity and conducted such trails thru the ending of the six year program in Spring 2018, and demonstrated steady progress increasing annual average biomass yields from 8.5 to 11.7 g/m2-day - a 38% increase from 2015 to 2018. Since the summer of 2018, the SOT framework established under ATP3 continues under another BETO funded lab-led consortium called DISCOVR "Development of Integrated Screening, Cultivar Optimization, and Verification Research." The DISCOVR team constitutes a powerful partnership among four of the DOE’s seventeen national laboratories: Los Alamos National Laboratory, National Renewable Energy Laboratory, Pacific Northwest National Laboratory (DISCOVR lead institution), and Sandia National Laboratories. With ASU’s AzCATI, they have joined forces to investigate algae strains that could lead to sustainable and clean algal biofuel and continue to generate open and transparent cultivation data sets for use by the algae biotechnology R&D community. In this talk, we will revisit the SOT results from ATP3, the SOT framework transition to DISCOVR with recent results from first full year of SOT cultivation under DISCOVR. With our recent advancements utilizing better cultivars verified thru DISCOVR’s pipeline, better crop protection and improved operational conditions, the combined effect was a further improvement in annual average productivity and reliability, leading to a year over year improvement in annual productivity from the 2018 baseline of 11.7 g/m2-day (generated across a total of 219 cultivation days) to 15.9 g/m2-day (326 cultivation days) – a 36% improvement for 2019! We will discuss these results in detail and highlight where AzCATI and the DISCOVR SOT is headed in 2020 and beyond.

09 BIOMASS FUELS↗
DOE OSTI2025

Comprehensive Neural Posterior Estimation for Galaxy-Galaxy Strong Lensing

We present a deep learning model based on neural posterior estimation (NPE) for comprehensive extraction of astrophysical parameters from galaxy-scale strong gravitational lenses. The unprecedentedly large amount of galaxy-scale strong lenses expected in future cosmological surveys (${\cal O}(10^5)$) promises to enable valuable statistical constraints in various studies ranging from galaxy formation to the nature of dark matter, but it also poses a significant challenge for traditional modelling pipelines. To this end, our automated model includes several new, state-of-the-art features and approaches leveraging the framework of simulation-based inference (SBI). We infer a total of 20 parameters describing the mass and light profiles of both lens and source galaxies, using simulated raw multi-band data modelled under noise and observing conditions expected by the Legacy Survey of Space and Time (LSST), with its summary statistics generated by a residual network. We examine the efficacy of multi-band data in extracting nearly 20 model parameters simultaneous from strong lensing images including lens light. Finally, We perform a comprehensive set of diagnostics for SBI models, evaluating the model's prediction accuracy, stability, and uncertainty quantification.

Zhao, Roy J. [Chicago U., KICP]↗
DOE OSTI2024

Predicting transcription factor activity using prior biological information

Dysregulation of normal transcription factor activity is a common driver of disease. Therefore, the detection of aberrant transcription factor activity is important to understand disease pathogenesis. We have developed Priori, a method to predict transcription factor activity from RNA sequencing data. Priori has two key advantages over existing methods. First, Priori utilizes literature-supported regulatory information to identify transcription factor-target gene relationships. It then applies linear models to determine the impact of transcription factor regulation on the expression of its target genes. Second, results from a third-party benchmarking pipeline reveals that Priori detects aberrant activity from 124 single-gene perturbation experiments with higher sensitivity and specificity than 11 other methods. We applied Priori and other top-performing methods to predict transcription factor activity from two large primary patient datasets. Our work demonstrates that Priori uniquely discovered significant determinants of survival in breast cancer and identified mediators of drug response in leukemia.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Critical Assessment of MetaProteome Investigation (CAMPI): a multi-laboratory comparison of established workflows

Metaproteomics has matured into a powerful tool to assess functional interactions in microbial communities. While many metaproteomic workflows are available, the impact of method choice on results remains unclear. Here, we carry out a community-driven, multi-laboratory comparison in metaproteomics: the critical assessment of metaproteome investigation study (CAMPI). Based on well-established workflows, we evaluate the effect of sample preparation, mass spectrometry, and bioinformatic analysis using two samples: a simplified, laboratory-assembled human intestinal model and a human fecal sample. We observe that variability at the peptide level is predominantly due to sample processing workflows, with a smaller contribution of bioinformatic pipelines. These peptide-level differences largely disappear at the protein group level. While differences are observed for predicted community composition, similar functional profiles are obtained across workflows. CAMPI demonstrates the robustness of present-day metaproteomics research, serves as a template for multi-laboratory studies in metaproteomics, and provides publicly available data sets for benchmarking future developments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

A Step-by-Step Protocol from METASPACE to Biological Interpretation

Mass spectrometry imaging (MSI) represents an exceptional tool for exploring complex biological systems spatially at the molecular level. However, due to its multidimensional nature and large-scale data output, it presents considerable challenges when it comes to extracting meaningful biological insights. Recent advancements, such as the METASPACE platform, have enabled researchers to efficiently process, annotate, and interpret MSI datasets by leveraging machine learning and cloud-based infrastructure. In this tutorial, we present a detailed and user-friendly R-pipeline designed to help METASPACE users navigate untargeted metabolomic annotations and transform them into practical insights about their biological systems. By combining METASPACE annotations with rapid R-based screening, this workflow not only streamlined the analytical process but also enhanced the understanding of spatial molecular distribution, especially for complex systems. Here, this easy-to-follow approach has the potential for applications in diagnostics, drug discovery, environmental and ecological processes, and more. We envision this pipeline to be particularly useful for newcomers to the field of MSI and

Moreno Pedraza, Abigail↗
DOE OSTI2025

The construction of large-scale structure catalogs for the Dark Energy Spectroscopic Instrument

We present the technical details on how large-scale structure (LSS) catalogs are constructed from redshifts measured from spectra observed by the Dark Energy Spectroscopic Instrument (DESI). The LSS catalogs provide the information needed to determine the relative number density of DESI tracers as a function of redshift and celestial coordinates and, e.g., determine clustering statistics. We produce catalogs that are weighted subsamples of the observed data, each matched to a weighted `random' catalog that forms an unclustered sampling of the probability density that DESI could have observed those data at each location. Precise knowledge of the DESI observing history and associated hardware performance allows for a determination of the DESI footprint and the number of times DESI has covered it at sub-arcsecond level precision. This enables the completeness of any DESI sample to be modeled at this same resolution. The pipeline developed to create LSS catalogs has been designed to easily allow robustness tests and enable future improvements. We describe how it allows ongoing work improving the match between galaxy and random catalogs, such as including further information when assigning redshifts to randoms, accounting for fluctuations in target density, accounting for variation in the redshift success rate, and accommodating blinding schemes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

XRIPL

XRIPL is a library of tools for processing x-ray radiographs and extracting contours of interest within the image using computer vision techniques. Features include spatial calibration, denoising, background and attenuation correction through pseudo-flatfielding, watershed segmentation, contour processing, and visualization. The details of the XRIPL analysis pipeline are published in the Proceedings of the 23rd Topical Conference on High-Temperature Plasma Diagnostics Proceedings (LA-UR-20-30133) [1]. [1] P. M. Kozlowski, Y. Kim, B. M. Haines, H. F. Robey, T. J. Murphy, H. M. Johns, and T. S. Perry. Use of Computer Vision for analysis of image data sets from high temperature plasma experiments. Review of Scientific Instruments 92, 033532 (2021) https://doi.org/10.1063/5.0040285

Kozlowski, Pawel↗
DOE OSTI2025

Firm Synthesizer and Supply-chain Simulator (SynthFirm) v2.0

SynthFirm is a national-scale agent-based freight demand model which generates a complete synthetic population of firms in the U.S. and the business-to-business commodity flows between them. Using publicly available data sources as inputs, SynthFirm simulates detailed firm and fleet characteristics, commodity production and consumption, formation of supply chains, and selection of shipping modes, all of which are essential drivers of commodity flow at a disaggregate level. The SynthFirm 2.0 version includes national commercial vehicle fleet generation, international trade simulation and automized model validation pipeline, which allows seemless deployment across the nation and build a comprehensive freight inventories at national scale or for selected region.

Yang, Hung-Chia [Lawrence Berkeley National Labora↗
DOE OSTI2022

PARETO 0.3.0 Release

PARETO 0.3.0 Release. Highlights: - Fixing Warning in get_data - Clean up of old main function which has been replaced by the run files - Adding plot_scatter() function - Support node capacity - Add Gurobi troubleshooting files to gitignore by @melody-shellman - Set slacks to zero - Water treatment case study - Fix defaultvalue treatement - Fix bug for incorrect number of input arguments in water quality - Pipeline configs documentation - Enable rendering Unicode characters in ReadTheDocs PDF builds - Adding support for units - Add discrete water quality to the model - Updating documentation for strategic model, and corresponding fixes to strategic model - Rework getting started docs build section and more - Enable Codecov reports in CI builds - Correct error in CompletionsPadSupplyBalanceRule

PARETO,PSE,Process Systems Engineering,Produced wa↗
DOE OSTI2023

Detecting and Characterizing Mg ii Absorption in DESI Survey Validation Quasar Spectra

Abstract We present findings of the detection of Magnesium II (Mg ii , λ = 2796, 2803 Å) absorbers from the early data release of the Dark Energy Spectroscopic Instrument (DESI). DESI is projected to obtain spectroscopy of approximately 3 million quasars (QSOs), of which over 99% are anticipated to be at redshifts greater than z > 0.3, such that DESI would be able to observe an associated or intervening Mg ii absorber illuminated by the background QSO. We have developed an autonomous supplementary spectral pipeline that detects these systems through an initial line-fitting process and then confirms the line properties using a Markov Chain Monte Carlo sampler. Based upon a visual inspection of the resulting systems, we estimate that this sample has a purity greater than 99%. We have also investigated the completeness of our sample in regard to both the signal-to-noise properties of the input spectra and the rest-frame equivalent width ( W 0 ) of the absorber systems. From a parent catalog containing 83,207 quasars, we detect a total of 23,921 Mg ii absorption systems following a series of quality cuts. Extrapolating from this occurrence rate of 28.8% implies a catalog at the completion of the five-year DESI survey that will contain over eight hundred thousand Mg ii absorbers. The cataloging of these systems will enable significant further research because they carry information regarding circumgalactic medium environments, the distribution of intervening galaxies, and the growth of metallicity across the redshift range 0.3 ≤ z < 2.5.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Learning dynamical systems from data: An introduction to physics-guided deep learning

Modeling complex physical dynamics is a fundamental task in science and engineering. Traditional physics-based models are first-principled, explainable, and sample-efficient. However, they often rely on strong modeling assumptions and expensive numerical integration, requiring significant computational resources and domain expertise. While deep learning (DL) provides efficient alternatives for modeling complex dynamics, they require a large amount of labeled training data. Furthermore, its predictions may disobey the governing physical laws and are difficult to interpret. Physics-guided DL aims to integrate first-principled physical knowledge into data-driven methods. It has the best of both worlds and is well equipped to better solve scientific problems. Recently, this field has gained great progress and has drawn considerable interest across discipline Here, we introduce the framework of physics-guided DL with a special emphasis on learning dynamical systems. We describe the learning pipeline and categorize state-of-the-art methods under this framework. We also offer our perspectives on the open challenges and emerging opportunities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

CostMAP: an open-source software package for developing cost surfaces using a multi-scale search kernel

Cost surfaces are a crucial aspect of route optimization and least cost path (LCP) calculations and are used in awide range of disciplines including computer science, landscape ecology, and energy-infrastructure modeling. Linear features introduce akey weakness to traditional routing calculations along cost surfaces because they cannot identify whether moving from acell to its adjacent neighbors constitutes crossing alinear barrier (increased cost) or following acorridor (reduced cost). Following and avoiding linear features can drastically change predicted routes. We introduce an approach to address this adjacency issue using asearch kernel that determines these critical barriers and corridors. We have built this method into anew Java-based open-source software package– CostMAP (cost surface multi-layer aggregation program)– which calculates cost surfaces and cost networks using the search kernel. CostMAP allows users to input multiple GIS data layers and to set weights and rules for developing aweighted-cost network. We compare CostMAP performance with traditional cost surface approaches and show significant performance gains– both following corridors and avoiding barriers– by modeling the movement of alarge terrestrial animal– the Baird’s Tapir (Tapirus bairdii)– in amovement ecology framework and by modeling pipeline routing for carbon capture and storage (CCS).

58 GEOSCIENCES↗
DOE OSTI2024

Comparative Study of Large Language Model Architectures on Frontier

Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.

Yin, Junqi↗
DOE OSTI2026

lumicap v0.1

Automated HDR luminance imaging system designed for daylighting research and building science. It controls a fisheye-lens camera to capture time-lapse bracket sequences, merges them into calibrated HDR images, and runs a full post-processing pipeline — all unattended. Features: - Scheduled LDR bracket capture via gphoto2 - HDR merging with vignetting, ND filter, and fisheye projection corrections - Illuminance and luminance meter integration (Konica Minolta T-10A, LS-100/150) - Daylight glare probability (DGP) and solar position computation - Automated false-color rendering, JPEG thumbnails, and daily time-lapse video - CSV data logging per timestep Uses: - Long-term monitoring of daylight conditions in buildings - Glare analysis for occupant comfort research - Solar irradiance and sky luminance studies Advantages: - End-to-end automation — capture, calibration, analysis, and archiving run without manual intervention - Built on the proven Radiance toolchain, ensuring photometrically accurate HDR output - Hardware-agnostic meter support via serial auto-detection - Lightweight — no GUI overhead, deployable on a headless Raspberry Pi or similar embedded system

Wang, Taoning [Lawrence Berkeley National Laborato↗