Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Modeling performance of data collection systems for high-energy physics

Exponential increases in scientific experimental data are outpacing silicon technology progress, necessitating heterogeneous computing systems—particularly those utilizing machine learning (ML)—to meet future scientific computing demands. The growing importance and complexity of heterogeneous computing systems require systematic modeling to understand and predict the effective roles for ML. We present a model that addresses this need by framing the key aspects of data collection pipelines and constraints and combining them with the important vectors of technology that shape alternatives, computing metrics that allow complex alternatives to be compared. For instance, a data collection pipeline may be characterized by parameters such as sensor sampling rates and the overall relevancy of retrieved samples. Alternatives to this pipeline are enabled by development vectors including ML, parallelization, advancing CMOS, and neuromorphic computing. By calculating metrics for each alternative such as overall F1 score, power, hardware cost, and energy expended per relevant sample, our model allows alternative data collection systems to be rigorously compared. We apply this model to the Compact Muon Solenoid experiment and its planned high luminosity-large hadron collider upgrade, evaluating novel technologies for the data acquisition system (DAQ), including ML-based filtering and parallelized software. The results demonstrate that improvements to early DAQ stages significantly reduce resources required later, with a power reduction of 60% and increased relevant data retrieval per unit power (from 0.065 to 0.31 samples/kJ). However, we predict that further advances will be required in order to meet overall power and cost constraints for the DAQ.

Olin-Ammentorp, Wilkie (ORCID:0000000224729862)↗

Archetype-based Redshift Estimation for the Dark Energy Spectroscopic Instrument Survey

We present a computationally efficient galaxy archetype-based redshift estimation and spectral classification method for the Dark Energy Survey Instrument (DESI) survey. The DESI survey currently relies on a redshift fitter and spectral classifier using a linear combination of principal component analysis–derived templates, which is very efficient in processing large volumes of DESI spectra within a short time frame. However, this method occasionally yields unphysical model fits for galaxies and fails to adequately absorb calibration errors that may still be occasionally visible in the reduced spectra. Our proposed approach improves upon this existing method by refitting the spectra with carefully generated physical galaxy archetypes combined with additional terms designed to absorb data reduction defects and provide more physical models to the DESI spectra. We test our method on an extensive data set derived from the survey validation (SV) and Year 1 (Y1) data of DESI. Our findings indicate that the new method delivers marginally better redshift success for SV tiles while reducing catastrophic redshift failure by 10%–30%. At the same time, results from millions of targets from the main survey show that our model has relatively higher redshift success and purity rates (0.5%–0.8% higher) for galaxy targets while having similar success for QSOs. These improvements also demonstrate that the main DESI redshift pipeline is generally robust. Additionally, it reduces the false-positive redshift estimation by 5%–40% for sky fibers. We also discuss the generic nature of our method and how it can be extended to other large spectroscopic surveys, along with possible future improvements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A MAGELLAN-IMACS-IFU SEARCH FOR DYNAMICAL DRIVERS OF NUCLEAR ACTIVITY. I. REDUCTION PIPELINE AND GALAXY CATALOG

Using the IMACS integral-field unit (IFU) on the 6.5 m Magellan telescope, we have designed the first statistically significant investigation of the two-dimensional distribution and kinematics of ionized gas and stars in the central kiloparsec regions of a well-matched sample of Seyfert and inactive control galaxies selected from the Sloan Digital Sky Survey. The goals of the project are to use the fine spatial sampling (0.2 arcsec pixel{sup -1}) and large wavelength coverage (4000-7000 A) of the IMACS-IFU to search for dynamical triggers of nuclear activity in the central region where active galactic nucleus (AGN) activity and dynamical timescales become comparable, to identify and assess the impact of AGN-driven outflows on the host galaxy and to provide a definitive sample of local galaxy kinematics for comparison with future three-dimensional kinematic studies of high-redshift systems. In this paper, we provide the first detailed description of the procedure to reduce and calibrate data from the IMACS-IFU in 'long mode' to obtain two-dimensional maps of the distribution and kinematics of ionized gas and stars. The sample selection criteria are presented, the observing strategy is described, and resulting maps of the sample galaxies are presented along with a description of the observed properties of each galaxy and the overall observed properties of the sample.

79 ASTRONOMY AND ASTROPHYSICS↗

Validation of the DESI DR2 measurements of baryon acoustic oscillations from galaxies and quasars

The Dark Energy Spectroscopic Instrument (DESI) Data Release 2 (DR2) galaxy and quasar clustering data represents a significant expansion of data from Data Release 1 (DR1), providing improved statistical precision in baryon acoustic oscillation (BAO) constraints across multiple tracers, including bright galaxies, luminous red galaxies, emission line galaxies, and quasars. In this paper, we validate the BAO analysis of DR2. We present the results of robustness tests on the blinded DR2 data and, after unblinding, consistency checks on the unblinded DR2 data. All results are compared with those obtained from a suite of mock catalogs that replicate the selection and clustering properties of the DR2 sample. We confirm the consistency of DR2 BAO measurements with DR1 while achieving a reduction in statistical uncertainties due to the increased survey volume and completeness. The combined BAO precision, including both statistical and systematic errors, improves from ∼0.52% in DR1 to 0.30% in DR2—a factor of 1.7 gain. We assess the impact of analysis choices, including different data vectors (correlation function vs power spectrum), modeling approaches and systematics treatments, and an assumption of the Gaussian likelihood, finding that our BAO constraints are stable across these variations and assumptions with a few minor refinements to the baseline setup of the DR1 BAO analysis. We summarize a series of pre-unblinding tests that confirmed the readiness of our analysis pipeline, the final systematic errors, and the DR2 BAO analysis baseline. The successful completion of these tests led to the unblinding of the DR2 BAO measurements, ultimately leading to the DESI DR2 cosmological analysis, with their implications for the expansion history of the Universe and the nature of dark energy presented in the DESI key paper (companion paper).

79 ASTRONOMY AND ASTROPHYSICS↗

Leveraging unlabeled SEM datasets with self-supervised learning for enhanced particle segmentation

Scanning Electron Microscopes (SEMs) are widely used in experimental science laboratories, often requiring cumbersome and repetitive user analysis. Automating SEM image analysis processes is highly desirable to address this challenge. In particle sample analysis, Machine Learning (ML) has emerged as the most effective approach for particle segmentation. However, the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches. Self-Supervised Learning (SSL) offers a promising alternative by enabling knowledge extraction from raw, unlabeled data. This study presents a framework for evaluating SSL techniques in SEM image analysis, focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection. A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods. The results demonstrate that ConvNeXtV2 models, with varying parameter counts, consistently outperform other techniques in particle detection across different length scales, achieving up to a 34% reduction in relative error compared to established SSL methods. Furthermore, an ablation study explores the relationship between dataset size and SSL performance, providing actionable insights for practitioners regarding model selection and resource efficiency. This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.

Rettenberger, Luca↗

CORE-BFS: Communication-Optimized REctangular-partitioned BFS Achieving 160.845 TeraTEPS on Frontier Supercomputer

Distributed Breadth-First Search (BFS) is fundamental to many large-scale graph applications, but its performance on parallel systems is often limited by high communication overhead. This paper presents CORE-BFS, an extremely scalable GPU-based BFS implementation that introduces a unique rectangular 2D partitioning-based design for Frontier supercomputer. To further improve performance, we propose four key optimizations: (1) Rectangular 2D-partition specific data formats that use two compressed row and one compressed column status array bitmaps combined with a Double Compressed Sparse Row (DCSR) format per partition, reducing memory footprint and inter-rank traffic; (2) Adaptive frontier & communication strategy that unifies top-down and bottom-up traversal on the rectangular layout, uses lazy synchronization in top-down levels, and switches variants based on frontier size to minimize communication overhead; (3) Frontier-split degree-aware update that maps frontier vertices to thread-centric, wavefront-centric, and block-centric kernels based on their degree to improve GPU utilization and memory coalescing; (4) Row-reduction pipeline that overlaps bottom-up adjacency list processing with row-wise bitmap reduction to hide inter-rank latency. Together, these techniques increase parallelism while reducing memory and communication overhead. On the Graph500 benchmark, CORE - BFS scales up to 9,248 Frontier nodes with scale-42 graphs and reaches 160.845 TTEPS, delivering a 5.42 × speedup over our previous Frontier implementation.

Yang, Haoshen [Rutgers University]↗

First M87 Event Horizon Telescope Results. III. Data Processing and Calibration

We present the calibration and reduction of Event Horizon Telescope (EHT) 1.3 mm radio wavelength observations of the supermassive black hole candidate at the center of the radio galaxy M87 and the quasar 3C 279, taken during the 2017 April 5–11 observing campaign. These global very long baseline interferometric observations include for the first time the highly sensitive Atacama Large Millimeter/submillimeter Array (ALMA); reaching an angular resolution of 25 μas, with characteristic sensitivity limits of ~1 mJy on baselines to ALMA and ~10 mJy on other baselines. The observations present challenges for existing data processing tools, arising from the rapid atmospheric phase fluctuations, wide recording bandwidth, and highly heterogeneous array. In response, we developed three independent pipelines for phase calibration and fringe detection, each tailored to the specific needs of the EHT. The final data products include calibrated total intensity amplitude and phase information. They are validated through a series of quality assurance tests that show consistency across pipelines and set limits on baseline systematic errors of 2% in amplitude and 1° in phase. The M87 data reveal the presence of two nulls in correlated flux density at ~3.4 and ~8.3 Gλ and temporal evolution in closure quantities, indicating intrinsic variability of compact structure on a timescale of days, or several light-crossing times for a few billion solar-mass black hole. These measurements provide the first opportunity to image horizon-scale structure in M87.

79 ASTRONOMY AND ASTROPHYSICS↗

Data for Impact of Non-Irrigation on 1G and 2G Bioethanol Potential of Oilcane Feedstock: A Field to Fuel Pipeline Study

This study evaluates the bioethanol potential in response to irrigation (IR) and non-irrigation (NIR) of oilcane (OC) during a seasonal drought prior harvest. The juice was extracted through mechanical pressing of stems and fermented by Ethanol Red® yeast to produce first-generation bioethanol. Hydrothermal pretreatment followed by enzymatic hydrolysis of bagasse was performed to produce monomeric sugars from structural carbohydrates. The hydrolysates were fermented with engineered yeast for second-generation bioethanol production. The irrigated oilcane juice (276.3 ± 8.9 g/L) constitutes higher sugar concentrations than non-irrigated oilcane juice (236.5 ± 2.2 g/L). The enzymatic hydrolysis of IR-OC and NIR-OC pretreated bagasse yielded similar concentrations of 247.5 ± 2.22 and 249.7 ± 4.98 g/L fermentable sugars. Industry-relevant bioethanol titers of ≥99 g/L and ≥75 g/L were achieved from juice and hydrolysates, respectively. Therefore, the non-irrigation regime did not impact the 1G and 2G bioethanol titers. However, the overall bioethanol yield can be lower due to the reduction of stem yield (8 %) per hectare.

Biomass Analytics↗

Andromeda (M31) optical and infrared disk survey. I. Insights in wide-field near-IR surface photometry

We present wide-field near-infrared J and K{sub s} images of the Andromeda Galaxy (M31) taken with WIRCam at the Canada-France-Hawaii Telescope as part of the Andromeda Optical and Infrared Disk Survey. This data set allows simultaneous observations of resolved stars and near-infrared (NIR) surface brightness across M31's entire bulge and disk (within R = 22 kpc), permitting a direct test of the stellar composition of near-infrared light in a nearby galaxy. Here we develop NIR observation and reduction methods to recover a uniform surface brightness map across the 3° × 1° disk of M31 with 27 WIRCam fields. Two sky-target nodding strategies are tested, and we find that strictly minimizing sky sampling latency cannot improve background subtraction accuracy to better than 2% of the background level due to spatio-temporal variations in the NIR skyglow. We fully describe our WIRCam reduction pipeline and advocate using flats built from night-sky images over a single night, rather than dome flats that do not capture the WIRCam illumination field. Contamination from scattered light and thermal background in sky flats has a negligible effect on the surface brightness shape compared to the stochastic differences in background shape between sky and galaxy disk fields, which are ∼0.3% of the background level. The most dramatic calibration step is the introduction of scalar sky offsets to each image that optimizes surface brightness continuity. Sky offsets reduce the mean surface brightness difference between observation blocks from 1% to <0.1% of the background level, though the absolute background level remains statistically uncertain to 0.15% of the background level. We present our WIRCam reduction pipeline and performance analysis to give specific recommendations for the improvement of NIR wide-field imaging methods.

79 ASTRONOMY AND ASTROPHYSICS↗

The BOSS bispectrum analysis at one loop from the Effective Field Theory of Large-Scale Structure

Abstract We analyze the BOSS power spectrum monopole and quadrupole, and the bispectrum monopole and quadrupole data, using the predictions from the Effective Field Theory of Large-Scale Structure (EFTofLSS). Specifically, we use the one loop prediction for the power spectrum and the bispectrum monopole, and the tree level for the bispectrum quadrupole. After validating our pipeline against numerical simulations as well as checking for several internal consistencies, we apply it to the observational data. We find that analyzing the bispectrum monopole to higher wavenumbers thanks to the one-loop prediction, as well as the addition of the tree-level quadrupole, significantly reduces the error bars with respect to our original analysis of the power spectrum at one loop and bispectrum monopole at tree level. After fixing the spectral tilt to Planck preferred value and using a Big Bang Nucleosynthesis prior, we measureσ 8 = 0.794 ± 0.037,h = 0.692 ± 0.011, and Ω m = 0.311 ± 0.010 to about 4.7%, 1.6%, and 3.2%, at 68% CL, respectively.This represents an error bar reduction with respect to the power spectrum-only analysis of about 30%, 18%, and 13% respectively. Remarkably, the results are compatible with the ones obtained with a power-spectrum-only analysis, showing the power of the EFTofLSS in simultaneously predicting several observables. We find no tension with Planck.

Astronomy & Astrophysics↗

Uncovering Structure–Conductivity Relationships in Anion Exchange Membranes (AEMs) Using Interpretable Machine Learning

Anion exchange membranes (AEMs) play a vital role in the performance of water electrolyzers and fuel cells, yet their discovery and optimization remain challenging due to the complexity of structure–property relationships. In this study, we introduce a machine learning framework that leverages conditional graph neural networks (cGNNs) and descriptor-based models and a hybrid graph neural network (HGARE) to predict and interpret ionic conductivity. The descriptor-based pipeline employs principal component analysis (PCA), ablation, and SHAP analysis to identify factors governing anion conductivity, revealing electronic, topological, and compositional descriptors as key contributors. Beyond prediction, dimensionality reduction and clustering are performed by employing t-SNE and KMeans as well as SOM, which reveal distinct membranes clusters, some of which were enriched with high anion conductivity. Among graph-based approaches, the graph convolutional (GCN) achieved strong predictive performance, while the Hybrid Graph Autoencoder-Regressor Ensemble (HGARE) achieved the highest accuracy. Additionally, atom-level saliency maps from GCN provide spatial explanations for conductive behavior, revealing the importance of polarizable and flexible regions. This work contributes to the accelerated and data-driven design of high-performance AEMs.

Naghshnejad, Pegah [Department of Chemical Enginee↗

Tightly integrated single- and multi-crystal data collection strategy calculation and parallelized data processing in JBluIce beamline control system

The calculation of single- and multi-crystal data collection strategies and a data processing pipeline have been tightly integrated into the macromolecular crystallographic data acquisition and beamline control software JBluIce . Both tasks employ wrapper scripts around existing crystallographic software. JBluIce executes scripts through a distributed resource management system to make efficient use of all available computing resources through parallel processing. The JBluIce single-crystal data collection strategy feature uses a choice of strategy programs to help users rank sample crystals and collect data. The strategy results can be conveniently exported to a data collection run. The JBluIce multi-crystal strategy feature calculates a collection strategy to optimize coverage of reciprocal space in cases where incomplete data are available from previous samples. The JBluIce data processing runs simultaneously with data collection using a choice of data reduction wrappers for integration and scaling of newly collected data, with an option for merging with pre-existing data. Data are processed separately if collected from multiple sites on a crystal or from multiple crystals, then scaled and merged. Results from all strategy and processing calculations are displayed in relevant tabs of JBluIce .

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

SnowPac : a multiscale cubic B-spline wavelet compressor for astronomical images

ABSTRACT As more advanced and complex survey telescopes are developed, the size and scale of data being captured grows at increasing rates. Across various domains, data compression through wavelets has enabled the reduction of data size and increase in computation efficiency. In this paper, we provide qualitative and quantitative tests of a new wavelet-based image compression method compared against the current standard for astronomical images. The analysis is improved by making use of state-of-the-art object detection systems to accurately measure the impact of the compression. We find that a combination of lossy wavelet-based methods, efficient quantization, and lossless dictionary compressors can preserve up to 98 per cent of astronomical objects at a 10:1 compression ratio. This significant reduction in file size also preserves astronomical object properties better than existing methods. These methods help further reduce future workloads for image-heavy processing pipelines.

Pulido, Jesus↗

Profitability, Scalability and Optimized Siting for Waste-to-Energy Supply Chains: A Case Study on Hydrothermal Liquefaction

Waste-to-Energy (WtE) conversion technologies have the potential to simultaneously reduce waste volumes, treatment-related emissions, and carbon intensity while producing valuable energy services. However, questions remain regarding under which conditions emerging WtE technologies are feasible to deploy. In this project, we develop an optimized siting method for WtE technologies to assess cost-effective WtE processor locations, throughput scales, and profitability. We apply the model using Hydrothermal Liquefaction (HTL) as an example WtE technology. We develop calibrated capital and operating expense cost curves based on literature data to study the techno-economic characteristics of WtE supply chains. Prior to solving the siting optimization, we analyze the model parameters to describe organic waste management system behavior under four different HTL deployment configurations: Co-Processing, where HTL plants send biocrude to existing conventional refineries via assumed pipelines; Co-Location, where biorefineries are co-located with HTL plants; and two alternative Standalone Biorefining cases, where distributed HTL plants transport biocrude intermediate via either assumed pipelines or trucking to centralized biorefineries. We also define and apply several feedstock gate fee calculations, representing different revenue distribution options, to investigate how bioproduct value could impact WtE supply chain economics in terms of waste producer and processor profit or cost reduction.

09 BIOMASS FUELS↗

Automatic optical detection and classification of marine animals around MHK converters using machine vision

Optical systems provide valuable information for evaluating interactions and associations between organisms and MHK energy converters and for capturing potentially rare encounters between marine organisms and MHK device. The deluge of optical data from cabled monitoring packages makes expert review time-consuming and expensive. We propose algorithms and a processing framework to automatically extract events of interest from underwater video. The open-source software framework consists of background subtraction, filtering, feature extraction and hierarchical classification algorithms. This principle classification pipeline was validated on real-world data collected with an experimental underwater monitoring package. An event detection rate of 100% was achieved using robust principal components analysis (RPCA), Fourier feature extraction and a support vector machine (SVM) binary classifier. The detected events were then further classified into more complex classes – algae | invertebrate | vertebrate, one species | multiple species of fish, and interest rank. Greater than 80% accuracy was achieved using a combination of machine learning techniques.

16 TIDAL AND WAVE POWER↗

Geologic Modeling and Simulation Report for the Aquistore Project

In this update, a regional-scale model extends beyond the 34-square-kilometer PTRC 3-D seismic survey area. A local grid refinement (LGR) system near both the injection and observation wells was introduced for the history-matching and uncertainty analysis. Spinner log survey and pressure test data provided by partners were used and evaluated to adjust the nearwellbore local permeability in order to history-match the field pressure data. As of January 9, 2016, approximately 24,000 tonnes of CO 2 has been injected during a series of relatively short operating periods. However, the injection rate during these short operating periods increased near the end of year 2015 to a level of 300 to 500 tonnes per day, based on the quantity of CO 2 made available from the pipeline.

54 ENVIRONMENTAL SCIENCES↗

Detecting Dwarf Galaxies Around NGC 55

In order to improve our understanding of the satellite populations of low-mass galaxies, the DECam Local Volume Exploration (DELVE) – DEEP Survey is performing a search for satellites around the isolated, low-mass galaxy, NGC 55. As part of this search, we characterize our dwarf galaxy detection sensitivity by injecting artificial dwarf galaxies into our data set and measuring the fraction that we can recover. Additionally, we inject and recover artificial dwarfs to test how the blending of resolved stars affects the detection of dwarf galaxies. We inject 650 artificial dwarfs across 26 DELVE-DEEP coadded images of the NGC 55 halo and use the software tool, Source Extractor, to perform source recovery. A dwarf-search pipeline is then run on our dwarf-injected catalogs, utilizing the photutils function, find peaks, and detections are recorded as a function of dwarf size and luminosity. Our results demonstrate that we are most sensitive to large, bright dwarf galaxies, and our sensitivity begins to decrease as we approach DELVE-DEEP’s detection limit or blending causes significant reductions in recovered magnitude. We also find that our results are hindered by the limitations of star-galaxy separation applied to distant, faint sources, which is possibly a consequence of the blending of small star clusters within dwarf galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

Design Optimization of Innovative High-Level Waste Pipeline Unplugging Technologies - 13341

Florida International University (FIU) is currently working on the development and optimization of two innovative pipeline unplugging methods: the asynchronous pulsing system (APS) and the peristaltic crawler system (PCS). Experiments were conducted on the APS to determine how air in the pipeline influences the system's performance as well as determine the effectiveness of air mitigation techniques in a pipeline. The results obtained during the experimental phase of the project, including data from pipeline pressure pulse tests along with air bubble compression tests are presented. Single-cycle pulse amplification caused by a fast-acting cylinder piston pump in 21.8, 30.5, and 43.6 m pipelines were evaluated. Experiments were conducted on fully flooded pipelines as well as pipelines that contained various amounts of air to evaluate the system's performance when air is present in the pipeline. Also presented are details of the improvements implemented to the third generation crawler system (PCS). The improvements include the redesign of the rims of the unit to accommodate a camera system that provides visual feedback of the conditions inside the pipeline. Visual feedback allows the crawler to be used as a pipeline unplugging and inspection tool. Tests conducted previously demonstrated a significant reduction of the crawler speed with increasing length of tether. Current improvements include the positioning of a pneumatic valve manifold system that is located in close proximity to the crawler, rendering tether length independent of crawler speed. Additional improvements to increase the crawler's speed were also investigated and presented. Descriptions of the test beds, which were designed to emulate possible scenarios present on the Department of Energy (DOE) pipelines, are presented. Finally, conclusions and recommendations for the systems are provided. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗