SparseX - A tool for distributed processing of multidimensional sparse data
Developed for atmospheric science: specifically, a tool for processing sparse photon propagation data
Corzine, Riley Zhou↗
SEARCH · Search NASA
Results for “distributed processing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing
In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.
Hou, Shiyue↗
Identifying electrochemical processes by distribution of relaxation times in proton exchange membrane electrolyzers
Distribution of relaxation time (DRT) is used to interpret electrochemical impedance spectroscopy (EIS) for proton exchange membrane (PEM) water electrolyzers, with an attempt to separate overlapped relaxation processes in Nyquist plots. By varying operating conditions and catalyst loadings, four main relaxation peaks arising from EIS can be identified and successfully separated from low to high frequencies as (P1) mass transport, (P2) oxygen evolution reaction kinetics, (P3) reaction kinetics (with faster time constant than P2), and (P4) ionic transport. Here, the shape, height, and frequency of the DRT peaks change with different membrane electrode assembly (MEA) configurations. Electron microscopy reveals distinct features from the cross-sectioned MEAs which verify critical DRT results in that increasing the iridium (Ir)-anode loading from 0.2 mgIr/cm 2 to 1.5 mgIr/cm 2 reduces kinetic losses due to higher site-access; a thick and compacted anode, however, also triggers higher ohmic resistances from membrane/catalyst layer hydration and increases transport losses due to longer ionomer pathways. DRT provides higher resolution to EIS for deconvoluting processes with different relaxation times and the quantification of DRT peaks improves the accounting of total losses from each process.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Decoding Resilience by Modeling Outage and Restoration Processes in Distribution Grids: A Pittsburgh Case Study
Climate-induced extreme weather events, such as floods and heatwaves, pose significant challenges to the resilience of urban power distribution grids. This paper examines the outage and restoration dynamics of Pittsburgh's power grid during the flood event in April 2024 and three heatwaves in June, July, and August 2024. We introduce a comprehensive modeling framework that integrates outage and restoration processes, enabling the quantification of resilience metrics, including total customer-hours of power outage, maximum residual values, and restoration durations across various ZIP codes. Our analysis highlights spatial disparities in outage impacts and restoration efficiencies, with ZIP Code 15222 experiencing the highest cumulative disruptions, while ZIP Code 15217 shows lower susceptibility. By linking statistical trends with weather events, this study underscores the critical need for targeted infrastructure upgrades and advanced restoration strategies. Furthermore, the proposed framework offers valuable insights for planning and managing resilient power systems in the face of increasing climate stress.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Optimizing Transportation Networks for E-Waste Reverse Logistics: A Multi-Modal Cost Allocation and Pricing Strategy
The exponential growth of electronic waste (e-waste) poses critical challenges for sustainable reverse logistics and transportation network optimization. This study develops a dual-channel transportation framework for e-waste logistics that integrates dynamic freight pricing, cost allocation mechanisms, and game-theoretic coordination. The model captures interactions between centralized hubs and distributed processing networks, accounting for freight rate elasticity, volume allocation, and capacity constraints. Using Stackelberg game theory and cost-sharing strategies, the framework optimizes transportation efficiency and profit distribution across logistics channels. Numerical simulations show that the dual-channel structure increases centralized hub profit by 226.8% compared to baseline single-channel operations, while boosting total transported volume by 1.2% and nearly doubling freight collector profit under cost-sharing. Scenario analyses across regional infrastructures reveal that network density, policy incentives, and logistics costs shape routing efficiency and profit allocation. These findings suggest that coordinated strategies combining dynamic pricing, targeted infrastructure investment, and strategic cost allocation are needed to design efficient, resilient, and regionally adaptable e-waste transportation systems.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Computationally efficient and error aware surrogate construction for numerical solutions of subsurface flow through porous media
Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy’s partial differential equation for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the partial differential equation for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical partial differential equation solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. The Gaussian process surrogate analyzes the ensemble of numerical pressure solutions at the critical location as noisy observations of the true pressure solution, enabling robust inference using the conditional Gaussian process distribution. Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O ( n log n ) instead of the typical O ( n 3 ) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy’s equation and the numerical solution. Finally, although our method is derived for building a surrogate for the solution of Darcy’s equation with a random permeability field, the framework broadly applies to solutions of other partial differential equations with random coefficients.
54 ENVIRONMENTAL SCIENCES↗
Modeling Framework for the Assessment of a Sustainable Hydrogen Production and Supply Chain Network in California
The cost-effective and sustainable deployment of hydrogen supply and demand networks, especially in large economic regions like California, can be challenging considering the spatial-temporal availability and variability of the different actors across the network such as production processes, distribution modes, and end-users. In this presentation, we will provide an overview and demonstration of a modeling framework used to assess the environmental, economic, and human health impacts of plausible hydrogen production and supply chain networks in California. Scenarios focus on green hydrogen production pathways using water electrolysis and biomass gasification. End-use applications included in the model are transit, medium and heavy-duty trucking, port authorities, and power and aviation companies that currently consume natural gas, diesel, and aviation fuel for their day-to-day operation. Representative locations for hydrogen production and end-use are based on recent projections of the hydrogen economy in California. All mass and energy flows, as well as estimated emissions, are based on H2A process model designs and projections of technology performance, literature review, and LBNL process, economic and life cycle modeling, and not on company data for the sake of this presentation. Human health impacts are included following methodologies developed for the University of California Irvine HyDeal project. Life cycle phases associated with hydrogen production include feedstock preparation (water and biomass), energy production and consumption (renewable, grid, and combination of renewable and grid electricity), maintenance (chemical utilization in electrolysis and natural gas combustion in gasification), carbon sequestration, hydrogen storage (compression and liquefaction), and distribution (truck and pipeline). We apply the framework utilizing California specific emission factors, financial data, and human health damages and explore the impact of network characteristics on results. Example variations include: the inclusion of policy incentives or not, different representations of the electricity grid and source, electrolysis versus gasification versus combinations of both for production, liquefaction versus compression based on producer capacity cutoffs, transportation truck versus pipeline based on existing infrastructure, and ultimate end use. Comparison of these different scenarios can help inform future projects by demonstrating the trade-offs among environmental, economic, and human health impacts. This model, automated in R, is a starting platform upon which new analysis, modeling capabilities, locations, and emission factors can be rapidly tested and integrated.
Zaki, Mohammed Tamim↗
The LCLStream Ecosystem for Multi-Institutional Dataset Exploration
We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.
Rogers, David [ORNL] (ORCID:0000000251871768)↗
Considerations for a Medium-Voltage DC Electrolysis Testbed
Here we present the results of a study focused on the feasibility of using Medium Voltage DC (MVDC) power distribution from wind power generation to electrolyzers for hydrogen production. This approach, using hybrid energy generation in a MVDC microgrid, offers many advantages. These include possible improvements in efficiency, reliability and installation cost compared to a more typical state-of-the-art AC distribution configuration. It also eliminates the need for transformers, which have recently been subject to price volatility and availability concerns. This study highlights the practical feasibility of MVDC distribution networks for integrating various energy sources, offering improved efficiency and reduced system complexity compared to conventional AC-based solutions. Future work will focus on enhancing fault protection strategies, scaling the system to larger renewable installations, and conducting hardware implementation at the National Renewable Energy Laboratory's (NREL) Flatirons Campus (FC). In the sections that follow we show that a DC Collection and Distribution System (DC CDS) reduces the losses associated with the electrical conversion / distribution process relative to a state-of-the-art AC approach, improving overall efficiency by 5%. On the qualitative side, reducing the number of conversion stages is likely to improve reliability, reduce capital investment cost, and enable simpler control algorithms to be used, and reduced risk of instabilities and malfunctions.
08 HYDROGEN↗
S-QGPU: Shared quantum gate processing unit for distributed quantum computing
We propose a distributed quantum computing (DQC) architecture in which individual small-sized quantum computers are connected to a shared quantum gate processing unit (S-QGPU). The S-QGPU comprises a collection of hybrid two-qubit gate modules for remote gate operations. In contrast to conventional DQC systems, where each quantum computer is equipped with dedicated communication qubits, S-QGPU effectively pools the resources (e.g., the communication qubits) together for remote gate operations, and, thus, significantly reduces the cost of not only the local quantum computers but also the overall distributed system. Our preliminary analysis and simulation show that S-QGPU's shared resources for remote gate operations enable efficient resource utilization. When not all computing qubits (also called data qubits) in the system require simultaneous remote gate operations, S-QGPU-based DQC architecture demands fewer communication qubits, further decreasing the overall cost. Alternatively, with the same number of communication qubits, it can support a larger number of simultaneous remote gate operations more efficiently, especially when these operations occur in a burst mode.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Understanding and Enhancing Silicon Nanoparticle Distribution during Electrode Processing
Silicon-dominant anodes are of great interest because of their potential to boost the cell-level energy of state-of-the-art Li-ion batteries. While silicon materials have been extensively studied, understanding interactions at the electrode level has recieved little attention, especially the coating process of Si particles, which plays an equally important role in unlocking the full potential of silicon anodes. Herein, the electrode processing of a Si-dominated anode (52.8 wt%, 3.5–4.5 mAh cm −2 ) is being investigated to understand the relationship of processing on the morphology and properties of Si anodes at the electrode level. It has been found that almost-undetectable Si agglomerates easily form during electrode processing, which grow into largeprotrusions after lithiation and trigger potential internal shorting and self-discharge problems. A facile slurry filtration step is proposed to homogenize the particle distribution within Si-dominant electrodes which improves the electrochemical performance and storage stability of Si-based Li ion batteries.
Wu, Bingbin↗
Catalytic Upgrading of Pyrolysis Condensables from Postconsumer Polyolefins Using HZSM-5
The conversion of plastic wastes to monomeric olefins is an attractive means for achieving a plastic circular economy. In our study, a fluidized bed reactor converts post-consumer waste high-density polyethylene (HDPE) and polypropylene (PP) to mostly condensed pyrolysis waxes and some oils, preventing carbon loss to gases. The pyrolysis condensables were upgraded to light olefins (C 2 –C 5 ) at carbon yields greater than 76 wt % using the HZSM-5 zeolite catalyst at a post pyrolysis process that employed a micropyrolyzer. These results were comparable to olefin monomer yields from direct ex situ catalytic pyrolysis of the original waste plastics without condensing the vapors, highlighting the potential applicability of this approach in plastic waste recycling. Our results suggest that a centralized catalytic upgrading facility fed by pyrolysis condensables sourced from distributed thermochemical processing plants is a promising pathway to a circular economy. Such an approach enables utilization of available catalytic cracking infrastructure while focusing on setting up distributed thermochemical processing plants close to material recovery facilities. As a result, the energy-dense pyrolysis waxes are more suitable for transportation, contributing to the overall scalability and economic viability of the proposed distributed approach.
10 SYNTHETIC FUELS↗
Gaussian processes for inferring parton distributions
The extraction of parton distribution functions (PDFs) from experimental or lattice QCD data is an ill-posed inverse problem, where regularization strongly impacts both systematic uncertainties and the reliability of the results. We study a framework based on Gaussian Process Regression (GPR) to reconstruct PDFs from lattice QCD matrix elements. Within a Bayesian framework, Gaussian processes serve as flexible priors that encode uncertainties, correlations, and constraints without imposing rigid functional forms. We investigate a wide range of kernel choices, mean functions, and hyperparameter treatments. We quantify information gained from the data using the Kullback-Leibler divergence. Synthetic data tests demonstrate the consistency and robustness of the method. Our study establishes GPR as a systematic and non-parametric approach to PDF reconstruction, offering controlled uncertainty estimates and reduced model bias in lattice QCD analyses.
hadronic spectroscopy↗
Signal Processing in SBND with Calibrated and Validated Electronics and Field Responses
SBND is a liquid argon time projection chamber in Fermilab’s Short-Baseline Neutrino Program, located 110 m from the neutrino source and operating in a high-rate environment with unprecedented statistics. Charged particles from neutrino interactions ionize the argon, and the resulting electrons drift to the anode wires, inducing current signals recorded as raw waveforms. These waveforms are a convolution of deposited charge with the electronics and TPC field responses, making accurate signal processing essential for recovering the true charge distribution. Signal processing forms the starting point for SBND reconstruction, directly impacting hit finding, charge calibration, clustering, and the reconstruction of tracks and showers, and therefore playing a key role in energy reconstruction and particle identification. In this poster, we present an overview of the SBND signal processing chain, including noise removal, channel-by-channel electronics correction, signal identification, and deconvolution using measured electronics and TPC field responses. We demonstrate that the two kernel functions—electronics and field responses—achieve high precision when compared to data, ensuring that the SBND signal processing chain provides a robust and accurate foundation for event reconstruction and precision physics measurements.
Singh, Prabhjot [Louisiana State U.] (ORCID:000000↗
Consideration of Decabromodiphenyl Ether Flame Retardant in Thermal and Radiation Aging of Crosslinked Polyethylene Cable Insulation
Decabromodiphenyl ether (decaBDE) has been used as a flame-retardant additive in nuclear-grade electrical cable insulation. However, decaBDE has been identified as a persistent, bioaccumulative and toxic (PBT) substance, leading to regulatory scrutiny. On January 6, 2021, the Environmental Protection Agency (EPA) published a final rule to phase out decaBDE. The 2021 rule set a two-year compliance deadline for “processing and distribution in commerce of decaBDE for use in wire and cable insulation in nuclear power generation facilities.” In recognition of industry concerns following a sudden discontinuation of decaBDE-containing Class 1E wire and cable essential for nuclear power operations and the time needed for qualifying the individual components using the alternative insulation technology, an extended compliance deadline was set in the finalized amendments to the 2021 rule as published by the Environmental Appeals Board on November 12, 2024. The 2024 rule set the compliance deadline for processing and distributing decaBDE-containing wire and cable insulation until the end of the service life of these materials. Since decaBDE has long been relied upon as the flame retardant in one of the most common cross-linked polyethylene (XLPE) nuclear cable insulation formulations, RSCC Firewall III insulation, questions have naturally arisen regarding whether changes in cable performance might be expected for XLPE containing a decaBDE alternative, especially for safety-related cables that must perform their safety function in a design basis event such as a loss of coolant accident.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Independent Electron and Ion Energy Distribution Control for Semiconductor Processing Phase I Final Technical Report
Explore the source record for details and available documents.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Reactor assemblies and methods of performing reactions
Reactors are provided that can include a first set of fluid channels and a second set of fluid channels oriented in thermal contact with the first set of fluid channels. The reactor assemblies can also provide where the channels of either one or both of the first of the set of fluid channels are non-linear. Other implementations provide for at least one of the first set of fluid channels being in thermal contact with a plurality of other channels of the second set of fluid channels. Reactor assemblies are also provided that can include a first set of fluid channels defining at least one non-linear channel having a positive function, and a second set of fluid channels defining at least another non-linear channel having a negative function in relation to the positive function of the one non-linear channel of the first set of fluid channels. Processes for distributing energy across a reactor are provided. The processes can include transporting reactants via a first set of fluid channels to a second set of fluid channels, and thermally engaging at least one of the first set of fluid channels with at least two of the second set of fluid channels.
Zheng, Richard F.↗
Evolution of DUNE’s Production System
The DUNE experiment will start running in 2029 and record 30 PB/year of raw waveforms from Liquid Argon TPCs and photon detectors. The size of individual readouts can range from 100 MB to a typical 8 GB full readout of the detector, and even 100 TB for extended readouts from supernova candidates. These data then need to be cataloged, stored and distributed for processing worldwide. This massive amount of data and a heterogeneous computing environment necessitates a powerful and robust distributed computing infrastructure. In the process of building up that infrastructure, DUNE’s production system has recently undergone an overhaul, in which it has integrated 1) a new workflow management system (justIN) 2) a new data catalog (MetaCat) and 3) a state-of-the-art data management system (Rucio). Simulations of DUNE’s Far Detector and its prototypes ProtoDUNE Horizontal Drift (ProtoDUNE-HD) and ProtoDUNE Vertical Drift (ProtoDUNE-VD), as well as data from ProtoDUNE-HD serve as the first tests of this infrastructure.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗