SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗

DOE OSTI2020

Cognition at the Point of Sensing

Over the last 15 years, compressive sensing techniques have been developed which have the potential to greatly reduce the amount of data collected by systems while preserving the amount of information obtained. A cost of this efficiency is that a computationally-intensive optimization routine must be used to put the sensed data into a form that a person can interpret. At the same time, machine learning techniques have experienced tremendous growth as well. Machines have demonstrated the ability learn how to effectively perform tasks such as detection and classification at speeds much faster than humanly possible. Our goal in this project was to study the feasibility of using compressive sensing systems "at the edge." That is, how can compressive sensing sensors be deployed such that information is created at the remote sensor rather than sending raw data to a central processing location? Studies were performed to analyze whether machine learning could be done on the compressively sensed data in its raw form. If a machine is performing the task, is it possible to do so without putting the data into a human interpretable form? We show that this is possible for some systems, in particular a compressive sensing snapshot imaging spectrometer. Machine learning tasks were demonstrated to be more effective and more robust to noise when the machine learning algorithm worked on data in its raw form. This system is shown to outperform a traditional spectrometer. Techniques for reducing the complexity of the reconstruction routine were also analyzed. Techniques for such as data regularization, deep neural networks, and matrix completion were studied and shown to have benefits over traditional reconstruction techniques. In this project we showed that compressive sensing sensors are indeed feasible at the edge. As always, sensors and algorithms must be carefully tuned to work in the constrained environment. In this project we developed tools and techniques to enable those analyses.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

A control-oriented combustion model framework for compression ignition engines operating on low-reactivity fuel

This work focuses on zero-dimensional modeling of the heat release rate in a compression ignition engine operating on gasoline-like fuels. Due to the properties of gasoline, such as high volatility and longer ignition delay than diesel, the injection strategies can vary significantly from the operation with conventional diesel fuel. Different injection strategies are commonly used to achieve varying degrees of in-cylinder stratification in order to shape the combustion event and maximize efficiency. The proposed zero-dimensional combustion model was developed to account for the different stages in combustion caused by the fuel stratification. As the ignition delay model is an integral part of the entire combustion process and significantly affects the prediction accuracy, special attention has been paid to local phenomena influencing ignition delay. A one-dimensional spray model by Musculus and Kattke was employed in conjunction with a Lagrangian tracking approach in order to estimate the local air–fuel ratio within the spray tip, as a proxy for reactivity. The local air–fuel ratio, in-cylinder temperature and pressure were used in an integral fashion to estimate the ignition delay. Heat release rates were modeled using first-order non-linear differential equations. The proposed combustion model was validated against experimental data of a heavy-duty compression ignition engine with up to three injection events at mostly 1038 r/min and 14 bar brake mean effective pressure. Further validation of the model was carried out at other engine loads and speeds. Model prediction errors in CA50 of less than 1 °CA across all conditions were found. Modeling results of other combustion metrics such as combustion duration and indicated mean effective pressure are also highly satisfactory. In addition, the model has been shown to be capable of estimating the ringing intensity for most conditions.

Pamminger, Michael↗

DOE OSTI2026

Expanding Access to Science Participation: A FAIR Framework for Petascale Data Visualization and Analytics

The massive data generated by scientists daily serve as both a major catalyst for new discoveries and innovations, as well as a significant roadblock that restricts access to the data. Here, our paper introduces a new approach to removing Big Data barriers and democratizing access to petascale data for the broader scientific community. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hiding the complexities of dealing with file systems or cloud services. We enable FAIR (Findable, Accessible, Interoperable, and Reusable) access to datasets such as NASA’s petascale climate datasets. Our paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction utilizes state-of-the art progressive compression algorithms and machine-learning insights to power scalable visualization dashboards for petascale data. The result provides users with the ability to identify extreme events or trends dynamically, expanding access to scientific data and further enabling discoveries. We validate our approach by improving the ability of climate scientists to visually explore their data via three fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution. These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Computer science↗

DOE OSTI2024

Pathway to Decarbonization Through Industrial Energy Efficiency: Micro and Macro Perspectives from Compressed Air Usage

Abstract Energy audits directly provided the industrial sector with reduced energy costs and avoided emissions. Still, they also lead to far-reaching indirect and induced local, regional, and national benefits. This paper aims to present the techno-economic-environmental analysis to achieve decarbonization through implementing industrial energy efficiency at micro and macro levels. An integrated techno-economic-environmental methodology is developed. Case studies of micro-level carbon reduction efforts through industrial energy efficiency technologies are presented. The broader macroeconomic and environmental effects of technology on society are analyzed using data from 206 energy audits of industrial compressed air systems conducted over 13 years. The impacts show that energy-efficient improvements lead to direct cost savings for manufacturers, boost economic activity across sectors, and affect carbon dioxide emissions both short-term and long-term in the region. Given their extensive benefits, energy audits significantly influence policymaking. We devised a methodology to link micro-level energy audit data with macroeconomic and environmental analyses to quantify these cascading benefits. The economic scenario analysis shows that $228 M has been saved from direct industrial energy savings from implementing all compressed air recommendations in the studied periods and the region. In addition, the investment made through manufacturers would create 2,025 jobs and $383 M annually, cascading regional economic impacts. The environmental analysis shows that the regional manufacturers have directly avoided about 2.8 M metric tons of carbon dioxide emissions.

Engineering↗

DOE OSTI2025

FedCSpc: A Cross-Silo Federated Learning System With Error-Bounded Lossy Parameter Compression

Cross-Silo federated learning is widely used for scaling deep neural network (DNN) training over data silos from different locations worldwide while guaranteeing data privacy. Communication has been identified as the main bottleneck when training large-scale models due to large-volume model parameters and gradient transmission across public networks with limited bandwidth. Most previous works focus on gradient compression, while limited work tries to compress parameters that can not be ignored and extremely affect communication performance during the training. Here, to bridge this gap, we propose FedCSpc: an efficient cross-silo federated learning system with an XAI-driven adaptive parameter compression strategy for large-scale model training. Our work substantially differs from existing gradient compression techniques due to the distinct data features of gradient and parameter. The key contributions of this paper are fourfold. (1) Our designed FedCSpc proposes to compress the parameter during the training using the state-of-the-art error-bounded lossy compressor – SZ3. (2) We develop an adaptive compression error bound adjustment algorithm to guarantee the model accuracy effectively. (3) We exploit an efficient approach to utilize the idle CPU resources of clients to compress the parameters. (4) We perform a comprehensive evaluation with a wide range of models and benchmarks on a GPU cluster with 65 GPUs. Results show that FedCSpc can achieve the same model accuracy as FedAvg while reducing the data volume of parameters and gradients in communication by up to 7.39× and 288×, respectively. With 32 clients on a 4 Gb size model, FedCSpc significantly outperforms FedAvg in wall-clock time in the emulated WAN environment (at the bandwidth of 1 Gbps or lower without loss of generality).

SZ3↗

DOE OSTI2025

Localized material compression to correct distortion in wire arc additive manufacturing

Wire Arc Additive Manufacturing (WAAM) is an advanced manufacturing technology which utilizes welding systems to generate three dimensional geometries in a layer-by-layer fashion. Distortion or warping of a print substrate and WAAM components due to thermally induced residual stresses is an ongoing challenge limiting the widespread adoption of WAAM technologies for producing components. In this manuscript, a novel approach is described to address thermal distortion in deposited components by applying lateral compressions along the length of the deposited material. To demonstrate this method, a series of single-track walls were printed and compressed at evenly spaced intervals using a modified hydraulic cutter tool. The jaws of the tool were modified to compress material rather than to shear it. A mathematical model was developed to relate the curvature of the deposited material to the volume of compression required to eliminate this distortion. Validation of this model was performed using 3D scan data to compare the change in wall curvature induced by compression to the volume of the applied compressions. Substrate deflection was also compared against a control wall, and implementation of wall compression reduced maximum deflections by 93% across a series of four depositions and subsequent compressions. Wall cross sections were also analyzed to determine the impact of compression on material hardness and grain structure. The results demonstrate that successively placed lateral compressions can effectively control and potentially eliminate bending distortion in printed parts. This methodology can be further developed to form a robust model for correction of thermally-induced distortion in WAAM components.

Additive manufacturing↗

DOE OSTI2021

4D Imaging of ZnO-Coated Nanoporous Al 2 O 3 Aerogels by Chemically Sensitive Ptychographic Tomography: Implications for Designer Catalysts

The 3D chemical structure (4D spectromicroscopy) of nanoporous Al 2 O 3 aerogels coated with ZnO by atomic layer deposition (ALD) was studied by multienergy scanning transmission X-ray microscopy. These materials are representative of a class of designer catalysts in which the nanoporous support is prepared separately from the active catalyst material, which is subsequently introduced by ALD, thereby allowing independent optimization of the morphology, chemistry, and spatial distribution of the support and catalyst. The samples studied were prepared by Ga ion and Xe plasma focused ion beam (FIB) milling as well as drop casting from water suspension. Zn L and Al K edge spectra of six samples with three different ZnO loadings were measured to investigate how loading and different sample preparation methods affect the 3D distribution of the ZnO and Al 2 O 3 . Scanning transmission X-ray microscopy (STXM) and ptychographic imaging at two energies each at the Zn L 3 and Al K edge were measured. The ptychography data were analyzed by using the SHARP reconstruction code to generate quantitative 2D chemical maps of the Al 2 O 3 and the ZnO. The STXM and ptychography maps were then measured at a sequence of tilt angles, covering up to 160° of rotation. The 3D structure of the ZnO and Al 2 O 3 was derived from the tilt series data by tomographic reconstruction using a compressed sensing algorithm. A twodimensional spatial resolution (half-period) of 6 nm, measured by Fourier ring correlation, and a 3D spatial resolution (half-period) of 9 nm, measured by Fourier shell correlation, were achieved when using the COSMIC beamline at the Advanced Light Source (ALS). The results show that for all of the ZnO loadings studied there is nonuniform coverage of the ZnO on the Al 2 O 3 aerogel framework. In addition, we found that both FIB methods create sample artifacts, although the distortion was less with Xe plasma than Ga ion FIB.

36 MATERIALS SCIENCE↗

DOE OSTI2019

bokeh streaming [SWR 19-08]

Simple examples of bokeh streaming data. The plots are intended to be generic and able to infer some of the plot data from the structure of the data sent. The receiver accepts zlib compressed pickled python dictionaries.

Simpson, Jeffery↗

DOE OSTI2021

Orange County Transportation Authority Fuel Cell Electric Bus Progress Report (Feb-Jul 2020) [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by Orange County Transportation Authority (OCTA) in Southern California. The ten FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in late-2019, and were placed in service on February 9, 2020. The data period covers February 2020 through July 2020. NREL collects data on ten 2016 model year compressed natural gas (CNG) buses as a baseline comparison at OCTA.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2021

SunLine Transit Agency Fuel Cell Electric Bus Progress Report (Data Period Focus Jan-Jun 2020) [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by SunLine Transit Agency in the Coachella Valley area of California. The five FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in mid-2019. The data period covers January 2020 through July 2020. NREL collects data on five 2019 model year compressed natural gas (CNG) buses as a baseline comparison at SunLine. These new CNG buses were phased into service beginning in April 2020.

25 ENERGY STORAGE↗

DOE OSTI2022

Orange County Transportation Authority Fuel Cell Electric Bus Progress Report - Data Period Focus: Feb. 2020 through Jul. 2021 [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by Orange County Transportation Authority (OCTA) in Southern California. The ten FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in late-2019, and were placed in service on February 9, 2020. The data period covers February 2020 through July 2021. NREL collects data on ten 2016 model year compressed natural gas (CNG) buses as a baseline comparison at OCTA.

30 DIRECT ENERGY CONVERSION↗

DOE OSTI2022

SunLine Transit Agency Fuel Cell Electric Bus Progress Report - Data Period Focus: Jan. 2020 through Dec. 2021 [Slides]

This report presents early results from a deployment of fuel cell electric buses (FCEBs) operated by SunLine Transit Agency in the Coachella Valley area of California. The five FCEBs, produced by New Flyer, feature an electric drive propulsion system powered by a Ballard fuel cell system. The project team is collaborating with the U.S. Department of Energy (DOE) and DOE's National Renewable Energy Laboratory (NREL) to evaluate the buses in revenue service. The goal of this evaluation is to compare the FCEB performance to that of conventional technology and to track progress over time toward meeting the technical targets set by DOE and the Department of Transportation (DOT). The FCEBs were delivered beginning in mid-2019. The data period covers January 2020 through December 2021. NREL collects data on five 2019 model year compressed natural gas (CNG) buses as a baseline comparison at SunLine. These new CNG buses were phased into service beginning in April 2020.

08 HYDROGEN↗

DOE OSTI2024

DME-Propane Autoignition Measurements inside a Shock Tube

Dimethyl ether (DME) is a biofuel that has the potential to replace diesel in heavy-duty engines. A blend of propane (C3H8) and DME could reduce emissions when compared to diesel at heavy-duty compression engine-relevant conditions. Testing a potential new mixture in a compression engine requires a high-fidelity chemical kinetics model that accurately predicts autoignition delay times, which is especially necessary for a compression engine. In this work, autoignition data has been gathered at an equivalence ratio of 2.0 for pressures of 60 and 80 bar. DME and propane were combusted in synthetic air while excited hydroxyl (OH*) chemiluminescence was used to gather ignition delay times. Data was compared to recent chemical kinetic mechanisms for the two different pressures. Using the mechanism with the best fit of autoignition data, a sensitivity analysis was conducted to analyze the chemical kinetics of the DME/C3H8 combustion at elevated pressures.

Mohammed, Zuhayr Pasha↗

DOE OSTI2025

Dynamic Low-Rank Training with Spectral Regularization: Achieving Robustness in Compressed Representations

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94\% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗

DOE OSTI2025

Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94 compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗

DOE OSTI2023

A MHz X-ray diffraction set-up for dynamic compression experiments in the diamond anvil cell

An experimental platform for dynamic diamond anvil cell (dDAC) research has been developed at the High Energy Density (HED) Instrument at the European X-ray Free Electron Laser (European XFEL). Advantage was taken of the high repetition rate of the European XFEL (up to 4.5 MHz) to collect pulse-resolved MHz X-ray diffraction data from samples as they are dynamically compressed at intermediate strain rates (≤10 3 s −1 ), where up to 352 diffraction images can be collected from a single pulse train. The set-up employs piezo-driven dDACs capable of compressing samples in ≥340 µs, compatible with the maximum length of the pulse train (550 µs). Results from rapid compression experiments on a wide range of sample systems with different X-ray scattering powers are presented. A maximum compression rate of 87 TPa s −1 was observed during the fast compression of Au, while a strain rate of ∼1100 s −1 was achieved during the rapid compression of N 2 at 23 TPa s −1 .

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Dynamic signal recovery in distribution grids using compressive lossy measurements

Distribution system state estimation requires reliable aggregation of the measured data. However, the large volume of the measured data imposes a significant stress on the underlying communication infrastructure. With the challenges associated with measurement availability, current distribution systems are typically unobservable. To cope with the unobservability issue, compressive sensing theory allows us to recover system state information from a small number of measurements provided the states of the distribution system exhibit sparsity. In this paper, we evaluate the robustness of an updated Kalman filtered modified compressive sensing (KF-ModCS) technique that dynamically estimates the grid states using a small fraction of measured data. In practice, measurements used for sparsity based state estimation may also be intermittent due to communication network induced losses. Further, to understand the effect of packet losses on KF-ModCS, we provide an upper bound for the expected variances of the state estimation error for a given rate of information loss. This upper bound is further improved if the support set of the sparse signal that characterizes the state dynamics does not change over time and/or the reduced model is observable. Simulations based on two practical data sets collected from actual customers in a distribution grid validate the theoretical results.

24 POWER TRANSMISSION AND DISTRIBUTION↗