Search NASA⌕ Search

SEARCH · Search NASA

Results for “DATA COMPRESSOR”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A General Framework for Progressive Data Compression and Retrieval

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. Here, this paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

NUMERICAL PREDICTIONS OF MEAN PERFORMANCE AND DYNAMIC BEHAVIOR OF A 10 MWe SCO2 COMPRESSOR WITH TEST DATA VALIDATION

High-fidelity aerodynamic analysis has been demonstrated for mean performance and unsteady dynamics in a sCO2 compressor designed by Hanwha Power Systems Americas for a 10 MWe Concentrating Solar Power plant. Simulations were performed with CRUNCH CFD® software tool that was matured to accurately model near critical real fluid effects in sCO2. Pre-test predictions for mean performance were validated with test data collected later. Performance predictions were accurate and captured sensitivity of the efficiency to inlet temperature of CO2 as well as steep drop-off at high flow rates due to condensation in the inlet throat. Detailed analysis was performed to understand the source of these performance losses at near critical conditions. Unsteady dynamic effects in the compressor at off-design conditions were also identified and quantified. In particular a system wide “condensation surge” condition was detected at high flow coefficients that results in large amplitude pulsations with accompanying mass flow fluctuations at low frequencies and has potential to cause damage in closed loop systems.

hosangadi, ashvin↗
DOE OSTI2023

Black-box statistical prediction of lossy compression ratios for scientific data

Lossy compressors are increasingly adopted in scientific research, tackling volumes of data from experiments or parallel numerical simulations and facilitating data storage and movement. In contrast with the notion of entropy in lossless compression, no theoretical or data-based quantification of lossy compressibility exists for scientific data. Users rely on trial and error to assess lossy compression performance. As a strong data-driven effort toward quantifying lossy compressibility of scientific datasets, we provide a statistical framework to predict compression ratios of lossy compressors. Our method is a two-step framework where (i) compressor-agnostic predictors are computed and (ii) statistical prediction models relying on these predictors are trained on observed compression ratios. Proposed predictors exploit spatial correlations and notions of entropy and lossyness via the quantized entropy. We study 8+ compressors on 6 scientific datasets and achieve a median percentage prediction error less than 12%, which is substantially smaller than that of other methods while achieving at least a 8.8× speedup for searching for a specific compression ratio and 7.8× speedup for determining the best compressor out of a collection.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Data compression system using base values and methods thereof

In some embodiments, a memory controller in a processor includes a base value cache, a compressor, and a metadata cache. The compressor is coupled to the base value cache and the metadata cache. The compressor compresses a data block using at least a base value and delta values. The compressor determines whether the size of the data block exceeds a data block threshold value. Based on the determination of whether the size of the compressed data block generated by the compressor exceeds the data block threshold value, the memory controller transfers only a set of the compressed delta values to memory for storage. A decompressor located in the lower level cache of the processor decompresses the compressed data block using the base value stored in the base value cache, metadata stored in the metadata cache and the delta values stored in memory.

Seyedzadehdelcheh, SeyedMohammad↗
DOE OSTI2023

Data compression system using base values and methods thereof

In some embodiments, a memory controller in a processor includes a base value cache, a compressor, and a metadata cache. The compressor is coupled to the base value cache and the metadata cache. The compressor compresses a data block using at least a base value and delta values. The compressor determines whether the size of the data block exceeds a data block threshold value. Based on the determination of whether the size of the compressed data block generated by the compressor exceeds the data block threshold value, the memory controller transfers only a set of the compressed delta values to memory for storage. A decompressor located in the lower level cache of the processor decompresses the compressed data block using the base value stored in the base value cache, metadata stored in the metadata cache and the delta values stored in memory.

Seyedzadehdelcheh, Seyed Mohammad↗
DOE OSTI2020

FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data

With ever-increasing volumes of scientific floating-point data being produced by high-performance computing applications, significantly reducing scientific floating-point data size is critical, and error-controlled lossy compressors have been developed for years. None of the existing scientific floating-point lossy data compressors, however, support effective fixed-ratio lossy compression. Yet fixed-ratio lossy compression for scientific floating-point data not only compresses to the requested ratio but also respects a user-specified error bound with higher fidelity. In this paper, we present FRaZ: a generic fixed-ratio lossy compression framework respecting user-specified error constraints. The contribution is twofold. (1) We develop an efficient iterative approach to accurately determine the appropriate error settings for different lossy compressors based on target compression ratios. (2) We perform a thorough performance and accuracy evaluation for our proposed fixed-ratio compression framework with multiple state-of-the-art error-controlled lossy compressors, using several real-world scientific floating-point datasets from different domains. Experiments show that FRaZ effectively identifies the optimum error setting in the entire error setting space of any given lossy compressor. While fixed-ratio lossy compression is slower than fixed-error compression, it provides an important new lossy compression technique for users of very large scientific floating-point datasets.

Underwood, Robert↗
DOE OSTI2025

Automatic Generation of Algorithms for High-Speed Reliable Lossy Data Compression (Final Report)

Fast reliable data compression is urgently needed for many leading-edge scientific instruments and for exascale high-performance computing applications because they produce vast amounts of data at extremely high rates. The goal of this project has been to develop a framework named LC that is able to automatically generate high-speed lossless and reliable lossy compression and decompression algorithms that can be customized for different kinds of data. The resulting LC framework is freely available on GitHub. To achieve high-speed operation, LC outputs optimized and parallelized CPU and GPU implementations of the generated algorithms. To ensure the quality of lossily compressed data, LC guarantees the user-provided error bound. To be able to customize the compression algorithm to various use cases, LC can synthesize millions of different algorithms and automatically search for the one that works best for the given data. We have already employed LC to create state-of-the-art lossless and lossy compressors for scientific data as well as leading lossless compressors for images. We hope that LC and the customized, fast, reliable, and CPU/GPU-compatible compression algorithms that it can generate will greatly benefit the many scientific applications that need not only high trustworthiness but also high performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

High-Temperature Heat Transfer Fluid Circulator for Concentrating Solar Power Systems

A high temperature Heat Transfer Fluid circulator compressor with wide turn down ratio aerodynamic performance was designed, fabricated and tested for application with high pressure super Critical CO2 fluid capable of temperatures and pressures to >750°C and 25 MPa respectively. The modular system decoupled the drive motor system from an independent compressor shaft to provide thermal isolation between the high temperature compressor shaft and the lower temperature capable motor elements. While the thermal management approach appears to be viable there are potential losses in the secondary cooling flows that need to be taken into account and fine-tuned during operation in order to achieve desired overall system efficiencies. The system was also shown to be viable and dynamically stable throughout the entire operating speed range, thereby validating the overall system configuration and structural design approach used. Finally, while it was not possible to achieve specific PTC-10 test points, the compressor performance tests provide sufficient data to indicate that the compressor has the ability to achieve the required performance in practice. The two key factors affecting compressor design and cost are the maximum expected temperature and pressure. The maximum allowable temperatures for current state-of-the-art (SOA) motor driven compressor technology is limited to of approximately 150-200°C, but inlet temperatures >750 °C may be encountered. While the optimal efficiency of compression is in the transcritical region, some concepts require inlet pressures to 225 bar.

14 SOLAR ENERGY↗
DOE OSTI2021

High-Temperature Heat Transfer Fluid Circulator for Concentrating Solar Power Systems How Mohawk and Velo3Dare Printing the Future of Energy

A high temperature Heat Transfer Fluid circulator compressor with wide turn down ratio aerodynamic performance was designed, fabricated and tested for application with high pressure super Critical CO2 fluid capable of temperatures and pressures to >750°C and 25 MPa respectively. The modular system decoupled the drive motor system from an independent compressor shaft to provide thermal isolation between the high temperature compressor shaft and the lower temperature capable motor elements. While the thermal management approach appears to be viable there are potential losses in the secondary cooling flows that need to be taken into account and fine-tuned during operation in order to achieve desired overall system efficiencies. The system was also shown to be viable and dynamically stable throughout the entire operating speed range, thereby validating the overall system configuration and structural design approach used. Finally, while it was not possible to achieve specific PTC-10 test points, the compressor performance tests provide sufficient data to indicate that the compressor has the ability to achieve the required performance in practice. The two key factors affecting compressor design and cost are the maximum expected temperature and pressure. The maximum allowable temperatures for current state-of-the-art (SOA) motor driven compressor technology is limited to of approximately 150-200°C, but inlet temperatures >750 °C may be encountered. While the optimal efficiency of compression is in the transcritical region, some concepts require inlet pressures to 225 bar.

14 SOLAR ENERGY↗
DOE OSTI2021

Radial Compressor Design and Off-Design for Trans-critical CO2 Operating Conditions

A multi-staged intercooled centrifugal compressor design is performed for an indirect-fired sCO2 cycle and the design point data are used to generate the performance maps using a 2D mean-line analysis code. Fluid and turbomachinery similarity laws are applied to find an appropriate surrogate ideal gas fluid to be used in the design and performance map generation codes. The performance maps are then scaled back for the sCO2 fluid case using the affinity laws. The developed methodology is validated using the experimental data for a compressor operating at similar inflow conditions.

Uysal, Selcuk Can↗
DOE OSTI2022

CEAZ: Accelerating Parallel I/O Via Hardware-Algorithm Co-Designed Adaptive Lossy Compression

As supercomputers continue to grow to exa-scale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the I/O performance. However, little work has been done for effectively offloading lossy compression onto FPGA-based SmartNICs to reduce the compression overhead. In this paper, we propose a hardware-algorithm co-design of efficient and adaptive lossy compressor for scientific data on FPGAs (called CEAZ) to accelerate parallel I/O. Our contribution is fourfold: (1) We propose an efficient Huffman coding approach that can adaptively update Huffman codewords online based on codewords generated offline (from a variety of representative scientific datasets). (2) We derive a theoretical analysis to support a precise control of compression ratio under an error-bounded compression mode, enabling accurate offline Huffman codewords generation. This also help us create a fixed-ratio compression mode for consistent throughput. (3) We develop an efficient compression pipeline by adopting cuSZ’s dual-quantization algorithm to our hardware use case. (4) We evaluate CEAC on five real-world datasets with both a single FPGA board and 256 nodes from Bridges2 supercomputer. Experiments show that CEAZ outperforms the second-best FPGA-based lossy compressor by 2× of throughput and 9.6× of compression ratio. It also improves MPI_File_write and MPI_Gather throughputs by up to 32.7× and 31.4×, respectively.

Zhang, Chengming↗
DOE OSTI2024

A General Framework for Error-controlled Unstructured Scientific Data Compression

Data compression plays a key role in reducing storage and I/O costs. Traditional lossy methods primarily target data on rectilinear grids and cannot leverage the spatial coherence in unstructured mesh data, leading to suboptimal compression ratios. We present a multi-component, error-bounded compression framework designed to enhance the compression of floating-point unstructured mesh data, which is common in scientific applications. Our approach involves interpolating mesh data onto a rectilinear grid and then separately compressing the grid interpolation and the interpolation residuals. This method is general, independent of mesh types and typologies, and can be seamlessly integrated with existing lossy compressors for improved performance. We evaluated our framework across twelve variables from two synthetic datasets and two real-world simulation datasets. The results indicate that the multi-component framework consistently outperforms state-of-the-art lossy compressors on unstructured data, achieving, on average, a 2.3 − 3.5× improvement in compression ratios, with error bounds ranging from 1 × 10 the −6 to 1×10−2. We further investigate impact of hyperparameters, such as grid spacing and error allocation, to deliver optimal compression ratios in diverse datasets.

Gong, Qian↗
DOE OSTI2020

Characterization of the β-Phrase Region of the SAS Compressor Bed 1 Material (LANA10)

Isotherm data have been acquired and analyzed on the SAS Compressor Bed 1 material in order to define a method to set the initial bed Q/M value when freshly filled after inventory. β-phase data from eleven protium and two deuterium isotherms have been combined to determine a method for calculating the bed Q/M value given a β-phase equilibrium pressure at temperature. This method is a two-step method; first, the plateau pressure is calculated based on the isotope used and the temperature of the bed, and second, the measured equilibrium pressure is divided by the computed plateau pressure to obtain the reduced pressure, which is then inserted into the β-phase equation to calculate the bed’s Q/M value. Error sources and effects are discussed, with the conclusion that the ‘true’ reduced isotherm has been determined in the β-phase which will allow virgin bed inventory to be determined to ~3-4% accuracy.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Toward Quantity-of-Interest Preserving Lossy Compression for Scientific Data

Today's scientific simulations and instruments are producing a large amount of data, leading to difficulties in storing, transmitting, and analyzing these data. While error-controlled lossy compressors are effective in significantly reducing data volumes and efficiently developing databases for multiple scientific applications, they mainly support error controls on raw data, which leaves a significant gap between the data and user's downstream analysis. This may cause unqualified uncertainties in the outcomes of the analysis, a.k.a quantities of interest (QoIs), which are the major concerns of users in adopting lossy compression in practice. In this paper, we propose rigorous mathematical theories to preserve four families of QoIs that are widely used in scientific analysis during lossy compression along with practical implementations. Specifically, we first develop the error control theory for univariate QoIs which are essential for computing physical properties such as kinetic energy, followed by multivariate QoIs that are more commonly used in real-world applications. The proposed method is integrated into a state-of-the-art compression framework in a modular fashion, which could easily adapt to new QoIs and new compression algorithms. Experiments on real-world datasets demonstrate that the proposed method provides faithful error control on important QoIs including kinetic energy, regional average, and isosurface without trials and errors, while offering compression ratios that are up to 4x of the compression ratios provided by state-of-the-art compressors.

Jiao, Pu↗
DOE OSTI2021

Energy efficiency of blower heater non-purge compressed air dryers

This research focuses on twin tower regenerative closed loop desiccant dryers, specifically: blower heater non-purge (BHNP) with and without cooling water pumps, compressed-air heater purge (CHP), blower heater purge (BHP) and pressure swing heaterless (PSH). The research was conducted by collecting and analysing real time current draw data on air compressors and associated dryers at eight different facilities (13 air compressors) in terms of energy, power and cost. A decision tool was developed to depict the operational characteristics (power, energy and cost) of each type of dryer if used in conjunction with the selected compressor system. Finally, this research, on an equivalent normalised basis, compared and contrasted the different types of dryers in terms of performance and cost. Here, the research concluded that of the five types of desiccant dryer types observed the most energy efficient was the BHNP (with cooling water pump), subject to the operational conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗