Search NASA⌕ Search

SEARCH · Search NASA

Results for “compression techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗
DOE OSTI2021

Fiber/matrix debonding evaluation of SiCf/SiC composites using micropillar compression technique

Third-generation silicon carbide (SiC) composites reinforced by SiC fibers (Hi-Nicalon S [HNS] and Tyranno SA3 [SA3]) are attractive for use in next generation reactors owing to their high strength and chemical inertness at high temperatures, as well as enhanced radiation tolerance under neutron irradiation environments. To optimize composite performance, the interfacial mechanical properties of chemical vapor–infiltrated (CVI) SiC f /SiC composites are investigated in this effort by using a slant interface micropillar compression testing procedure. The micropillar test specimens, containing an inclined pyrolytic carbon (PyC) interphase, are prepared using a focused ion beam. The novel microcompression testing successfully quantifies the debond shear strength and internal friction coefficient of micropillar test samples by using the Mohr-Coulomb formulation. Furthermore, according to four types of SiCf/SiC composite microcompression test results, interfacial properties and debond mechanisms are significantly affected by the PyC layer thickness, the local bonding mechanism of the PyC interphase on the SiC fiber surface, and the surface roughness of fibers. Regardless of PyC thicknesses, SA3-reinforced CVI SiC f /SiC composites are found to have much higher debond shear strengths than HNS-reinforced SiC f /SiC CVI composites. By using this micropillar compression technique alongside analytical methods, we uncover new understandings of PyC interface properties. Additionally, the micropillar test results obtained are correlated with macroscopic mechanical properties of neutron-irradiated CVI SiCf/SiC composites.

42 ENGINEERING↗
DOE OSTI2024

Evaluation of Mechanical and Thermomechanical Water Vapor Compression Techniques for Enabling High Temperature Lift Hydration-Based Chemical Heat Pumps

Achieving high temperature lifts (>200 K) via a chemical heat pump based on salt hydration/dehydration reactions requires the transport of water vapor from low to high pressure. Alternative compression approaches require condensing of low-pressure water vapor, pumping of liquid water, and subsequent evaporation when the low-side pressure corresponds to sub-ambient water saturation temperatures. Thus, this study compares four steam compression methods for use within a chemical heat pump system based on a reversible calcium oxide hydration/dehydration reaction with a temperature lift from 350 °C heat to >600 °C. Purely mechanical and thermochemical/mechanical compression technologies are considered. A parametric study of maximum allowable temperature, the isentropic efficiency of mechanical compressors, the effectiveness of heat exchangers, and the assumed allowable heat exchanger pressure drop is conducted to determine the mechanical and thermal energy consumed per kilogram of compressed steam. The system complexity in terms of the number of main system components, maximum pressure ratio, and maximum allowable temperature is estimated. Model results show an absorption-based steam compressor has the highest exergetic efficiency for the required chemical heat pump required conditions. As a result, this system configuration was then experimentally demonstrated to illustrate the impact of system performance on component effectiveness.

Advanced reactors↗
DOE OSTI2025

Scalable Hybrid Learning Techniques for Scientific Data Compression

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data (PD), scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Here, this article presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.

ITER↗
DOE OSTI2023

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗
DOE OSTI2023

Fast Algorithms for Scientific Data Compression

Many scientific simulations and experiments generate terabytes to petabytes of data daily, necessitating data compression techniques. Unlike video and image compression, scientists require methods that accurately preserve primary data (PD) and derived quantities of interest (QoIs). In our previous work, we demonstrated the effectiveness of hybrid compression techniques that combine machine learning with traditional approaches. This paper presents innovative computational techniques aimed at expediting the compression pipeline. Our experiments, conducted on two distinct platforms with a large-scale XGC-based fusion simulation, demonstrate that the overhead incurred by these new approaches is less than one percent of the computational resources needed for the simulation.

Banerjee, Tania↗
DOE OSTI2024

High‐speed 4‐dimensional scanning transmission electron microscopy using compressive sensing techniques

Abstract Here we show that compressive sensing allows 4‐dimensional (4‐D) STEM data to be obtained and accurately reconstructed with both high‐speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4‐D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal‐to‐noise ratio in the recovered phase using 0.3% of the total data. Lay abstract : Four‐dimensional scanning transmission electron microscopy (4‐D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2‐dimensional signal is acquired at each 2‐D probe location, equating to a 4‐D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4‐D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100‐1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4‐D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10‐100 times. The acquired data is then processed to fill‐in or inpaint the missing data, taking advantage of the inherently low‐complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4‐D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4‐D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100‐300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

Robinson, Alex W.↗
DOE OSTI2025

A Survey on Error-Bounded Lossy Compression for Scientific Datasets

Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.

Error-Bounded Lossy Compression↗
DOE OSTI2022

Data Compression in the NEXT-100 Data Acquisition System

NEXT collaboration detectors are based on energy measured by an array of photomultipliers (PMT) and topological event filtering based on an array of silicon photomultipliers (SiPMs). The readout of the PMT sensors for low-frequency noise effects and detector safety issues requires a grounded cathode connection that makes the readout AC-couple with variations in the signal baseline. Strict detector requirements of energy resolution better than 1% FWHM require a precise baseline reconstruction that is performed offline for data analysis and detector performance characterization. Baseline variations make it inefficient to apply traditional lossy data compression techniques, such as zero-suppression, that help to minimize data throughput and, therefore, the dead time of the system. However, for the readout of the SiPM sensors with less demanding requirements in terms of accuracy, a traditional zero-suppression is currently applied with a configuration that allows for a compression ratio of around 71%. The third stage in the NEXT detectors program, the NEXT-100 detector, is a 100 kg detector that instruments approximately five times more PMT sensors and twice the number of SiPM sensors than its predecessor, the NEXT-White detector, putting more pressure in the DAQ throughput, expected to be over 900 MB/s with the current configuration, which will worsen the dead time of the acquisition data system. This paper describes the data compression techniques applied to the sensor data in the NEXT-100 detector, which reduces data throughput and minimizes dead time while maintaining the event rate to the level of its predecessor, around 50 Hz.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

A General Framework for Progressive Data Compression and Retrieval

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. Here, this paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Terawatt-class femtosecond long-wave infrared laser

We utilized a nonlinear post-compression technique to generate 675-fs, 9.2-μm CO 2 laser pulses with a peak power of 1.6 TW. This achievement represents the highest peak power ever attained in the femtosecond pulse format within the long-wave infrared (LWIR) spectral range. The successful implementation of this post-compression technique opens avenues for the development of few-cycle, multi-terawatt 9–10 μm lasers, crucial for applications currently relying on near-infrared solid-state lasers, and which stand to benefit from the scaling of laser wavelengths into the long-wave infrared region.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

TopoSZ: Preserving Topology in Error-Bounded Lossy Compression

Existing error-bounded lossy compression techniques control the pointwise error during compression to guarantee the integrity of the decompressed data. However, they typically do not explicitly preserve the topological features in data. When performing post hoc analysis with decompressed data using topological methods, preserving topology in the compression process to obtain topologically consistent and correct scientific insights is desirable. In this paper, we introduce TopoSZ, an error-bounded lossy compression method that preserves the topological features in 2D and 3D scalar fields. Specifically, we aim to preserve the types and locations of local extrema as well as the level set relations among critical points captured by contour trees in the decompressed data. The main idea is to derive topological constraints from contour-tree-induced segmentation from the data domain, and incorporate such constraints with a customized error-controlled quantization strategy from the SZ compressor (version 1.4). In conclusion, our method allows users to control the pointwise error and the loss of topological features during the compression process with a global error bound and a persistence threshold.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Field Demonstration and Adoption Impacts of an Affordable Thermoelectric Heat Pump

This study presents the design, field validation, and utility impacts of an air source heat pump that uses thermoelectric (TE) technology to provide supplemental heating at outside air temperatures below 17°F. Conventional heat pumps experience reduced heating capacity and COP at low outdoor temperatures due to the temperature difference between the source and demand sides of the vapor compression cycle. Consequently, during colder months, supplemental heating such as electric resistance or natural gas furnace is often needed to provide additional warmth, which increases the overall equipment and energy costs. To address this challenge, a TE subcooler (an array of solid-state, thermoelectric modules that provides heat pumping using non-vapor compression technique) is fabricated and integrated with the vapor compression cycle of a conventional air-source heat pump. In colder ambient conditions, TE subcooler exchanges heat with the refrigerant to provide additional subcooling, thereby enhancing the capacity of the heat pump. The operation of thermoelectric-integrated heat pump was demonstrated in an occupied single-family home in Nashville, TN. The TE subcooler increased the system’s heating capacity at 15°F outdoor temperature by 25–30% and reduced electricity demand by 10%. It also met the heating load in the house without supplemental electric resistance down to 5°F outdoors, delivering the additional heat at nearly 80% greater efficiency compared to electric resistance heating. With a modest first cost increment of 10–15% over conventional single-speed heat pumps, this approach offers a promising, energy-efficient solution to accelerate residential heat pump adoption in the U.S.

KRISHNAMOORTHY, Sreenidhi [Electric Power Research↗
DOE OSTI2024

Diamond precipitation dynamics from hydrocarbons at icy planet interior conditions

The pressure and temperature conditions at which precipitation of diamond occurs from hydrocarbon mixtures is important for modelling the interior dynamics of icy planets. However, there is substantial disagreement from laboratory experiments, with those using dynamic compression techniques finding much more extreme conditions are required than in static compression. Here we report the time-resolved observation of diamond formation from statically compressed polystyrene, (C 8 H 8 ) n , heated using the 4.5 MHz X-ray pulse trains at the European X-ray Free Electron Laser facility. Diamond formation is observed above 2,500 K from 19 GPa to 27 GPa, conditions representative of Uranus’s and Neptune’s shallow interiors, on 30 μs to 40 μs timescales. This is much slower than may be observed during the ~10 ns duration of typical dynamic compression experiments, revealing reaction kinetics to be the reason for the discrepancy. In conclusion, reduced pressure and temperature conditions for diamond formation has implications for icy planetary interiors, where diamond subduction leads to heating and could drive convection in the conductive ice layer that has a role in their magnetic fields.

79 ASTRONOMY AND ASTROPHYSICS↗