Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Error-controlled, progressive, and adaptable retrieval of scientific data with multilevel decomposition

Extreme-scale simulations and high-resolution instruments have been generating an increasing amount of data, which poses significant challenges to not only data storage during the run, but also post-processing where data will be repeatedly retrieved and analyzed for a long period of time. The challenges in satisfying a wide range of post-hoc analysis needs while minimizing the I/O overhead caused by inappropriate and/or excessive data retrieval should never be left unmanaged. In this paper, we propose a data refactoring, compressing, and retrieval framework capable of 1) fine-grained data refactoring with regard to precision; 2) incrementally retrieving and recomposing the data in terms of various error bounds; and 3) adaptively retrieving data in multi-precision and multi-resolution with respect to different analysis. With the progressive data re-composition and the adaptable retrieval algorithms, our framework significantly reduces the amount of data retrieved when multiple incremental precision are requested and/or the downstream analysis time when coarse resolution is used. Experiments show that the amount of data retrieved under the same progressively requested error bound using our framework is 64% less than that using state-of-the-art single-error-bounded approaches. Parallel experiments with up to 1, 024 cores and ~ 600 GB data in total show that our approach yields 1.36× and 2.52× performance over existing approaches in writing to and reading from persistent storage systems, respectively.

Liang, Xin↗
NASA NTRS1986

Adaptive Quantizer for Burst Synthetic-Aperture Radar

Adaptive quantizer for burst-mode synthetic-aperture radar reduces data rate of return signal. Device, called block floating-point quantizer (BFPQ) basically analog-to-digital converter that covers wide dynamic range and discards appropriate lower order bits. BFPQ is, in effect, digital approximator with automatic gain control. Moves floatingpoint marker binary representation of signal data in accordance with perceived dynamic range. Available step sizes thus limited to multiples of underlying smallest quantization step (represented by lowest order bit). Retains only first K most significant bits of signal; (L,K) BFPQ is one that does K-bit quantization of signal originally quantized to L bits. Quantization error simply difference between actual signal level and its binary approximation. Other potential applications for BFPQ include speech compression and picture data compression.

Joo, T. H.↗
NASA NTRS1991

Emerging standards for still image compression: A software implementation and simulation study

The software implementation is described of an emerging standard for the lossy compression of continuous tone still images. This software program can be used to compress planetary images and other 2-D instrument data. It provides a high compression image coding capability that preserves image fidelity at compression rates competitive or superior to most known techniques. This software implementation confirms the usefulness of such data compression and allows its performance to be compared with other schemes used in deep space missions and for data based storage.

Pollara, F.↗
DOE OSTI2023

CHIPFoam Modification to Allow for Modelling SWIFT Foams

SWIFT (Silicon-water in Familiar Template) foams are silicone foams that have a continuous porosity without having any obvious spherical or ellipsoidal pore shapes. Like other foamed rubber, SWIFT foams can experience large reversible deformations. Because of their microstructure, SWIFT foams have a stiffering behavior in compression that is very abrupt compared to most other foamed rubbers. Work by Benedikt reported by Miller showed that fitting compressive test data with the CHIPFoam model in its current form is impractical and inaccurate. This report describes an extension made to CHIPFoam to allow the modeling of SWIFT foams. While the extension allows the modeling of SWIFT foams, it also extends the range of porosity of foams that can be modelled with CHIPFoam to include very high porosity (>99%) foamed rubber. The previous model was unstable for porosities greater than 70%. The changes made to CHIPFoam include the introduction of a critical porosity at which the compressible Danielsson strain energy function that couples volumetric and deviatoric mechanical behavior begins to increase with compression. This new feature also required the introduction of a third term in the multiplicative decomposition of the relative volume, J . The organization of this report is described here. The first section, SWIFT Foam Structure and Mechanical Behavior, describes the structure of SWIFT foams as contrasted to other foamed rubbers. It also contrasts the mechanical behavior of SWIFT foams to other more traditional foamed rubbers. The second section, CHIPFoam Modifications, describes the specific modifications done to be able to modify the delayed rapid compressive stiffening seen in SWIFT foams. The third section, Results, demonstrates the use of the modified model to fit compressive loading response data from a SWIFT foam sample. The fourth section, Conclusions and Further Work, puts the work in perspective and presents likely further work.

36 MATERIALS SCIENCE↗
DOE OSTI2020

FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data

With ever-increasing volumes of scientific floating-point data being produced by high-performance computing applications, significantly reducing scientific floating-point data size is critical, and error-controlled lossy compressors have been developed for years. None of the existing scientific floating-point lossy data compressors, however, support effective fixed-ratio lossy compression. Yet fixed-ratio lossy compression for scientific floating-point data not only compresses to the requested ratio but also respects a user-specified error bound with higher fidelity. In this paper, we present FRaZ: a generic fixed-ratio lossy compression framework respecting user-specified error constraints. The contribution is twofold. (1) We develop an efficient iterative approach to accurately determine the appropriate error settings for different lossy compressors based on target compression ratios. (2) We perform a thorough performance and accuracy evaluation for our proposed fixed-ratio compression framework with multiple state-of-the-art error-controlled lossy compressors, using several real-world scientific floating-point datasets from different domains. Experiments show that FRaZ effectively identifies the optimum error setting in the entire error setting space of any given lossy compressor. While fixed-ratio lossy compression is slower than fixed-error compression, it provides an important new lossy compression technique for users of very large scientific floating-point datasets.

Underwood, Robert↗
DOE OSTI2022

A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1972

Radio astronomy Explorer B antenna aspect processor

The antenna aspect system used on the Radio Astronomy Explorer B spacecraft is described. This system consists of two facsimile cameras, a data encoder, and a data processor. Emphasis is placed on the discussion of the data processor, which contains a data compressor and a source encoder. With this compression scheme a compression ratio of 8 is achieved on a typical line of camera data. These compressed data are then convolutionally encoded.

Miller, W. H.↗
NASA NTRS1989

Processing SAR Images On Board

Synthetic-aperture radar (SAR) processor operates aboard moving radar platform, turning raw signal data into images of scanned terrain. In conventional SAR systems, raw data either transmitted to stations on ground or recorded on magnetic tape by high-density digital recorders for subsequent processing on ground. Advantages include making SAR images available immediately and processed data transmitted or recorded only one-fifth as voluminous as raw data. Onboard image compression reduces volume of data even further.

Liu, Kuang Y.↗
NASA NTRS1992

A comparison of three SAR interference filters

Three interference filters are discussed and their ability to remove the interference while leaving target signal data is quantified. The filters are adaptive frequency-domain spike filtering; adaptive frequency-domain interference determination and masking; and interference-target correlation masking. Using Jet Propulsion Laboratory Airborne Synthetic Aperture Radar (JPL AIRSAR) data, the algorithms are applied to three data sets. The resultant compressed Stokes matrix data are compared with the normally processed interference-free image in a quantitative sense. The algorithms are evaluated by measuring the polarization signature, SNR, equivalent number of looks, and impulse response of the processed images.

Chapman, B.↗
NASA NTRS1966

Data analysis and reporting

Selection and compression of flight data for real time monitoring and data reduction for Gemini flights

DATA PROCESSING↗
NASA NTRS1978

Adiabatic heating in impulsive solar flares

A study is made of adiabatic heating in two impulsive solar flares on the basis of dynamic X-ray spectra in the 28-254 keV range, H-alpha, microwave, and meter-wave radio observations. It is found that the X-ray spectra of the events are like those of thermal bremsstrahlung from single-temperature plasmas in the 10-60 keV range if photospheric albedo is taken into account. The temperature-emission correlation indicates adiabatic compression followed by adiabatic expansion and that the electron distribution remains isotropic. H-alpha data suggest compressive energy transfer. The projected areas and volumes of the flares are estimated assuming that X-ray and microwave emissions are produced in a single thermal plasma. Electron densities of about 10 to the 9th/cu cm are found for homogeneous, spherically symmetric sources. It is noted that the strong self-absorption of hot-plasma gyrosynchrotron radiation reveals low magnetic field strengths.

Maetzler, C.↗
NASA NTRS2002

Tensile Creep of Polycrystalline Near-Stoichiometric NiAl

Long term tensile creep studies were conducted on binary NiAl in the temperature range 700-1200 K with the objectives of characterizing and understanding the creep mechanisms. Inverse and normal primary creep curves were observed depending on stress and temperature. It was concluded that the creep of NiAl is limited by dislocation mobility. The stress exponent for creep, n, increased from 5.5 at 1200 K to 13.9 at 700 K. The true activation energy for creep, Qc, was constant and equal to about 400 kJ per mole between 20 and 50 MPa but decreased to a constant value of 250 kJ per mole between 50 and 110 MPa. The activation energy was observed to be stress dependent above 110 MPa. The tensile creep results reported in this investigation were compared with compression creep data reported in the literature. A detailed discussion of the probable dislocation creep mechanisms governing compressive and tensile creep of NiAl is presented. It is concluded that the non-conservative motion of jogs on screw dislocations influenced the nature of the primary creep curves, where the climb of these jogs involves either the next nearest neighbor or the six-jump cycle vacancy diffusion mechanism. The probable nature of the atom vacancy exchange that occur within the core of an edge dislocation undergoing climb in NiAl are schematically examined.

Raj, Sai V.↗
DOE OSTI2024

A novel method may reveal bulk metallic glass compressive ductility trends in high data rate nanoindentation

Recent methods allow novel amorphous alloy compositions to be rapidly manufactured at small scale; however, obtaining materials properties such as compressive ductility from these smaller specimens has remained a challenge. Here, we suggest a potential high-throughput nanoindentation method that may be able to rapidly characterize the relative compressive ductility between these alloys based on their serration characteristics. The properties of emergent serrations, when interpreted in a simple micromechanical stress relaxation model, may order these materials by their compressive plastic strain to failure. These results are consistent with the ordering obtained from compressed specimens as well as with model simulations, suggesting that this model may be broadly useful for interpreting compressive ductility from nanoindentation serrations. After it is validated on more materials, this new method will match the rapid pace of amorphous alloy development, thus allowing metallic glass properties to be fine-tuned for each application prior to scale prototyping.

36 MATERIALS SCIENCE↗
NASA NTRS2015

A Priori Analysis of a Compressible Flamelet Model using RANS Data for a Dual-Mode Scramjet Combustor

In an effort to make large eddy simulation of hydrocarbon-fueled scramjet combustors more computationally accessible using realistic chemical reaction mechanisms, a compressible flamelet/progress variable (FPV) model was proposed that extends current FPV model formulations to high-speed, compressible flows. Development of this model relied on observations garnered from an a priori analysis of the Reynolds-Averaged Navier-Stokes (RANS) data obtained for the Hypersonic International Flight Research and Experimentation (HI-FiRE) dual-mode scramjet combustor. The RANS data were obtained using a reduced chemical mechanism for the combustion of a JP-7 surrogate and were validated using avail- able experimental data. These RANS data were then post-processed to obtain, in an a priori fashion, the scalar fields corresponding to an FPV-based modeling approach. In the current work, in addition to the proposed compressible flamelet model, a standard incompressible FPV model was also considered. Several candidate progress variables were investigated for their ability to recover static temperature and major and minor product species. The effects of pressure and temperature on the tabulated progress variable source term were characterized, and model coupling terms embedded in the Reynolds- averaged Navier-Stokes equations were studied. Finally, results for the novel compressible flamelet/progress variable model were presented to demonstrate the improvement attained by modeling the effects of pressure and flamelet boundary conditions on the combustion.

Quinlan, Jesse R.↗
DOE OSTI2018

Scalable Data Management, Analysis, and Visualization (SDAV) Institute

Analytics-driven data layout optimizations and compression (NCSU): We have developed solutions that address challenges related to scientific data indexing, compression, and storage layout problems. Moreover, in addition to focusing on mainstream uniform mesh data, we have also addressed Adaptive Mesh Refinement (AMR) data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Neural-network learning of SPOD latent dynamics

Here, we aim to reconstruct the latent space dynamics of high dimensional, quasi-stationary systems using model order reduction via the spectral proper orthogonal decomposition (SPOD). The proposed method is based on three fundamental steps: in the first, once that the mean flow field has been subtracted from the realizations (also referred to as snapshots), we compress the data from a high-dimensional representation to a lower dimensional one by constructing the SPOD latent space; in the second, we build the time-dependent coefficients by projecting the snapshots containing the fluctuations onto the SPOD basis and we learn their evolution in time with the aid of recurrent neural networks; in the third, we reconstruct the high-dimensional data from the learnt lower -dimensional representation. The proposed method is demonstrated on two different test cases, namely, a compressible jet flow, and a geophysical problem known as the Madden-Julian Oscillation. An extensive comparison between SPOD and the equivalent POD-based counterpart is provided and differences between the two approaches are highlighted. The numerical results suggest that the proposed model is able to provide low rank predictions of complex statistically stationary data and to provide insights into the evolution of phenomena characterized by specific range of frequencies. The comparison between POD and SPOD surrogate strategies highlights the need for further work on the characterization of the interplay of error between data reduction techniques and neural network forecasts.

97 MATHEMATICS AND COMPUTING↗