Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Comparison of the lossy image data compressions for the MESUR Pathfinder and for the Huygens Titan Probe

The commercial JPEG standard complies well with the specific requirements of exploratory space missions. Therefore, JPEG has been chosen to be the baseline for a series of spaceborne image data compressions (e.g. MARS94-HRSC, -WAOSS, HUYGENS-DISR, MESUR-IMP). One S/W-implementation (IMP) and one H/W-implementation (DISR) of image data compression are presented. Details of the modifications applied to standard JPEG are outlined. Finally, a performance comparison of the two implementations is given.

Rueffer, P.↗
NASA NTRS2000

A Real-Time High Performance Data Compression Technique For Space Applications

A high performance lossy data compression technique is currently being developed for space science applications under the requirement of high-speed push-broom scanning. The technique is also error-resilient in that error propagation is contained within a few scan lines. The algorithm is based on block-transform combined with bit-plane encoding; this combination results in an embedded bit string with exactly the desirable compression rate. The lossy coder is described. The compression scheme performs well on a suite of test images typical of images from spacecraft instruments. Hardware implementations are in development; a functional chip set is expected by the end of 2001.

Yeh, Pen-Shu↗
NASA NTRS2002

The CCDS Data Compression Recommendations: Development and Status

The Consultative Committee for Space Data Systems (CCSDS) has been engaging in recommending data compression standards for space applications. The first effort focused on a lossless scheme that was adopted in 1997. Since then, space missions benefiting from this recommendation range from deep space probes to near Earth observatories. The cost savings result not only from reduced onboard storage and reduced bandwidth, but also in ground archive of mission data. In many instances, this recommendation also enables more science data to be collected for added scientific value. Since 1998, the compression sub-panel of CCSDS has been investigating lossy image compression schemes and is currently working towards a common solution for a single recommendation. The recommendation will fulfill the requirements for remote sensing conducted on space platforms.

Yeh, Pen-Shu↗
NASA NTRS1995

Quantization Distortion in Block Transform-Compressed Data

The popular JPEG image compression standard is an example of a block transform-based compression scheme; the image is systematically subdivided into block that are individually transformed, quantized, and encoded. The compression is achieved by quantizing the transformed data, reducing the data entropy and thus facilitating efficient encoding. A generic block transform model is introduced.

block transform-compressed data quantization image↗
DOE OSTI2020

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Data compression experiments with LANDSAT thematic mapper and Nimbus-7 coastal zone color scanner data

A case study is presented where an image segmentation based compression technique is applied to LANDSAT Thematic Mapper (TM) and Nimbus-7 Coastal Zone Color Scanner (CZCS) data. The compression technique, called Spatially Constrained Clustering (SCC), can be regarded as an adaptive vector quantization approach. The SCC can be applied to either single or multiple spectral bands of image data. The segmented image resulting from SCC is encoded in small rectangular blocks, with the codebook varying from block to block. Lossless compression potential (LDP) of sample TM and CZCS images are evaluated. For the TM test image, the LCP is 2.79. For the CZCS test image the LCP is 1.89, even though when only a cloud-free section of the image is considered the LCP increases to 3.48. Examples of compressed images are shown at several compression ratios ranging from 4 to 15. In the case of TM data, the compressed data are classified using the Bayes' classifier. The results show an improvement in the similarity between the classification results and ground truth when compressed data are used, thus showing that compression is, in fact, a useful first step in the analysis.

Tilton, James C.↗
NASA NTRS1994

A comparative study of SAR data compression schemes

The amount of data collected from spaceborne remote sensing has substantially increased in the last years. During same time period, the ability to store or transmit data has not increased as quickly. At this time, there is a growing interest in developing compression schemes that could provide both higher compression ratios and lower encoding/decoding errors. In the case of the spaceborne Synthetic Aperture Radar (SAR) earth observation system developed by the French Space Agency (CNES), the volume of data to be processed will exceed both the on-board storage capacities and the telecommunication link. The objective of this paper is twofold: to present various compression schemes adapted to SAR data; and to define a set of evaluation criteria and compare the algorithms on SAR data. In this paper, we review two classical methods of SAR data compression and propose novel approaches based on Fourier Transforms and spectrum coding.

Lambert-Nebout, C.↗
NASA NTRS1992

Applications of data compression techniques in modal analysis for on-orbit system identification

Data compression techniques have been investigated for use with modal analysis applications. A redundancy-reduction algorithm was used to compress frequency response functions (FRFs) in order to reduce the amount of disk space necessary to store the data and/or save time in processing it. Tests were performed for both single- and multiple-degree-of-freedom (SDOF and MDOF, respectively) systems, with varying amounts of noise. Analysis was done on both the compressed and uncompressed FRFs using an SDOF Nyquist curve fit as well as the Eigensystem Realization Algorithm. Significant savings were realized with minimal errors incurred by the compression process.

Carlin, Robert A.↗
NASA NTRS1968

Information rates for data compression.

Upper and lower bounds on required information rate for data compression fidelity criterion, comparing bounds with equivalent mean-square-error result

Davisson, L. D.↗
NASA NTRS1992

Data compression of stereopairs

This paper addresses the problem of stereopair data compression. Two fundamentally different techniques for compressing stereopairs are discussed. The first technique, called disparity-compensated transform-domain predictive coding, attempts to minimize the mean-square error between the original stereopair and the compressed stereopair. The second technique, called mixed-resolution coding, is a psychophysically justified technique that exploits known facts about human stereovision to code stereopairs in a subjectively acceptable manner. A method for assessing the quality of compressed stereopairs is also presented. It involves measuring the ability of an observer to perceive depth in coded stereopairs. It is found that observers generally perceived objects to be further away in compressed stereopairs than they did in originals. Finally, the problem of coding stereopairs is considered from a rate-distortion perspective. It is proved that the rate distortion limit for coding stereopairs cannot in general be achieved by a coder that first codes and decodes the right picture sequence independently of the left picture sequence, and then codes and decodes the left picture sequence given the decoded right picture sequence.

Perkins, Michael G.↗
DOE OSTI2020

Deep learning for in situ data compression of large turbulent flow simulations

As the size of turbulent flow simulations continues to grow, in situ data compression is becoming increasingly important for visualization, analysis, and restart checkpointing. For these applications, single-pass compression techniques with low computational and communication overhead are crucial. In this paper we present a deep-learning approach to in situ compression using an autoencoder architecture that is customized for three-dimensional turbulent flows and is well suited for contemporary heterogeneous computing resources. The autoencoder is compared against a recently introduced randomized single-pass singular value decomposition (SVD) for three different canonical turbulent flows: decaying homogeneous isotropic turbulence, a Taylor-Green vortex, and turbulent channel flow. Our proposed fully convolutional autoencoder architecture compresses turbulent flow snapshots by a factor of 64 with a single pass, allows for arbitrarily sized input fields, is cheaper to compute than the randomized single-pass SVD for typical simulation sizes, performs well on unseen flow configurations, and has been made publicly available. The results reported here show that the autoencoder dramatically outperforms a randomized single-pass SVD with similar compression ratio and yields comparable performance to a higher-rank decomposition with an order of magnitude less compression in regard to preserving a number of important statistical quantities such as turbulent kinetic energy, enstrophy, and Reynolds stresses.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1996

Electromagnetic Scattered Field Evaluation and Data Compression Using Imaging Techniques

This is the final report on Project #727625 between The Ohio State University and NASA, Lewis Research Center, Cleveland, Ohio. Under this project, a data compression technique for scattered field data of electrically large targets is developed. The technique was applied to the scattered fields of two targets of interest. The backscattered fields of the scale models of these targets were measured in a ra compact range. For one of the targets, the backscattered fields were also calculated using XPATCH computer code. Using the technique all scattered field data sets were compressed successfully. A compression ratio of the order 40 was achieved. In this report, the technique is described briefly and some sample results are included.

Gupta, I. J.↗
NASA NTRS1993

An investigative study of multispectral data compression for remotely-sensed images using vector quantization and difference-mapped shift-coding

A study is conducted to investigate the effects and advantages of data compression techniques on multispectral imagery data acquired by NASA's airborne scanners at the Stennis Space Center. The first technique used was vector quantization. The vector is defined in the multispectral imagery context as an array of pixels from the same location from each channel. The error obtained in substituting the reconstructed images for the original set is compared for different compression ratios. Also, the eigenvalues of the covariance matrix obtained from the reconstructed data set are compared with the eigenvalues of the original set. The effects of varying the size of the vector codebook on the quality of the compression and on subsequent classification are also presented. The output data from the Vector Quantization algorithm was further compressed by a lossless technique called Difference-mapped Shift-extended Huffman coding. The overall compression for 7 channels of data acquired by the Calibrated Airborne Multispectral Scanner (CAMS), with an RMS error of 15.8 pixels was 195:1 (0.41 bpp) and with an RMS error of 3.6 pixels was 18:1 (.447 bpp). The algorithms were implemented in software and interfaced with the help of dedicated image processing boards to an 80386 PC compatible computer. Modules were developed for the task of image compression and image analysis. Also, supporting software to perform image processing for visual display and interpretation of the compressed/classified images was developed.

Jaggi, S.↗
DOE OSTI2021

Strategies for on-chip digital data compression for X-ray pixel detectors

Here, the continued desire for X-ray pixel detectors with higher frame rates will stress the ability of application-specific integrated circuit (ASIC) designers to provide sufficient off-chip bandwidth to reach continuous frame rates in the 1 MHz regime. To move from the current 10 kHz to the 1 MHz frame rate regime, ASIC designers will continue to pack as many power-hungry high-speed transceivers at the periphery of the ASIC as possible. In this paper, however, we present new strategies to make the most efficient use of the off-chip bandwidth by utilizing data compression schemes for X-ray photon-counting and charge-integrating pixel detectors. In particular, we describe a novel in-pixel compression scheme that converts from analog to digital converter units to encoded photon counts near the photon Poisson noise level and achieves a compression ratio of >1.5x independent of the dataset. In addition, we describe a simple yet efficient zero-suppression compression scheme called "zeromask" (ZM) located at the ASIC's edge before streaming data off the ASIC chip. ZM achieves average compression ratios of >4x, >7x, and >8x for high-energy X-ray diffraction, ptychography, and X-ray photon correlation spectroscopy datasets, respectively. We present the conceptual designs, register-transfer level block diagrams, and the physical ASIC implementation of these compression schemes in 65 nm CMOS. When combined, these two digital compression schemes could increase the effective off-chip bandwidth by a factor of 6-12x.

47 OTHER INSTRUMENTATION↗
NASA NTRS1993

Proposed data compression schemes for the Galileo S-band contingency mission

The Galileo spacecraft is currently on its way to Jupiter and its moons. In April 1991, the high gain antenna (HGA) failed to deploy as commanded. In case the current efforts to deploy the HGA fails, communications during the Jupiter encounters will be through one of two low gain antenna (LGA) on an S-band (2.3 GHz) carrier. A lot of effort has been and will be conducted to attempt to open the HGA. Also various options for improving Galileo's telemetry downlink performance are being evaluated in the event that the HGA will not open at Jupiter arrival. Among all viable options the most promising and powerful one is to perform image and non-image data compression in software onboard the spacecraft. This involves in-flight re-programming of the existing flight software of Galileo's Command and Data Subsystem processors and Attitude and Articulation Control System (AACS) processor, which have very limited computational and memory resources. In this article we describe the proposed data compression algorithms and give their respective compression performance. The planned image compression algorithm is a 4 x 4 or an 8 x 8 multiplication-free integer cosine transform (ICT) scheme, which can be viewed as an integer approximation of the popular discrete cosine transform (DCT) scheme. The implementation complexity of the ICT schemes is much lower than the DCT-based schemes, yet the performances of the two algorithms are indistinguishable. The proposed non-image compression algorith is a Lempel-Ziv-Welch (LZW) variant, which is a lossless universal compression algorithm based on a dynamic dictionary lookup table. We developed a simple and efficient hashing function to perform the string search.

Cheung, Kar-Ming↗
NASA NTRS1993

An adaptive, lossless data compression algorithm and VLSI implementations

This paper first provides an overview of an adaptive, lossless, data compression algorithm originally devised by Rice in the early '70s. It then reports the development of a VLSI encoder/decoder chip set developed which implements this algorithm. A recent effort in making a space qualified version of the encoder is described along with several enhancements to the algorithm. The performance of the enhanced algorithm is compared with those from other currently available lossless compression techniques on multiple sets of test data. The results favor our implemented technique in many applications.

Venbrux, Jack↗