Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Systems aspects of COBE science data compression

A general approach to compression of diverse data from large scientific projects has been developed and this paper addresses the appropriate system and scientific constraints together with the algorithm development and test strategy. This framework has been implemented for the COsmic Background Explorer spacecraft (COBE) by retrofitting the existing VAS-based data management system with high-performance compression software permitting random access to the data. Algorithms which incorporate scientific knowledge and consume relatively few system resources are preferred over ad hoc methods. COBE exceeded its planned storage by a large and growing factor and the retrieval of data significantly affects the processing, delaying the availability of data for scientific usage and software test. Embedded compression software is planned to make the project tractable by reducing the data storage volume to an acceptable level during normal processing.

Freedman, I.↗
NASA NTRS1993

A high-speed lossless data compression system for space applications

This paper reports on the integration of a lossless data compression/decompression chipset into a space data system architecture. For its compression engine, the data system incorporates the Universal Source Encoder (USE) designed for the NASA/Goddard Space Flight Center. Currently, the data compression testbed generates video frames consisting of 512 lines of 512 pixels having 8-bit resolution. Each image is passed through the USE where the lines are internally partitioned into 16-word blocks. These blocks are adaptively encoded across widely varying entropy levels using a Rice 12-option set coding algorithm. The current system operates at an Input/Output rate of 10 Msamples/s or 80 Mbits/s for each buffered input line. Frame and line synchronization for each image are maintained through the use of uniquely decodable command words. Length information of each variable length compressed image line is also included in the output stream. The data and command information are passed to the next stage of the system architecture through a serial fiber-optic transmitter. The initial segment of this stage consists of packetizer hardware which adds an appropriate CCSDS header to the received source data. An uncompressed mode is optionally available to pass image lines directly to the packetizer hardware. A data decompression testbed has also been developed to confirm the data compression operation.

Miko, Joe↗
DOE OSTI2014

Towards efficient backward-in-time adjoint computations using data compression techniques

In the context of a posteriori error estimation for nonlinear time-dependent partial differential equations, the state-of-the-practice is to use adjoint approaches which require the solution of a backward-in-time problem defined by a linearization of the forward problem. One of the major obstacles in the practical application of these approaches, we found, is the need to store, or recompute, the forward solution to define the adjoint problem and to evaluate the error representation. Our study considers the use of data compression techniques to approximate forward solutions employed in the backward-in-time integration. The development derives an error representation that accounts for the difference between the standard-approach and the compressed approximation of the forward solution. This representation is algorithmically similar to the standard representation and only requires the computation of the quantity of interest for the forward solution and the data-compressed reconstructed solution (i.e. scalar quantities that can be evaluated as the forward problem is integrated). This approach is then compared with existing techniques, such as checkpointing and time-averaged adjoints. Lastly, we provide numerical results indicating the potential efficiency of our approach on a transient diffusion–reaction equation and on the Navier–Stokes equations. These results demonstrate memory compression ratios up to 450×450× while maintaining reasonable accuracy in the error-estimates.

42 ENGINEERING↗
NASA NTRS1972

An operational data compression system using minicomputers.

Functional description of the real-time programmable data compression system that has been developed for operation between the data acquisition and satellite service ground stations and the ATS 1 and 3 satellites. The source data consist of TV pictures from these satellites that are received, compressed, transmitted, and reconstructed in real-time. General purpose minicomputers and specially designed I/O channels are used for compression and reconstruction. The installed compression system has passed all system input-output tests. Because of the great flexibility of the minicomputers, it is possible to vary the compression and to perform auxiliary tasks, such as data formatting.

Davisson, L. D.↗
DOE OSTI2023

Fast Algorithms for Scientific Data Compression

Many scientific simulations and experiments generate terabytes to petabytes of data daily, necessitating data compression techniques. Unlike video and image compression, scientists require methods that accurately preserve primary data (PD) and derived quantities of interest (QoIs). In our previous work, we demonstrated the effectiveness of hybrid compression techniques that combine machine learning with traditional approaches. This paper presents innovative computational techniques aimed at expediting the compression pipeline. Our experiments, conducted on two distinct platforms with a large-scale XGC-based fusion simulation, demonstrate that the overhead incurred by these new approaches is less than one percent of the computational resources needed for the simulation.

Banerjee, Tania↗
DOE OSTI2020

Reliable voltage scaled links for compressed data

Systems, apparatuses, and methods for reliably transmitting data over voltage scaled links are disclosed. A computing system includes at least first and second devices connected via a link. In one implementation, if a data block can be compressed to less than or equal to half the original size of the data block, then the data block is compressed and sent on the link in a single clock cycle rather than two clock cycles. If the data block cannot be compressed to half the original size, but if the data block can be compressed enough to include error correction code (ECC) bits without exceeding the original size, then ECC bits are added to the compressed block which is sent on the link at a reduced voltage. The ECC bits help to correct for any errors that are generated as a result of operating the link at the reduced voltage.

42 ENGINEERING↗
NASA NTRS1982

Image data compression application to imaging spectrometers

The potential of image data compression techniques to satisfy the anticipated requirements of imaging spectrometer missions is discussed. Noiseless coding, rate controlled compression, cluster compression, and error protection are addressed.

Rice, R. F.↗
NASA NTRS1994

Comparison of the lossy image data compressions for the MESUR Pathfinder and for the Huygens Titan Probe

The commercial JPEG standard complies well with the specific requirements of exploratory space missions. Therefore, JPEG has been chosen to be the baseline for a series of spaceborne image data compressions (e.g. MARS94-HRSC, -WAOSS, HUYGENS-DISR, MESUR-IMP). One S/W-implementation (IMP) and one H/W-implementation (DISR) of image data compression are presented. Details of the modifications applied to standard JPEG are outlined. Finally, a performance comparison of the two implementations is given.

Rueffer, P.↗
NASA NTRS2000

A Real-Time High Performance Data Compression Technique For Space Applications

A high performance lossy data compression technique is currently being developed for space science applications under the requirement of high-speed push-broom scanning. The technique is also error-resilient in that error propagation is contained within a few scan lines. The algorithm is based on block-transform combined with bit-plane encoding; this combination results in an embedded bit string with exactly the desirable compression rate. The lossy coder is described. The compression scheme performs well on a suite of test images typical of images from spacecraft instruments. Hardware implementations are in development; a functional chip set is expected by the end of 2001.

Yeh, Pen-Shu↗
NASA NTRS2002

The CCDS Data Compression Recommendations: Development and Status

The Consultative Committee for Space Data Systems (CCSDS) has been engaging in recommending data compression standards for space applications. The first effort focused on a lossless scheme that was adopted in 1997. Since then, space missions benefiting from this recommendation range from deep space probes to near Earth observatories. The cost savings result not only from reduced onboard storage and reduced bandwidth, but also in ground archive of mission data. In many instances, this recommendation also enables more science data to be collected for added scientific value. Since 1998, the compression sub-panel of CCSDS has been investigating lossy image compression schemes and is currently working towards a common solution for a single recommendation. The recommendation will fulfill the requirements for remote sensing conducted on space platforms.

Yeh, Pen-Shu↗
NASA NTRS1995

Quantization Distortion in Block Transform-Compressed Data

The popular JPEG image compression standard is an example of a block transform-based compression scheme; the image is systematically subdivided into block that are individually transformed, quantized, and encoded. The compression is achieved by quantizing the transformed data, reducing the data entropy and thus facilitating efficient encoding. A generic block transform model is introduced.

block transform-compressed data quantization image↗
DOE OSTI2020

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Data compression experiments with LANDSAT thematic mapper and Nimbus-7 coastal zone color scanner data

A case study is presented where an image segmentation based compression technique is applied to LANDSAT Thematic Mapper (TM) and Nimbus-7 Coastal Zone Color Scanner (CZCS) data. The compression technique, called Spatially Constrained Clustering (SCC), can be regarded as an adaptive vector quantization approach. The SCC can be applied to either single or multiple spectral bands of image data. The segmented image resulting from SCC is encoded in small rectangular blocks, with the codebook varying from block to block. Lossless compression potential (LDP) of sample TM and CZCS images are evaluated. For the TM test image, the LCP is 2.79. For the CZCS test image the LCP is 1.89, even though when only a cloud-free section of the image is considered the LCP increases to 3.48. Examples of compressed images are shown at several compression ratios ranging from 4 to 15. In the case of TM data, the compressed data are classified using the Bayes' classifier. The results show an improvement in the similarity between the classification results and ground truth when compressed data are used, thus showing that compression is, in fact, a useful first step in the analysis.

Tilton, James C.↗
NASA NTRS1994

A comparative study of SAR data compression schemes

The amount of data collected from spaceborne remote sensing has substantially increased in the last years. During same time period, the ability to store or transmit data has not increased as quickly. At this time, there is a growing interest in developing compression schemes that could provide both higher compression ratios and lower encoding/decoding errors. In the case of the spaceborne Synthetic Aperture Radar (SAR) earth observation system developed by the French Space Agency (CNES), the volume of data to be processed will exceed both the on-board storage capacities and the telecommunication link. The objective of this paper is twofold: to present various compression schemes adapted to SAR data; and to define a set of evaluation criteria and compare the algorithms on SAR data. In this paper, we review two classical methods of SAR data compression and propose novel approaches based on Fourier Transforms and spectrum coding.

Lambert-Nebout, C.↗
NASA NTRS1992

Applications of data compression techniques in modal analysis for on-orbit system identification

Data compression techniques have been investigated for use with modal analysis applications. A redundancy-reduction algorithm was used to compress frequency response functions (FRFs) in order to reduce the amount of disk space necessary to store the data and/or save time in processing it. Tests were performed for both single- and multiple-degree-of-freedom (SDOF and MDOF, respectively) systems, with varying amounts of noise. Analysis was done on both the compressed and uncompressed FRFs using an SDOF Nyquist curve fit as well as the Eigensystem Realization Algorithm. Significant savings were realized with minimal errors incurred by the compression process.

Carlin, Robert A.↗
NASA NTRS1968

Information rates for data compression.

Upper and lower bounds on required information rate for data compression fidelity criterion, comparing bounds with equivalent mean-square-error result

Davisson, L. D.↗