Search NASA⌕ Search

SEARCH · Search NASA

Results for “lossy compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A Real-Time High Performance Data Compression Technique For Space Applications

A high performance lossy data compression technique is currently being developed for space science applications under the requirement of high-speed push-broom scanning. The technique is also error-resilient in that error propagation is contained within a few scan lines. The algorithm is based on block-transform combined with bit-plane encoding; this combination results in an embedded bit string with exactly the desirable compression rate. The lossy coder is described. The compression scheme performs well on a suite of test images typical of images from spacecraft instruments. Hardware implementations are in development; a functional chip set is expected by the end of 2001.

Yeh, Pen-Shu↗

A Posteriori Restoration of Block Transform-Compressed Data

The Galileo spacecraft will use lossy data compression for the transmission of its science imagery over the low-bandwidth communication system. The technique chosen for image compression is a block transform technique based on the Integer Cosine Transform, a derivative of the JPEG image compression standard. Considered here are two known a posteriori enhancement techniques, which are adapted.

data compression galileo image compression compres↗

The CCDS Data Compression Recommendations: Development and Status

The Consultative Committee for Space Data Systems (CCSDS) has been engaging in recommending data compression standards for space applications. The first effort focused on a lossless scheme that was adopted in 1997. Since then, space missions benefiting from this recommendation range from deep space probes to near Earth observatories. The cost savings result not only from reduced onboard storage and reduced bandwidth, but also in ground archive of mission data. In many instances, this recommendation also enables more science data to be collected for added scientific value. Since 1998, the compression sub-panel of CCSDS has been investigating lossy image compression schemes and is currently working towards a common solution for a single recommendation. The recommendation will fulfill the requirements for remote sensing conducted on space platforms.

Yeh, Pen-Shu↗

Wavelet compression techniques for hyperspectral data

Hyperspectral sensors are electro-optic sensors which typically operate in visible and near infrared bands. Their characteristic property is the ability to resolve a relatively large number (i.e., tens to hundreds) of contiguous spectral bands to produce a detailed profile of the electromagnetic spectrum. In contrast, multispectral sensors measure relatively few non-contiguous spectral bands. Like multispectral sensors, hyperspectral sensors are often also imaging sensors, measuring spectra over an array of spatial resolution cells. The data produced may thus be viewed as a three dimensional array of samples in which two dimensions correspond to spatial position and the third to wavelength. Because they multiply the already large storage/transmission bandwidth requirements of conventional digital images, hyperspectral sensors generate formidable torrents of data. Their fine spectral resolution typically results in high redundancy in the spectral dimension, so that hyperspectral data sets are excellent candidates for compression. Although there have been a number of studies of compression algorithms for multispectral data, we are not aware of any published results for hyperspectral data. Three algorithms for hyperspectral data compression are compared. They were selected as representatives of three major approaches for extending conventional lossy image compression techniques to hyperspectral data. The simplest approach treats the data as an ensemble of images and compresses each image independently, ignoring the correlation between spectral bands. The second approach transforms the data to decorrelate the spectral bands, and then compresses the transformed data as a set of independent images. The third approach directly generalizes two-dimensional transform coding by applying a three-dimensional transform as part of the usual transform-quantize-entropy code procedure. The algorithms studied all use the discrete wavelet transform. In the first two cases, a wavelet transform coder was used for the two-dimensional compression. The third case used a three dimensional extension of this same algorithm.

Evans, Bruce↗

Data compression for near Earth and deep space to Earth transmission

Key issues of data compression for near Earth and deep space to Earth transmission discussion group are briefly presented. Specific recommendations as made by the group are as follows: (1) since data compression is a cost effective way to improve communications and storage capacity, NASA should use lossless data compression wherever possible; (2) NASA should conduct experiments and studies on the value and effectiveness of lossy data compression; (3) NASA should develop and select approaches to high ratio compression of operational data such as voice and video; (4) NASA should develop data compression integrated circuits for a few key approaches identified in the preceding recommendation; (5) NASA should examine new data compression approaches such as combining source and channel encoding, where high payoff gaps are identified in currently available schemes; and (6) users and developers of data compression technologies should be in closer communication within NASA and with academia, industry, and other government agencies.

Erickson, Daniel E.↗

Entropy reduction via simplified image contourization

The process of contourization is presented which converts a raster image into a set of plateaux or contours. These contours can be grouped into a hierarchical structure, defining total spatial inclusion, called a contour tree. A contour coder has been developed which fully describes these contours in a compact and efficient manner and is the basis for an image compression method. Simplification of the contour tree has been undertaken by merging contour tree nodes thus lowering the contour tree's entropy. This can be exploited by the contour coder to increase the image compression ratio. By applying general and simple rules derived from physiological experiments on the human vision system, lossy image compression can be achieved which minimizes noticeable artifacts in the simplified image.

Turner, Martin J.↗

Transform coding for space applications

Data compression coding requirements for aerospace applications differ somewhat from the compression requirements for entertainment systems. On the one hand, entertainment applications are bit rate driven with the goal of getting the best quality possible with a given bandwidth. Science applications are quality driven with the goal of getting the lowest bit rate for a given level of reconstruction quality. In the past, the required quality level has been nothing less than perfect allowing only the use of lossless compression methods (if that). With the advent of better, faster, cheaper missions, an opportunity has arisen for lossy data compression methods to find a use in science applications as requirements for perfect quality reconstruction runs into cost constraints. This paper presents a review of the data compression problem from the space application perspective. Transform coding techniques are described and some simple, integer transforms are presented. The application of these transforms to space-based data compression problems is discussed. Integer transforms have an advantage over conventional transforms in computational complexity. Space applications are different from broadcast or entertainment in that it is desirable to have a simple encoder (in space) and tolerate a more complicated decoder (on the ground) rather than vice versa. Energy compaction with new transforms are compared with the Walsh-Hadamard (WHT), Discrete Cosine (DCT), and Integer Cosine (ICT) transforms.

Glover, Daniel↗

Onboard Data Compression of Synthetic Aperture Radar Data: Status and Prospects

Synthetic aperture radar (SAR) instruments on spacecraft are capable of producing huge quantities of data. Onboard lossy data compression is commonly used to reduce the burden on the communication link. In this paper an overview is given of various SAR data compression techniques, along with an assessment of how much improvement is possible (and practical) and how to approach the problem of obtaining it. Synthetic aperture radar (SAR) instruments on spacecraft are capable of acquiring huge quantities of data. As a result, the available downlink rate and onboard storage capacity can be limiting factors in mission design for spacecraft with SAR instruments. This is true both for Earth-orbiting missions and missions to more distant targets such as Venus, Titan, and Europa. (Of course for missions beyond Earth orbit downlink rates are much lower and thus potentially much more limiting.) Typically spacecraft with SAR instruments use some form of data compression in order to reduce the storage size and/or downlink rate necessary to accommodate the SAR data. Our aim here is to give an overview of SAR data compression strategies that have been considered, and to assess the prospects for additional improvements.

Klimesh, Matthew A.↗

Effects of Tunable Data Compression on Geophysical Products Retrieved from Surface Radar Observations with Applications to Spaceborne Meteorological Radars

This paper presents results and analyses of applying an international space data compression standard to weather radar measurements that can easily span 8 orders of magnitude and typically require a large storage capacity as well as significant bandwidth for transmission. By varying the degree of the data compression, we analyzed the non-linear response of models that relate measured radar reflectivity and/or Doppler spectra to the moments and properties of the particle size distribution characterizing clouds and precipitation. Preliminary results for the meteorologically important phenomena of clouds and light rain indicate that for a 0.5 dB calibration uncertainty, typical for the ground-based pulsed-Doppler 94 GHz (or 3.2 mm, W-band) weather radar used as a proxy for spaceborne radar in this study, a lossless compression ratio of only 1.2 is achievable. However, further analyses of the non-linear response of various models of rainfall rate, liquid water content and median volume diameter show that a lossy data compression ratio exceeding 15 is realizable. The exploratory analyses presented are relevant to future satellite missions, where the transmission bandwidth is premium and storage requirements of vast volumes of data, potentially problematic.

cloud properties↗

Automated Assessment of Visual Quality of Digital Video

The advent of widespread distribution of digital video creates a need for automated methods for evaluating visual quality of digital video. This is particularly so since most digital video is compressed using lossy methods, which involve the controlled introduction of potentially visible artifacts. Compounding the problem is the bursty nature of digital video, which requires adaptive bit allocation based on visual quality metrics. In previous work, we have developed visual quality metrics for evaluating, controlling, and optimizing the quality of compressed still images[1-4]. These metrics incorporate simplified models of human visual sensitivity to spatial and chromatic visual signals. The challenge of video quality metrics is to extend these simplified models to temporal signals as well. In this presentation I will discuss a number of the issues that must be resolved in the design of effective video quality metrics. Among these are spatial, temporal, and chromatic sensitivity and their interactions, visual masking, and implementation complexity. I will also touch on the question of how to evaluate the performance of these metrics.

Watson, Andrew B.↗

Towards a Visual Quality Metric for Digital Video

The advent of widespread distribution of digital video creates a need for automated methods for evaluating visual quality of digital video. This is particularly so since most digital video is compressed using lossy methods, which involve the controlled introduction of potentially visible artifacts. Compounding the problem is the bursty nature of digital video, which requires adaptive bit allocation based on visual quality metrics. In previous work, we have developed visual quality metrics for evaluating, controlling, and optimizing the quality of compressed still images. These metrics incorporate simplified models of human visual sensitivity to spatial and chromatic visual signals. The challenge of video quality metrics is to extend these simplified models to temporal signals as well. In this presentation I will discuss a number of the issues that must be resolved in the design of effective video quality metrics. Among these are spatial, temporal, and chromatic sensitivity and their interactions, visual masking, and implementation complexity. I will also touch on the question of how to evaluate the performance of these metrics.

Watson, Andrew B.↗

The moderate resolution imaging spectrometer: An EOS facility instrument candidate for application of data compression methods

The Moderate Resolution Imaging Spectrometer (MODIS) observing facility will operate on the Earth Observing System (EOS) in the late 1990's. It is estimated that this observing facility will produce over 200 gigabytes of data per day requiring a storage capability of just over 300 gigabytes per day. Archiving, browsing, and distributing the data associated with MODIS represents a rich opportunity for testing and applying both lossless and lossy data compression methods.

Salomonson, Vincent V.↗

Pre-coding method and apparatus for multiple source or time-shifted single source data and corresponding inverse post-decoding method and apparatus

A pre-coding method and device for improving data compression performance by removing correlation between a first original data set and a second original data set, each having M members, respectively. The pre-coding method produces a compression-efficiency-enhancing double-difference data set. The method and device produce a double-difference data set, i.e., an adjacent-delta calculation performed on a cross-delta data set or a cross-delta calculation performed on two adjacent-delta data sets, from either one of (1) two adjacent spectral bands coming from two discrete sources, respectively, or (2) two time-shifted data sets coming from a single source. The resulting double-difference data set is then coded using either a distortionless data encoding scheme (entropy encoding) or a lossy data compression scheme. Also, a post-decoding method and device for recovering a second original data set having been represented by such a double-difference data set.

Yeh, Pen-Shu↗

Pre-coding method and apparatus for multiple source or time-shifted single source data and corresponding inverse post-decoding method and apparatus

A pre-coding method and device for improving data compression performance by removing correlation between a first original data set and a second original data set, each having M members, respectively. The pre-coding method produces a compression-efficiency-enhancing double-difference data set. The method and device produce a double-difference data set, i.e., an adjacent-delta calculation performed on a cross-delta data set or a cross-delta calculation performed on two adjacent-delta data sets, from either one of (1) two adjacent spectral bands coming from two discrete sources, respectively, or (2) two time-shifted data sets coming from a single source. The resulting double-difference data set is then coded using either a distortionless data encoding scheme (entropy encoding) or a lossy data compression scheme. Also, a post-decoding method and device for recovering a second original data set having been represented by such a double-difference data set.

Yeh, Pen-Shu↗

Feasibility of Formulating Ecosystem Biogeochemical Models From Established Physical Rules

Abstract To improve the predictive capability of ecosystem biogeochemical models (EBMs), we discuss the feasibility of formulating biogeochemical processes using physical rules that have underpinned the many successes in computational physics and chemistry. We argue that the currently popular empirically based approaches, such as multiplicative empirical response functions and the law of the minimum, will not lead to EBM formulations that can be continuously refined to incorporate improved mechanistic understanding and empirical observations of biogeochemical processes. Instead, we propose that EBM parameterizations, as a lossy data compression problem, can be better formulated using established physical rules widely used in computational physics and chemistry, and different biogeochemical processes can be more robustly integrated within a reactive‐transport framework. Through several examples, we demonstrate how mathematical representations derived from physical rules can improve understanding of relevant biogeochemical processes and enable more effective communication between modelers, observationalists, and experimentalists regarding essential questions, such as what measurements are needed to meaningfully inform models and how can models generate new process‐level hypotheses to test in empirical studies. Finally, while empirical models with more parameters are often less robust, physical rules‐based models can be more robust and show lower predictive equifinality, stemming from their enhanced consistency in representations of processes, interactions and spatial scaling.

54 ENVIRONMENTAL SCIENCES↗

Variational autoencoders for at-source data reduction and anomaly detection in high energy particle detectors

Detectors in next-generation high-energy physics experiments face several daunting requirements, such as high data rates, damaging radiation exposure, and stringent constraints on power, space, and latency. To address these challenges, machine learning in readout electronics can be leveraged for smart detector designs, enabling intelligent inference and data reduction at-source. Variational autoencoders (VAEs) offer a variety of benefits for front-end readout; an on-sensor encoder can perform efficient lossy data compression while simultaneously providing a latent space representation that can be used for anomaly detection. Results are presented from low-latency and resource-efficient VAEs for front-end data processing in a futuristic silicon pixel detector. Encoder-based data compression is found to preserve good performance of off-detector analysis while significantly reducing the off-detector data rate as compared to a similarly sized data filtering approach. Furthermore, the latent space information is found to be a useful discriminator in the context of real-time sensor defect monitoring. Together, these results highlight the multifaceted utility of autoencoder-based front-end readout schemes and motivate their consideration in future detector designs.

47 OTHER INSTRUMENTATION↗