SEARCH · Search NASA

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Electronic device and method for compressing sampled data

An electronic device for compressing sampled data comprises a memory element and a processing element. The memory element is configured to store sampled data points and sampled times. The processing element is in electronic communication with the memory element and is configured to receive a plurality of sampled data points, a slope for each sampled data point in succession, the slope being a value of a change between the sampled data point and its successive sampled data point, and store the sampled data point in the memory element when the slope changes in value from a previous sampled data point.

Morrow, Mitchell Hedges↗

DOE OSTI2021

Electronic device and method for compressing sampled data

Tohlen, Michael Aaron↗

DOE OSTI2021

Electronic device and method for compressing sampled data

Morrow, Mitchell Hedges↗

NASA NTRS1994

Algorithm For Integer Cosine Transforms

Algorithm computes coefficients of integer cosine transforms, containing only integer coefficients approximately proportional to floating-point coefficients of discrete cosine transforms. Used in compression of image data in blocks of N x N pixels, discrete cosine transforms involve large amounts of computation, typically amounting to 80 percent of computational load in data-compression scheme. In constructing integer approximations, competing requirements are to obtain data-compression and data-decompression performances comparable to those of corresponding descrete cosine transforms while taking advantage of simplicity afforded by integer arithmetic to reduce complexity and amount of computation.

Pollara, Fabrizio↗

NASA NTRS1988

Adaptive Bandwidth Compression For Moving Images

Data compressed in images with fast motion and expanded when motion slows. Proposed data-compression scheme adapts transmission rate for moving image to speed of motion of image. Laser rangefinder measures speed of target. Adaptive sampler determines required resolution of target image on basis of measured speed and provides decommutator at control station with more or fewer partial image fields per unit time. When monitor displays maximum number of fields simultaneously, resolution is highest.

Graham, Olin L.↗

NASA NTRS1976

Super high compression of line drawing data

Models which can be used to accurately represent the type of line drawings which occur in teleconferencing and transmission for remote classrooms and which permit considerable data compression were described. The objective was to encode these pictures in binary sequences of shortest length but such that the pictures can be reconstructed without loss of important structure. It was shown that exploitation of reasonably simple structure permits compressions in the range of 30-100 to 1. When dealing with highly stylized material such as electronic or logic circuit schematics, it is unnecessary to reproduce configurations exactly. Rather, the symbols and configurations must be understood and be reproduced, but one can use fixed font symbols for resistors, diodes, capacitors, etc. Compression of pictures of natural phenomena such as can be realized by taking a similar approach, or essentially zero error reproducibility can be achieved but at a lower level of compression.

Cooper, D. B.↗

DOE OSTI2023

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1991

Techniques for containing error propagation in compression/decompression schemes

Data compression has the potential for increasing the risk of data loss. It can also cause bit error propagation, resulting in catastrophic failures. There are a number of approaches possible for containing error propagation due to data compression: (1) data retransmission; (2) data interpolation; (3) error containment; and (4) error correction. The most fruitful techniques will be ones where error containment and error correction are integrated with data compression to provide optimal performance for both. The error containment characteristics of existing compression schemes should be analyzed for their behavior under different data and error conditions. The error tolerance requirements of different data sets need to be understood, so guidelines can then be developed for matching error requirements to suitable compression algorithms.

Kobler, Ben↗

NASA NTRS1996

Communications and information research: Improved space link performance via concatenated forward error correction coding

With the development of new advanced instruments for remote sensing applications, sensor data will be generated at a rate that not only requires increased onboard processing and storage capability, but imposes demands on the space to ground communication link and ground data management-communication system. Data compression and error control codes provide viable means to alleviate these demands. Two types of data compression have been studied by many researchers in the area of information theory: a lossless technique that guarantees full reconstruction of the data, and a lossy technique which generally gives higher data compaction ratio but incurs some distortion in the reconstructed data. To satisfy the many science disciplines which NASA supports, lossless data compression becomes a primary focus for the technology development. While transmitting the data obtained by any lossless data compression, it is very important to use some error-control code. For a long time, convolutional codes have been widely used in satellite telecommunications. To more efficiently transform the data obtained by the Rice algorithm, it is required to meet the a posteriori probability (APP) for each decoded bit. A relevant algorithm for this purpose has been proposed which minimizes the bit error probability in the decoding linear block and convolutional codes and meets the APP for each decoded bit. However, recent results on iterative decoding of 'Turbo codes', turn conventional wisdom on its head and suggest fundamentally new techniques. During the past several months of this research, the following approaches have been developed: (1) a new lossless data compression algorithm, which is much better than the extended Rice algorithm for various types of sensor data, (2) a new approach to determine the generalized Hamming weights of the algebraic-geometric codes defined by a large class of curves in high-dimensional spaces, (3) some efficient improved geometric Goppa codes for disk memory systems and high-speed mass memory systems, and (4) a tree based approach for data compression using dynamic programming.

Rao, T. R. N.↗

NASA NTRS1994

Advanced End-to-end Simulation for On-board Processing (AESOP)

Developers of data compression algorithms typically use their own software together with commercial packages to implement, evaluate and demonstrate their work. While convenient for an individual developer, this approach makes it difficult to build on or use another's work without intimate knowledge of each component. When several people or groups work on different parts of the same problem, the larger view can be lost. What's needed is a simple piece of software to stand in the gap and link together the efforts of different people, enabling them to build on each other's work, and providing a base for engineers and scientists to evaluate the parts as a cohesive whole and make design decisions. AESOP (Advanced End-to-end Simulation for On-board Processing) attempts to meet this need by providing a graphical interface to a developer-selected set of algorithms, interfacing with compiled code and standalone programs, as well as procedures written in the IDL and PV-Wave command languages. As a proof of concept, AESOP is outfitted with several data compression algorithms integrating previous work on different processors (AT&T DSP32C, TI TMS320C30, SPARC). The user can specify at run-time the processor on which individual parts of the compression should run. Compressed data is then fed through simulated transmission and uncompression to evaluate the effects of compression parameters, noise and error correction algorithms. The following sections describe AESOP in detail. Section 2 describes fundamental goals for usability. Section 3 describes the implementation. Sections 4 through 5 describe how to add new functionality to the system and present the existing data compression algorithms. Sections 6 and 7 discuss portability and future work.

Mazer, Alan S.↗

NASA NTRS1991

Recent advances in coding theory for near error-free communications

Channel and source coding theories are discussed. The following subject areas are covered: large constraint length convolutional codes (the Galileo code); decoder design (the big Viterbi decoder); Voyager's and Galileo's data compression scheme; current research in data compression for images; neural networks for soft decoding; neural networks for source decoding; finite-state codes; and fractals for data compression.

Cheung, K.-M.↗

NASA NTRS1993

Applications of wavelet-based compression to multidimensional Earth science data

A data compression algorithm involving vector quantization (VQ) and the discrete wavelet transform (DWT) is applied to two different types of multidimensional digital earth-science data. The algorithms (WVQ) is optimized for each particular application through an optimization procedure that assigns VQ parameters to the wavelet transform subbands subject to constraints on compression ratio and encoding complexity. Preliminary results of compressing global ocean model data generated on a Thinking Machines CM-200 supercomputer are presented. The WVQ scheme is used in both a predictive and nonpredictive mode. Parameters generated by the optimization algorithm are reported, as are signal-to-noise (SNR) measurements of actual quantized data. The problem of extrapolating hydrodynamic variables across the continental landmasses in order to compute the DWT on a rectangular grid is discussed. Results are also presented for compressing Landsat TM 7-band data using the WVQ scheme. The formulation of the optimization problem is presented along with SNR measurements of actual quantized data. Postprocessing applications are considered in which the seven spectral bands are clustered into 256 clusters using a k-means algorithm and analyzed using the Los Alamos multispectral data analysis program, SPECTRUM, both before and after being compressed using the WVQ program.

Bradley, Jonathan N.↗

NASA NTRS2004

The New CCSDS Image Compression Recommendation

The Consultative Committee for Space Data Systems (CCSDS) data compression working group has recently adopted a recommendation for image data compression, with a final release expected in 2005. The algorithm adopted in the recommendation consists a two dimensional discrete wavelet transform of the image, followed by progressive bit-plane coding of the transformed data. The algorithm can provide both lossless and lossy compression, and allows a user to directly control the compressed data volume or the fidelity with which the wavelet-transformed data can be reconstructed. The algorithm is suitable for both frame-based image data and scan-based sensor data, and has applications for near-earth and deep-space missions. The standard will be accompanied by free software sources on a future web site. An ASIC implementation of the compressor is currently under development. This paper describes the compression algorithm along with the requirements that drove the selection of the algorithm.

image compression↗

NASA NTRS2005

The New CCSDS Image Compression Recommendation

The Consultative Committee for Space Data Systems (CCSDS) data compression working group has recently adopted a recommendation for image data compression, with a final release expected in 2005. The algorithm adopted in the recommendation consists of a two-dimensional discrete wavelet transform of the image, followed by progressive bit-plane coding of the transformed data. The algorithm can provide both lossless and lossy compression, and allows a user to directly control the compressed data volume or the fidelity with which the wavelet-transformed data can be reconstructed. The algorithm is suitable for both frame-based image data and scan-based sensor data, and has applications for near-Earth and deep-space missions. The standard will be accompanied by free software sources on a future web site. An Application-Specific Integrated Circuit (ASIC) implementation of the compressor is currently under development. This paper describes the compression algorithm along with the requirements that drove the selection of the algorithm. Performance results and comparisons with other compressors are given for a test set of space images.

Yeh, Pen-Shu↗

DOE OSTI2023

Compression of tokamak boundary plasma simulation data using a maximum volume algorithm for matrix skeleton decomposition

This report demonstrates satisfactory data compression of SOLPS-ITER simulation output ranging from 2D fields, 1D profiles, and 0D scalar variables with a novel matrix decomposition approach. The singular value decomposition (SVD) scales poorly for large matrix sizes and is unsuited to the application on high dimensional data common to fusion plasma physics simulation. In this work, we employ the columns-submatrix-rows (CUR) matrix factorization technique in order to compute a low-rank approximation up to two orders of magnitude faster than the SVD, but within a nominal L2-norm relative error of ε = 10 –2 . In addition, the CUR approach maintains the original format of the data, in its extracted columns and rows, allowing for interpretable data storage at the original resolution of the simulation. We utilize an iterative algorithm to compute the CUR decomposition of simulation output by maximizing the volume, or linearly independent information content, of a low-rank submatrix contained within the data. Experiments over $\textit{n} × \textit{n}$ randomized test matrices with embedded rank-deficient features show that this maximum volume implementation of CUR matrix approximation has reduced asymptotic computational complexity on the order of n compared to the SVD, which scales approximately as $n^3$. These results show that the CUR technique can be used to effectively select time step snapshots (columns) of over 140 SOLPS-ITER output variables and the associated discretized coordinate timeseries (rows) allowing for reconstruction of the complete simulation dynamics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Fast digital lossy compression for X-ray ptychographic data

Increases in X-ray brightness from synchrotron light sources lead to a requirement for higher frame rates from hybrid pixel array detectors (HPADs), while also favoring charge integration over photon counting. However, transfer of the full uncompressed data will begin to constrain detector design, as well as limit the achievable continuous frame rate. Here a data compression scheme that is easy to implement in a HPAD's application-specific integrated circuit (ASIC) is described, and how different degrees of compression affect image quality in ptychography, a commonly employed coherent imaging method, is examined. Using adaptive encoding quantization, it is shown in simulations that one can digitize signals up to 16383 photons per pixel (corresponding to 14 bits of information) using only 8 or 9 bits for data transfer, with negligible effect on the reconstructed image.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2016

SU-G-JeP1-15: Sliding Window Prior Data Assisted Compressed Sensing for MRI Lung Tumor Tracking

Purpose: Prior Data Assisted Compressed Sensing (PDACS) is a partial k-space acquisition and reconstruction method for mobile tumour (i.e. lung) tracking using on-line MRI in radiotherapy. PDACS partially relies on prior data acquired at the beginning of dynamic scans, and is therefore susceptible to artifacts in longer duration scan due to slow drifts in MR signal. A novel sliding window strategy is presented to mitigate this effect. Methods: MRI acceleration is simulated by retrospective removal of data from the fully sampled sets. Six lung cancer patients were scanned (clinical 3T MRI) using a balanced steady state free precession (bSSFP) sequence for 3 minutes at approximately 4 frames per second, for a total of 650 dynamics. PDACS acceleration is achieved by undersampling of k-space in a single pseudo-random pattern. Reconstruction iteratively minimizes the total variations while constraining the images to satisfy both the currently acquired data and the prior data in missing k-space. Our novel sliding window technique (SW-PDACS), uses a series of distinct pseudo-random under-sampling patterns of partial k-space – with the prior data drawn from a sliding window of the most recent data available. Under-sampled data, simulating 2 – 5x acceleration are reconstructed using PDACS and SW-PDACS. Three quantitative metrics: artifact power, centroid error and Dice’s coefficient are computed for comparison. Results: Quantitively metric values from all 6 patients are averaged in 3 bins, each containing approximately one minute of dynamic data. For the first minute bin, PDACS and SW-PDACS give comparable results. Progressive decline in image quality metrics in bins 2 and 3 are observed for PDACS. No decline in image quality is observed for SW-PDACS. Conclusion: The novel approach presented (SW-PDACS) is a more robust for accelerating longer duration (>1 minute) dynamic MRI scans for tracking lung tumour motion using on-line MRI in radiotherapy. B.G. Fallone is a co-founder and CEO of MagnetTx Oncology Solutions (under discussions to license Alberta bi-planar linac MR for commercialization).

60 APPLIED LIFE SCIENCES↗

NASA NTRS2009

Lossless Compression of Data into Fixed-Length Packets

A computer program effects lossless compression of data samples from a one-dimensional source into fixed-length data packets. The software makes use of adaptive prediction: it exploits the data structure in such a way as to increase the efficiency of compression beyond that otherwise achievable. Adaptive linear filtering is used to predict each sample value based on past sample values. The difference between predicted and actual sample values is encoded using a Golomb code.

Kiely, Aaron B.↗