SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

NASA NTRS2002

Onboard Processor for Compressing HSI Data

With EO-1 Hyperion and MightySat in orbit NASA and the DoD are showing their continued commitment to hyperspectral imaging (HSI). As HSI sensor technology continues to mature, the ever-increasing amounts of sensor data generated will result in a need for more cost effective communication and data handling systems. Lockheed Martin, with considerable experience in spacecraft design and developing special purpose onboard processors, has teamed with Applied Signal & Image Technology (ASIT), who has an extensive heritage in HSI, to develop a real-time and intelligent onboard processing (OBP) system to reduce HSI sensor downlink requirements. Our goal is to reduce the downlink requirement by a factor greater than 100, while retaining the necessary spectral fidelity of the sensor data needed to satisfy the many science, military, and intelligence goals of these systems. Our initial spectral compression experiments leverage commercial-off-the-shelf (COTS) spectral exploitation algorithms for segmentation, material identification and spectral compression that ASIT has developed. ASIT will also support the modification and integration of this COTS software into the OBP. Other commercially available COTS software for spatial compression will also be employed as part of the overall compression processing sequence. Over the next year elements of a high-performance reconfigurable OBP will be developed to implement proven preprocessing steps that distill the HSI data stream in both spectral and spatial dimensions. The system will intelligently reduce the volume of data that must be stored, transmitted to the ground, and processed while minimizing the loss of information.

Cook, Sid↗

NASA NTRS1982

End-to-end imaging information rate advantages of various alternative communication systems

The efficiency of various deep space communication systems which are required to transmit both imaging and a typically error sensitive class of data called general science and engineering (gse) are compared. The approach jointly treats the imaging and gse transmission problems, allowing comparisons of systems which include various channel coding and data compression alternatives. Actual system comparisons include an advanced imaging communication system (AICS) which exhibits the rather significant advantages of sophisticated data compression coupled with powerful yet practical channel coding. For example, under certain conditions the improved AICS efficiency could provide as much as two orders of magnitude increase in imaging information rate compared to a single channel uncoded, uncompressed system while maintaining the same gse data rate in both systems. Additional details describing AICS compression and coding concepts as well as efforts to apply them are provided in support of the system analysis.

Rice, R. F.↗

DOE OSTI2020

Machine Learning for Memory Reduction in the Implicit Monte Carlo Simulations of Thermal Radiative Transfer [Slides]

Project Goal: Use parametric Machine Learning methods in order to reduce memory requirements at checkpointing & restarting in the IMC simulations of Thermal Radiative Transfer using: Expectation Maximization and Weighted Gaussian Mixture Model-based approach for `particle-data compression', introduced in Plasma Physics to model Maxwellian particle distributions by Luis Chacon and Guangye Chen; Expectation Maximization with Weighted Hyper-Erlang Model in order to compress isotropic IMC particle data in the frequency domain; and Expectation Maximization and von Mises-Fisher Mixture Model for compression of anisotropic IMC particle data in the angular domain (work-in-progress).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS2009

Sequential Principal Component Analysis -An Optimal and Hardware-Implementable Transform for Image Compression

This paper presents the JPL-developed Sequential Principal Component Analysis (SPCA) algorithm for feature extraction / image compression, based on "dominant-term selection" unsupervised learning technique that requires an order-of-magnitude lesser computation and has simpler architecture compared to the state of the art gradient-descent techniques. This algorithm is inherently amenable to a compact, low power and high speed VLSI hardware embodiment. The paper compares the lossless image compression performance of the JPL's SPCA algorithm with the state of the art JPEG2000, widely used due to its simplified hardware implementability. JPEG2000 is not an optimal data compression technique because of its fixed transform characteristics, regardless of its data structure. On the other hand, conventional Principal Component Analysis based transform (PCA-transform) is a data-dependent-structure transform. However, it is not easy to implement the PCA in compact VLSI hardware, due to its highly computational and architectural complexity. In contrast, the JPL's "dominant-term selection" SPCA algorithm allows, for the first time, a compact, low-power hardware implementation of the powerful PCA algorithm. This paper presents a direct comparison of the JPL's SPCA versus JPEG2000, incorporating the Huffman and arithmetic coding for completeness of the data compression operation. The simulation results show that JPL's SPCA algorithm is superior as an optimal data-dependent-transform over the state of the art JPEG2000. When implemented in hardware, this technique is projected to be ideally suited to future NASA missions for autonomous on-board image data processing to improve the bandwidth of communication.

Duong, Tuan A.↗

NASA NTRS1972

Compression and R-wave detection of ECG/VCG data

Application of information theory to eliminate redundant part of electrocardiogram or vectorcardiogram is described. Operation of medical equipment to obtain three dimensional study of patient is discussed. Use of fast Fourier transform to accomplish data compression is explained.

Hayden, W. L.↗

NASA NTRS1989

Image processing using Gallium Arsenide (GaAs) technology

The need to increase the information return from space-borne imaging systems has increased in the past decade. The use of multi-spectral data has resulted in the need for finer spatial resolution and greater spectral coverage. Onboard signal processing will be necessary in order to utilize the available Tracking and Data Relay Satellite System (TDRSS) communication channel at high efficiency. A generally recognized approach to the increased efficiency of channel usage is through data compression techniques. The compression technique implemented is a differential pulse code modulation (DPCM) scheme with a non-uniform quantizer. The need to advance the state-of-the-art of onboard processing was recognized and a GaAs integrated circuit technology was chosen. An Adaptive Programmable Processor (APP) chip set was developed which is based on an 8-bit slice general processor. The reason for choosing the compression technique for the Multi-spectral Linear Array (MLA) instrument is described. Also a description is given of the GaAs integrated circuit chip set which will demonstrate that data compression can be performed onboard in real time at data rate in the order of 500 Mb/s.

Miller, Warner H.↗

NASA NTRS1993

Compression of spectral meteorological imagery

Data compression is essential to current low-earth-orbit spectral sensors with global coverage, e.g., meteorological sensors. Such sensors routinely produce in excess of 30 Gb of data per orbit (over 4 Mb/s for about 110 min) while typically limited to less than 10 Gb of downlink capacity per orbit (15 minutes at 10 Mb/s). Astro-Space Division develops spaceborne compression systems for compression ratios from as little as three to as much as twenty-to-one for high-fidelity reconstructions. Current hardware production and development at Astro-Space Division focuses on discrete cosine transform (DCT) systems implemented with the GE PFFT chip, a 32x32 2D-DCT engine. Spectral relations in the data are exploited through block mean extraction followed by orthonormal transformation. The transformation produces blocks with spatial correlation that are suitable for further compression with any block-oriented spatial compression system, e.g., Astro-Space Division's Laplacian modeler and analytic encoder of DCT coefficients.

Miettinen, Kristo↗

NASA NTRS2003

Evaluation of Algorithms for Compressing Hyperspectral Data

With EO-1 Hyperion in orbit NASA is showing their continued commitment to hyperspectral imaging (HSI). As HSI sensor technology continues to mature, the ever-increasing amounts of sensor data generated will result in a need for more cost effective communication and data handling systems. Lockheed Martin, with considerable experience in spacecraft design and developing special purpose onboard processors, has teamed with Applied Signal & Image Technology (ASIT), who has an extensive heritage in HSI spectral compression and Mapping Science (MSI) for JPEG 2000 spatial compression expertise, to develop a real-time and intelligent onboard processing (OBP) system to reduce HSI sensor downlink requirements. Our goal is to reduce the downlink requirement by a factor > 100, while retaining the necessary spectral and spatial fidelity of the sensor data needed to satisfy the many science, military, and intelligence goals of these systems. Our compression algorithms leverage commercial-off-the-shelf (COTS) spectral and spatial exploitation algorithms. We are currently in the process of evaluating these compression algorithms using statistical analysis and NASA scientists. We are also developing special purpose processors for executing these algorithms onboard a spacecraft.

Cook, Sid↗

NASA NTRS1994

Improved Locally Adaptive Vector Quantization

Several refinements introduced to improve performance of data-compression scheme described in "Adaptive Vector-Quantization Scheme" (NPO-18186). Principal advantages of LAVQ are that complexity and coding time less than those of some other data-compression schemes. Also, does not require priori knowledge of either codebook or statistics of source data.

Cheung, Kar-Ming↗

NASA NTRS1978

Potential end-to-end imaging information rate advantages of various alternative communication systems

Various communication systems were considered which are required to transmit both imaging and a typically error sensitive, class of data called general science/engineering (gse) over a Gaussian channel. The approach jointly treats the imaging and gse transmission problems, allowing comparisons of systems which include various channel coding and data compression alternatives. Actual system comparisons include an Advanced Imaging Communication System (AICS) which exhibits the rather significant potential advantages of sophisticated data compression coupled with powerful yet practical channel coding.

Rice, R. F.↗

DOE OSTI2025

Variational autoencoders for at-source data reduction and anomaly detection in high energy particle detectors

Detectors in next-generation high-energy physics experiments face several daunting requirements, such as high data rates, damaging radiation exposure, and stringent constraints on power, space, and latency. To address these challenges, machine learning in readout electronics can be leveraged for smart detector designs, enabling intelligent inference and data reduction at-source. Variational autoencoders (VAEs) offer a variety of benefits for front-end readout; an on-sensor encoder can perform efficient lossy data compression while simultaneously providing a latent space representation that can be used for anomaly detection. Results are presented from low-latency and resource-efficient VAEs for front-end data processing in a futuristic silicon pixel detector. Encoder-based data compression is found to preserve good performance of off-detector analysis while significantly reducing the off-detector data rate as compared to a similarly sized data filtering approach. Furthermore, the latent space information is found to be a useful discriminator in the context of real-time sensor defect monitoring. Together, these results highlight the multifaceted utility of autoencoder-based front-end readout schemes and motivate their consideration in future detector designs.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Data-Driven Compression of Electron-Phonon Interactions

First-principles calculations of electron interactions in materials have seen rapid progress in recent years, with electron-phonon ( e − ph ) interactions being a prime example. However, these techniques use large matrices encoding the interactions on dense momentum grids, which reduces computational efficiency and obscures interpretability. For e − ph interactions, existing interpolation techniques leverage locality in real space, but the high dimensionality of the data remains a bottleneck to balance cost and accuracy. Here we show an efficient way to compress e − ph interactions based on singular value decomposition (SVD), a widely used matrix and image compression technique. Leveraging (un)constrained SVD methods, we accurately predict material properties related to e − ph interactions—including charge mobility, spin relaxation times, band renormalization, and superconducting critical temperature—while using only a small fraction (1%–2%) of the interaction data. These findings unveil the hidden low-dimensional nature of e − ph interactions. Furthermore, they accelerate state-of-the-art first-principles e − ph calculations by about 2 orders of magnitude without sacrificing accuracy. Our Pareto-optimal parametrization of e − ph interactions can be readily generalized to electron-electron and electron-defect interactions, as well as to other couplings, advancing quantitative studies of condensed matter. Published by the American Physical Society 2024

Physics↗

NASA NTRS1990

Design of QMF (Quadrature Mirror Filter) in spatial domain and edge encoding

Simoncelli and Adelson have extended the one dimensional Quadrature Mirror Filter (QMF) to two dimensions with hexagon symmetry and three dimensional spatio-temporal extensions with rhombic-duodecahedray symmetry. Jain and Crochiere presented an excellent QMF design technique in the time domain. It is proposed to extend the design of a two dimensional QMF over a rectangular lattice in the spatial domain based primarily on the extension of the idea of Jain and Crochiere. In addition, the design will investigate the use of two dimensional Z-transformations. Since this proposed QMF is intended for the applications in image processing, all the important and interesting engineering issues will be addressed throughout the development phase. The design of a two dimensional QMF is discussed. The motivation is to achieve an extremely high data compression ratio. It is entirely possible to achieve dramatic results when pattern recognition techniques are employed. The final goal is the demonstration of extremely high data compression ratios using NASA pictures.

Wang, Paul P.↗

NASA NTRS1998

Pre-coding method and apparatus for multiple source or time-shifted single source data and corresponding inverse post-decoding method and apparatus

A pre-coding method and device for improving data compression performance by removing correlation between a first original data set and a second original data set, each having M members, respectively. The pre-coding method produces a compression-efficiency-enhancing double-difference data set. The method and device produce a double-difference data set, i.e., an adjacent-delta calculation performed on a cross-delta data set or a cross-delta calculation performed on two adjacent-delta data sets, from either one of (1) two adjacent spectral bands coming from two discrete sources, respectively, or (2) two time-shifted data sets coming from a single source. The resulting double-difference data set is then coded using either a distortionless data encoding scheme (entropy encoding) or a lossy data compression scheme. Also, a post-decoding method and device for recovering a second original data set having been represented by such a double-difference data set.

Yeh, Pen-Shu↗

NASA NTRS1997

Pre-coding method and apparatus for multiple source or time-shifted single source data and corresponding inverse post-decoding method and apparatus

Yeh, Pen-Shu↗

NASA NTRS1995

Compression research on the REINAS Project

We present approaches to integrating data compression technology into a database system designed to support research of air, sea, and land phenomena of interest to meteorology, oceanography, and earth science. A key element of the Real-Time Environmental Information Network and Analysis System (REINAS) system is the real-time component: to provide data as soon as acquired. Compression approaches being considered for REINAS include compression of raw data on the way into the database, compression of data produced by scientific visualization on the way out of the database, compression of modeling results, and compression of database query results. These compression needs are being incorporated through client-server, API, utility, and application code development.

Rosen, Eric↗

NASA NTRS2014

Organizing Compression of Hyperspectral Imagery to Allow Efficient Parallel Decompression

family of schemes has been devised for organizing the output of an algorithm for predictive data compression of hyperspectral imagery so as to allow efficient parallelization in both the compressor and decompressor. In these schemes, the compressor performs a number of iterations, during each of which a portion of the data is compressed via parallel threads operating on independent portions of the data. The general idea is that for each iteration it is predetermined how much compressed data will be produced from each thread.

Klimesh, Matthew A.↗

NASA NTRS2001

Determining Biosignatures by Complexity Analysis in Antarctic Cryptoendolithic Communities

One of the most difficult problems of life detection is that of identifying biosignatures across a wide range of scales using multiple co-registered probes. The technique should be of equal utility across a wide range of search spaces from remote sensors probing volumes of space or planetary surfaces, visual eye or camera searches across the surface of a rock in Antarctica, low resolution microscopic scanning of a rock or a space craft in situ, or high resolution electron microscope and computerized tomography scanning of geobiological samples. We describe here an approach to this problem which derives in large part from past work done in the area of astrophysics - namely the analysis of complexity in galactic signals by data compression methods. This approach is a radically new one for geobiology and astrobiology, and allows us to assess the complexity (and thus potential biogenicity) of an object being examined. This is done by considering the information within pixels of an image (regardless the sensor used to gather the information) as an energetic system capable of description in terms of classical thermodynamics. The image data space is searched by an algorithm that judges complexity via data compression (e.g., the more compressible it is, the less complex, and vice versa) and maximum entropy as originally outlined by Shannon. At present we are implementing methods to utilize images from multiple sensors gathering different kinds of information (e.g., visible gray-scale data, color analyses, UV fluorescence, chemical information, etc). We present here preliminary data from deep UV fluorescence and ESEM (Environmental Scanning Electron Microscope) images from a layered cryptoendolithic community of an Antarctic rock.

Storrie-Lombardi, M. C.↗