SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Lossy compression of statistical data using quantum annealer

Abstract We present a new lossy compression algorithm for statistical floating-point data through a representation learning with binary variables. The algorithm finds a set of basis vectors and their binary coefficients that precisely reconstruct the original data. The optimization for the basis vectors is performed classically, while binary coefficients are retrieved through both simulated and quantum annealing for comparison. A bias correction procedure is also presented to estimate and eliminate the error and bias introduced from the inexact reconstruction of the lossy compression for statistical data analyses. The compression algorithm is demonstrated on two different datasets of lattice quantum chromodynamics simulations. The results obtained using simulated annealing show 3–3.5 times better compression performance than the algorithm based on neural-network autoencoder. Calculations using quantum annealing also show promising results, but performance is limited by the integrated control error of the quantum processing unit, which yields large uncertainties in the biases and coupling parameters. Hardware comparison is further studied between the previous generation D-Wave 2000Q and the current D-Wave Advantage system. Our study shows that the Advantage system is more likely to obtain low-energy solutions for the problems than the 2000Q.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

First test results of the HGCAL concentrator ASICs: ECON-T and ECON-D

With over 6 million channels, the High Granularity Calorimeter for the CMS HL-LHC upgrade presents a unique data transmission challenge. The ECON ASICs provide a critical stage of on-detector data compression and selection for the trigger path (ECON-T) and data acquisition path (ECON-D) of the HGCAL. The ASICs, fabricated in 65 nm CMOS, are radiation tolerant up to 200 Mrad and require low power consumption: < 2.5 mW/sensor-channel per chip. Here, we report on the first functionality and radiation tests for the ECON-D-P1 full-functionality prototype. We present a comparison of single event effect (SEE) cross sections measured for different methods of triple modular redundancy using test results from the ECON-T-P1 full-functionality prototype.

47 OTHER INSTRUMENTATION↗

NASA NTRS1975

On-board data management study for EOPAP

The requirements, implementation techniques, and mission analysis associated with on-board data management for EOPAP were studied. SEASAT-A was used as a baseline, and the storage requirements, data rates, and information extraction requirements were investigated for each of the following proposed SEASAT sensors: a short pulse 13.9 GHz radar, a long pulse 13.9 GHz radar, a synthetic aperture radar, a multispectral passive microwave radiometer facility, and an infrared/visible very high resolution radiometer (VHRR). Rate distortion theory was applied to determine theoretical minimum data rates and compared with the rates required by practical techniques. It was concluded that practical techniques can be used which approach the theoretically optimum based upon an empirically determined source random process model. The results of the preceding investigations were used to recommend an on-board data management system for (1) data compression through information extraction, optimal noiseless coding, source coding with distortion, data buffering, and data selection under command or as a function of data activity, (2) for command handling, (3) for spacecraft operation and control, and (4) for experiment operation and monitoring.

Davisson, L. D.↗

NASA NTRS2012

Algorithm for Compressing Time-Series Data

An algorithm based on Chebyshev polynomials effects lossy compression of time-series data or other one-dimensional data streams (e.g., spectral data) that are arranged in blocks for sequential transmission. The algorithm was developed for use in transmitting data from spacecraft scientific instruments to Earth stations. In spite of its lossy nature, the algorithm preserves the information needed for scientific analysis. The algorithm is computationally simple, yet compresses data streams by factors much greater than two. The algorithm is not restricted to spacecraft or scientific uses: it is applicable to time-series data in general. The algorithm can also be applied to general multidimensional data that have been converted to time-series data, a typical example being image data acquired by raster scanning. However, unlike most prior image-data-compression algorithms, this algorithm neither depends on nor exploits the two-dimensional spatial correlations that are generally present in images. In order to understand the essence of this compression algorithm, it is necessary to understand that the net effect of this algorithm and the associated decompression algorithm is to approximate the original stream of data as a sequence of finite series of Chebyshev polynomials. For the purpose of this algorithm, a block of data or interval of time for which a Chebyshev polynomial series is fitted to the original data is denoted a fitting interval. Chebyshev approximation has two properties that make it particularly effective for compressing serial data streams with minimal loss of scientific information: The errors associated with a Chebyshev approximation are nearly uniformly distributed over the fitting interval (this is known in the art as the "equal error property"); and the maximum deviations of the fitted Chebyshev polynomial from the original data have the smallest possible values (this is known in the art as the "min-max property").

Hawkins, S. Edward, III↗

DOE OSTI2020

The impact of the fiducial cosmology assumption on BAO distance scale measurements

ABSTRACT Standard analysis pipelines for measurements of Baryon Acoustic Oscillations (BAO) in galaxy surveys make use of a fiducial cosmological model to guide the data compression required to transform from observed redshifts and angles to the measured angular and radial BAO peak positions. In order to remove any dependence on the fiducial cosmology from the results, all models compared to the data should mimic the compression and its dependence on the fiducial model. In practice, approximations are made when testing models: (1) There is assumed to be no residual dependence on the fiducial cosmology after reconstruction, (2) differences in the distance–redshift relationship are assumed to match a linear scaling, and (3) differences in clustering between true and fiducial models are assumed to be removed by the free parameters used to null the non-BAO signal. We test these approximations using the current standard measurement procedure with a set of halo catalogues from the aemulus suite of N-body simulations, which span a range of wCDM cosmological models. We focus on reconstruction of the primordial BAO and locating the BAO. For the range of wCDM cosmologies covered by the aemulus suite, we find no evidence for systematic errors in the measured BAO shift parameters α∥ and α⊥ to $\lt 0.1\%$. However, the measured errors $\sigma _{\alpha _{\parallel }}$ and $\sigma _{\alpha _{\bot }}$ show a notable absolute increase by up to +0.001 and +0.002, respectively, in the case that the fiducial cosmology does not match the truth. These effects on the inferred BAO scale will be important, given the precision of measurements expected from future surveys including DESI, Euclid, and WFIRST.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

AEflow (Autoencoder fluid flow compression network) [SWR-22-29]

As the size of turbulent flow simulations continues to grow, in situ data compression is becoming increasingly important for visualization, analysis, and restart checkpointing. For these applications, single-pass compression techniques with low computational and communication overhead are crucial. In this paper we present a deep-learning approach to in situ compression using an autoencoder architecture that is customized for three-dimensional turbulent flows and is well suited for contemporary heterogeneous computing resources. The autoencoder is compared against a recently introduced randomized single-pass singular value decomposition (SVD) for three different canonical turbulent flows: decaying homogeneous isotropic turbulence, a Taylor-Green vortex, and turbulent channel flow. Our proposed fully convolutional autoencoder architecture compresses turbulent flow snapshots by a factor of 64 with a single pass, allows for arbitrarily sized input fields, is cheaper to compute than the randomized single-pass SVD for typical simulation sizes, performs well on unseen flow configurations, and has been made publicly available. The results reported here show that the autoencoder dramatically outperforms a randomized single-pass SVD with similar compression ratio and yields comparable performance to a higher-rank decomposition with an order of magnitude less compression in regard to preserving a number of important statistical quantities such as turbulent kinetic energy, enstrophy, and Reynolds stresses.

King, Ryan↗

NASA NTRS1991

Advanced techniques and technology for efficient data storage, access, and transfer

Advanced techniques for efficiently representing most forms of data are being implemented in practical hardware and software form through the joint efforts of three NASA centers. These techniques adapt to local statistical variations to continually provide near optimum code efficiency when representing data without error. Demonstrated in several earlier space applications, these techniques are the basis of initial NASA data compression standards specifications. Since the techniques clearly apply to most NASA science data, NASA invested in the development of both hardware and software implementations for general use. This investment includes high-speed single-chip very large scale integration (VLSI) coding and decoding modules as well as machine-transferrable software routines. The hardware chips were tested in the laboratory at data rates as high as 700 Mbits/s. A coding module's definition includes a predictive preprocessing stage and a powerful adaptive coding stage. The function of the preprocessor is to optimally process incoming data into a standard form data source that the second stage can handle.The built-in preprocessor of the VLSI coder chips is ideal for high-speed sampled data applications such as imaging and high-quality audio, but additionally, the second stage adaptive coder can be used separately with any source that can be externally preprocessed into the 'standard form'. This generic functionality assures that the applicability of these techniques and their recent high-speed implementations should be equally broad outside of NASA.

Rice, Robert F.↗

NASA NTRS1992

Use of AVIRIS data to the definition of optimised specifications for land applications with future spaceborne imaging spectrometers

Recent experience with airborne imaging spectrometers demonstrated the advantages of narrow band sensors over broad band scanners for characterizing the nature, extent, and physical status of typical land surfaces. Information on key spectral features associated with various land surfaces can be obtained from the data of such instruments, which can be used to simulate spaceborne imaging spectrometer data and to assess their information content if comprehensive underpinning is provided by ground data. The collection of such information was an issue of airborne imaging spectrometer campaigns like the NASA MAC-Europe 1991. Airborne and ground data obtained from different test sites in Europe are utilized for a comparative analysis of the spectral signatures of various land surfaces (vegetation, bare soils and rocks, and mixed soil/rock-vegetation) as seen from different imaging spectrometers like Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), GERIS 63 band scanner, and CASI. The following items are discussed: (1) the significance of different spectral regions within the wavelength interval between 0.4 m and 2.5 m for the differentiation of different land units; (2) recommendations on the optimum band selection and band-widths to be used for the application of future satellite-based imaging spectrometers for land applications; (3) the boundaries for the detection of plant features in mixed-soil plant spectra and the influence of different soil properties on the mixture of the spectra; (4) recommendations on the optimum spatial resolution and recording dates for the discrimination of spectral features of various surface types; and (5) evaluation of different data compressing techniques for the optimum extraction of spectral information from imaging spectrometry data.

Bodechtel, J.↗

NASA NTRS1993

LVQ and backpropagation neural networks applied to NASA SSME data

Feedfoward neural networks with backpropagation learning have been used as function approximators for modeling the space shuttle main engine (SSME) sensor signals. The modeling of these sensor signals is aimed at the development of a sensor fault detection system that can be used during ground test firings. The generalization capability of a neural network based function approximator depends on the training vectors which in this application may be derived from a number of SSME ground test-firings. This yields a large number of training vectors. Large training sets can cause the time required to train the network to be very large. Also, the network may not be able to generalize for large training sets. To reduce the size of the training sets, the SSME test-firing data is reduced using the learning vector quantization (LVQ) based technique. Different compression ratios were used to obtain compressed data in training the neural network model. The performance of the neural model trained using reduced sets of training patterns is presented and compared with the performance of the model trained using complete data. The LVQ can also be used as a function approximator. The performance of the LVQ as a function approximator using reduced training sets is presented and compared with the performance of the backpropagation network.

Doniere, Timothy F.↗

NASA NTRS2009

Optimal Compression Methods for Floating-point Format Images

We report on the results of a comparison study of different techniques for compressing FITS images that have floating-point (real*4) pixel values. Standard file compression methods like GZIP are generally ineffective in this case (with compression ratios only in the range 1.2 - 1.6), so instead we use a technique of converting the floating-point values into quantized scaled integers which are compressed using the Rice algorithm. The compressed data stream is stored in FITS format using the tiled-image compression convention. This is technically a lossy compression method, since the pixel values are not exactly reproduced, however all the significant photometric and astrometric information content of the image can be preserved while still achieving file compression ratios in the range of 4 to 8. We also show that introducing dithering, or randomization, when assigning the quantized pixel-values can significantly improve the photometric and astrometric precision in the stellar images in the compressed file without adding additional noise. We quantify our results by comparing the stellar magnitudes and positions as measured in the original uncompressed image to those derived from the same image after applying successively greater amounts of compression.

Pence, W. D.↗

DOE OSTI2024

Kernelized approaches to streaming compression of scientific data

In this paper three algorithms are developed for the streaming compression of scientific data. The algorithms presented are reliant on the theory of vector-valued reproducing kernel Hilbert spaces and operator valued kernel. Further, the scientific data is modeled as a snapshot of time dependent vector field F(x, t) over a manifold M and the recovery of the data is framed as a learning problem. These processes are then appropriately modified and ana lyzed for the streaming scenario in which data is generated without the ability to revisit past entries.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1996

VLSI Neural Networks Help To Compress Video Signals

Advanced analog/digital electronic system for compression of video signals incorporates artificial neural networks. Performs motion-estimation and image-data-compression processing. Effectively eliminates temporal and spatial redundancies of sequences of video images; processes video image data, retaining only nonredundant parts to be transmitted, then transmits resulting data stream in form of efficient code. Reduces bandwidth and storage requirements for transmission and recording of video signal.

Fang, Wai-Chi↗

NASA NTRS1977

Cluster compression algorithm: A joint clustering/data compression concept

The Cluster Compression Algorithm (CCA), which was developed to reduce costs associated with transmitting, storing, distributing, and interpreting LANDSAT multispectral image data is described. The CCA is a preprocessing algorithm that uses feature extraction and data compression to more efficiently represent the information in the image data. The format of the preprocessed data enables simply a look-up table decoding and direct use of the extracted features to reduce user computation for either image reconstruction, or computer interpretation of the image data. Basically, the CCA uses spatially local clustering to extract features from the image data to describe spectral characteristics of the data set. In addition, the features may be used to form a sequence of scalar numbers that define each picture element in terms of the cluster features. This sequence, called the feature map, is then efficiently represented by using source encoding concepts. Various forms of the CCA are defined and experimental results are presented to show trade-offs and characteristics of the various implementations. Examples are provided that demonstrate the application of the cluster compression concept to multi-spectral images from LANDSAT and other sources.

Hilbert, E. E.↗

DOE OSTI2023

Ti-6Al-4V to over 1.2 TPa: Shock Hugoniot experiments, ab initio calculations, and a broad-range multiphase equation of state

Titanium alloys are used in a large array of applications. In this work we focus our attention on the most used alloy, Ti-6Al-4V (Ti64), which has excellent mechanical and biocompatibility properties with applications in aerospace, defense, biomedical, and other fields. Here we present high-fidelity experimental shock compression data measured on Sandia’s Z machine. We extend the principal shock Hugoniot for Ti64 to more than threefold compression, up to over 1.2 TPa. We use the data to validate our ab initio molecular dynamics simulations and to develop a highly reliable, multiphase equation of state (EOS) for Ti64, spanning a broad range of temperature and pressures. The first-principles simulations show very good agreement with Z data and with previous three-stage gas gun data from Sandia’s STAR facility. The resulting principal Hugoniot and the broad-range EOS and phase diagram up to 10 TPa and 10 5 K are suitable for use in shock experiments and in hydrodynamic simulations. The high-precision experimental results and high-fidelity simulations demonstrate that the Hugoniot of the Ti64 alloy is stiffer than that of pure Ti and reveal that Ti64 melts on the Hugoniot at a significantly lower pressure and temperature than previously modeled.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

NASA NTRSDate not supplied

Compressive Sensing Based Data Acquisition Architecture for Transient Stellar Events in Crowded Star Fields

Compressive sensing is a mathematical technique for simultaneous data acquisition and compression. In this work, we show a CS based architecture for acquiring and reconstructing transient stellar events. This architecture reconstructs a differenced image itself, eliminating the need for any sparse domain transforms, otherwise required for traditional CS reconstruction. The resulting reconstructed differenced image is of critical importance as the information required for generating a time-series photometric light curve is obtained only from a differenced image. Hence, reconstructing a crowded star spatial image, followed by differencing is wasteful. This architecture eliminates the need to 1.) transform an image to a sparse domain, 2.) Reconstruct a dense field, and then apply differencing on the image to obtain the image of critical value. We study the case of microlensing to depict a star source experiencing magnification in time. Our results show that this architecture is able to reconstruct star source magnitudes with magnification factors greater than 1 for clean images with error less than 2% using only 10% of the Nyquist rate samples.

Asmita Korde-Patel↗

NASA NTRSDate not supplied

Compressive Sensing Based Data Acquisition Architecture for Transient Stellar Events in Crowded Star Fields

Compressive sensing is a mathematical technique for simultaneous data acquisition and compression. In this work, we show a CS based architecture for acquiring and reconstructing transient stellar events. This architecture recon-structs a differenced image itself, eliminating the need for any sparse domain transforms, otherwise required for traditional CS reconstruction. The resulting reconstructed differenced image is of critical importance as the information required for generating a time-series photometric light curve is obtained only from a differenced image. Hence, reconstructing a crowded star spatial image, followed by differencing is wasteful. This architecture eliminates the need to 1.) transform an image to a sparse domain, 2.) Reconstruct a dense field, and then apply differencing on the image to obtain the image of critical value. We study the case of microlensing to depict a star source experiencing magnification in time. Our results show that this architecture is able to reconstruct star source magnitudes with magnification factors greater than 1 for clean images with error less than 2% using only 10% of the Nyquist rate samples.

Asmita Korde-patel↗

NASA NTRS1985

Evaluation of the Radiometric Quality of the TM Data Using Clustering, Linear Transformations and Multispectral Distance Measures

The radiometric quality of LANDSAT 4 TM data for the classification and identification of Earth surface features was evaluated. Techniques employed in the evaluation included clustering, data compression (linear transformations), multispectral distance measures, and hierarchical classification methods. TM and MSS data for the Chicago, Illinois test site were studied. In order to determine the radiometric quality of the TM thermal data for temperature mapping of surface water, a test site was selected within the area covered by the TM scene (Scene ID: 40101-16025) gathered over Illinois. This site was chosen because it includes a surface water body with a large range of temperatures, i.e., a cooling pond for the Dresden nuclear power plant and the junction of two rivers.

Bartolucci, L. A.↗

NASA NTRS1967

Conference on Adaptive Telemetry

Adaptive telemetry conference on adaptive data control and processing, prediction, data compression, encoding, decoding, and modulation.

TELEMETRY↗