SEARCH · Search NASA
Results for “data compaction and compression”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
An algorithm for computing the number of distinct spectral vectors in thematic mapper data
A computationally efficient method was developed to compute the number of distinct spectral vectors and their frequency of occurrence in Landsat-4 Thematic Mapper (TM) data. The algorithm first partitions the image into spectrally disjoint subsets and then computes the frequency distribution of distinct spectral vectors within each subset from a multidimensional histogram. The overall frequency distribution is tabulated by accumulating the results from each subset. The number of distinct spectral vectors could be used as a measure of potential storage compaction of alternate data representations for data compression, or as a measure of information content in the comparison of spectral band combinations and/or spatial resolutions for an image. Results from processing three 512 x 512 pixel Landsat-4 TM images and one Landsat-4 Multispectral Scanner (MSS) image are presented as examples. An algorithm for computing the frequency distribution of distinct spectral vectors in MSS data is given in the Appendix.
NASA Tech Briefs, July 2008
Topics covered include: Torque Sensor Based on Tunnel-Diode Oscillator; Shaft-Angle Sensor Based on Tunnel-Diode Oscillator; Ground Facility for Vicarious Calibration of Skyborne Sensors; Optical Pressure-Temperature Sensor for a Combustion Chamber; Impact-Locator Sensor Panels; Low-Loss Waveguides for Terahertz Frequencies; MEMS/ECD Method for Making Bi(2-x)Sb(x)Te3 Thermoelectric Devices; Low-Temperature Supercapacitors; Making a Back-Illuminated Imager with Back-Side Contact and Alignment Markers; Compact, Single-Stage MMIC InP HEMT Amplifier; Nb(x)Ti(1-x)N Superconducting-Nanowire Single-Photon Detectors; Improved Sand-Compaction Method for Lost-Foam Metal Casting; Improved Probe for Evaluating Compaction of Mold Sand; Polymer-Based Composite Catholytes for Li Thin-Film Cells; Using ALD To Bond CNTs to Substrates and Matrices; Alternating-Composition Layered Ceramic Barrier Coatings; Variable-Structure Control of a Model Glider Airplane; Axial Halbach Magnetic Bearings; Compact, Non-Pneumatic Rock-Powder Samplers; Biochips Containing Arrays of Carbon-Nanotube Electrodes; Nb(x)Ti(1-x)N Superconducting-Nanowire Single-Photon Detectors; Neon as a Buffer Gas for a Mercury-Ion Clock; Miniature Incandescent Lamps as Fiber-Optic Light Sources; Bidirectional Pressure-Regulator System; and Prism Window for Optical Alignment. Single-Grid-Pair Fourier Telescope for Imaging in Hard-X Rays and gamma Rays Range-Gated Metrology with Compact Optical Head Lossless, Multi-Spectral Data Compressor for Improved Compression for Pushbroom-Typetruments.
Communications and information research: Improved space link performance via concatenated forward error correction coding
With the development of new advanced instruments for remote sensing applications, sensor data will be generated at a rate that not only requires increased onboard processing and storage capability, but imposes demands on the space to ground communication link and ground data management-communication system. Data compression and error control codes provide viable means to alleviate these demands. Two types of data compression have been studied by many researchers in the area of information theory: a lossless technique that guarantees full reconstruction of the data, and a lossy technique which generally gives higher data compaction ratio but incurs some distortion in the reconstructed data. To satisfy the many science disciplines which NASA supports, lossless data compression becomes a primary focus for the technology development. While transmitting the data obtained by any lossless data compression, it is very important to use some error-control code. For a long time, convolutional codes have been widely used in satellite telecommunications. To more efficiently transform the data obtained by the Rice algorithm, it is required to meet the a posteriori probability (APP) for each decoded bit. A relevant algorithm for this purpose has been proposed which minimizes the bit error probability in the decoding linear block and convolutional codes and meets the APP for each decoded bit. However, recent results on iterative decoding of 'Turbo codes', turn conventional wisdom on its head and suggest fundamentally new techniques. During the past several months of this research, the following approaches have been developed: (1) a new lossless data compression algorithm, which is much better than the extended Rice algorithm for various types of sensor data, (2) a new approach to determine the generalized Hamming weights of the algebraic-geometric codes defined by a large class of curves in high-dimensional spaces, (3) some efficient improved geometric Goppa codes for disk memory systems and high-speed mass memory systems, and (4) a tree based approach for data compression using dynamic programming.
Experimental Studies on a Compact Storage Scheme for Wavelet-based Multiresolution Subregion Retrieval
Wavelet transforms, when combined with quantization and a suitable encoding, can be used to compress images effectively. In order to use them for image library systems, a compact storage scheme for quantized coefficient wavelet data must be developed with a support for fast subregion retrieval. We have designed such a scheme and in this paper we provide experimental studies to demonstrate that it achieves good image compression ratios, while providing a natural indexing mechanism that facilitates fast retrieval of portions of the image at various resolutions.
Sequential Principal Component Analysis -An Optimal and Hardware-Implementable Transform for Image Compression
This paper presents the JPL-developed Sequential Principal Component Analysis (SPCA) algorithm for feature extraction / image compression, based on "dominant-term selection" unsupervised learning technique that requires an order-of-magnitude lesser computation and has simpler architecture compared to the state of the art gradient-descent techniques. This algorithm is inherently amenable to a compact, low power and high speed VLSI hardware embodiment. The paper compares the lossless image compression performance of the JPL's SPCA algorithm with the state of the art JPEG2000, widely used due to its simplified hardware implementability. JPEG2000 is not an optimal data compression technique because of its fixed transform characteristics, regardless of its data structure. On the other hand, conventional Principal Component Analysis based transform (PCA-transform) is a data-dependent-structure transform. However, it is not easy to implement the PCA in compact VLSI hardware, due to its highly computational and architectural complexity. In contrast, the JPL's "dominant-term selection" SPCA algorithm allows, for the first time, a compact, low-power hardware implementation of the powerful PCA algorithm. This paper presents a direct comparison of the JPL's SPCA versus JPEG2000, incorporating the Huffman and arithmetic coding for completeness of the data compression operation. The simulation results show that JPL's SPCA algorithm is superior as an optimal data-dependent-transform over the state of the art JPEG2000. When implemented in hardware, this technique is projected to be ideally suited to future NASA missions for autonomous on-board image data processing to improve the bandwidth of communication.
Compaction by impact of unconsolidated lunar fines
New Hugoniot and release adiabat data for 1.8 g/cu cm lunar fines in the approximately 2 to 70 kbar range demonstrate that upon shock compression intrinsic crystal density (approximately 3.1 g/cu cm) is achieved under shock stress of 15 to 20 kbar. Release adiabat determinations indicate that measurable irreversible compaction occurs upon achieving shock pressures above approximately 4 kbar. For shocks in the approximately 7 to 15 kbar range, the inferred post-shock specific volumes observed decrease nearly linearly with increasing peak shock pressures. Upon shocking to approximately 15 kbar the post-shock density is approximately that of the intrinsic minerals. If the present data are taken to be representative of the response to impact of unconsolidated regolith material on the moon, it is inferred that the formation of appreciable quantities of soil breccia can be associated with the impact of meteoroids or ejecta at speeds as low as approximately 1 km/sec.
Image Segmentation, Registration, Compression, and Matching
A novel computational framework was developed of a 2D affine invariant matching exploiting a parameter space. Named as affine invariant parameter space (AIPS), the technique can be applied to many image-processing and computer-vision problems, including image registration, template matching, and object tracking from image sequence. The AIPS is formed by the parameters in an affine combination of a set of feature points in the image plane. In cases where the entire image can be assumed to have undergone a single affine transformation, the new AIPS match metric and matching framework becomes very effective (compared with the state-of-the-art methods at the time of this reporting). No knowledge about scaling or any other transformation parameters need to be known a priori to apply the AIPS framework. An automated suite of software tools has been created to provide accurate image segmentation (for data cleaning) and high-quality 2D image and 3D surface registration (for fusing multi-resolution terrain, image, and map data). These tools are capable of supporting existing GIS toolkits already in the marketplace, and will also be usable in a stand-alone fashion. The toolkit applies novel algorithmic approaches for image segmentation, feature extraction, and registration of 2D imagery and 3D surface data, which supports first-pass, batched, fully automatic feature extraction (for segmentation), and registration. A hierarchical and adaptive approach is taken for achieving automatic feature extraction, segmentation, and registration. Surface registration is the process of aligning two (or more) data sets to a common coordinate system, during which the transformation between their different coordinate systems is determined. Also developed here are a novel, volumetric surface modeling and compression technique that provide both quality-guaranteed mesh surface approximations and compaction of the model sizes by efficiently coding the geometry and connectivity/topology components of the generated models. The highly efficient triangular mesh compression compacts the connectivity information at the rate of 1.5-4 bits per vertex (on average for triangle meshes), while reducing the 3D geometry by 40-50 percent. Finally, taking into consideration the characteristics of 3D terrain data, and using the innovative, regularized binary decomposition mesh modeling, a multistage, pattern-drive modeling, and compression technique has been developed to provide an effective framework for compressing digital elevation model (DEM) surfaces, high-resolution aerial imagery, and other types of NASA data.
Compression of Solar Spectroscopic Observations: a Case Study of MgII k Spectral Line Profiles Observed by NASA’s IRIS Satellite
In this study we extract the deep features and investigate the compression of the MgII k spectral line profiles observed in quiet Sun regions by NASA’s IRIS satellite. The data set of line profiles used for the analysis was obtained on April 20th, 2020, at the center of the solar disc, and contains almost 300,000 individual MgII k line profiles after data cleaning. The data are separated into train and test subsets. The train subset was used to train the autoencoder of the varying embedding layer size. The early stopping criterion was implemented on the test subset to prevent the model from overfitting. Our results indicate that it is possible to compress the spectral line profiles more than 27 times (which corresponds to the reduction of the data dimensionality from 110 to 4) while having a 4DN average reconstruction error, which is comparable to the variations in the line continuum. The mean squared error and the reconstruction error of even statistical moments sharply decrease when the dimensionality of the embedding layer increases from 1 to 4 and almost stop decreasing for higher numbers. The observed occasional improvements in training for values higher than 4 indicate that a better compact embedding may potentially be obtained if other training strategies and longer training times are used. The features learned for the critical four-dimensional case can be interpreted. In particular, three of these four features mainly control the line width, line asymmetry, and line dip formation respectively. The presented results are the first attempt to obtain a compact embedding for spectroscopic line profiles and confirm the value of this approach, in particular for feature extraction, data compression, and denoising.
Electromagnetic Scattered Field Evaluation and Data Compression Using Imaging Techniques
This is the final report on Project #727625 between The Ohio State University and NASA, Lewis Research Center, Cleveland, Ohio. Under this project, a data compression technique for scattered field data of electrically large targets is developed. The technique was applied to the scattered fields of two targets of interest. The backscattered fields of the scale models of these targets were measured in a ra compact range. For one of the targets, the backscattered fields were also calculated using XPATCH computer code. Using the technique all scattered field data sets were compressed successfully. A compression ratio of the order 40 was achieved. In this report, the technique is described briefly and some sample results are included.
Nonlinear Unsteady Aerodynamic Modeling Using Empirical Orthogonal Functions
Empirical orthogonal function modeling is explained and applied to identify compact discrete-time nonlinear unsteady aerodynamic models from data generated by an unsteady three-dimensional compressible Navier-Stokes flow solver for an airfoil undergoing various pitching motions. Model structures, model parameter estimates, and model parameter uncertainty estimates for nondimensional lift, drag, and pitching moment coefficient models were determined autonomously and directly from the data. Prediction tests using data that were not used in the modeling process showed that the identified models exhibited excellent prediction capability, which is a strong indicator of an accurate model.
NASA Tech Briefs, August 2006
Topics covered include: Measurement and Controls Data Acquisition System IMU/GPS System Provides Position and Attitude Data Using Artificial Intelligence to Inform Pilots of Weather Fast Lossless Compression of Multispectral-Image Data Developing Signal-Pattern-Recognition Programs Implementing Access to Data Distributed on Many Processors Compact, Efficient Drive Circuit for a Piezoelectric Pump; Dual Common Planes for Time Multiplexing of Dual-Color QWIPs; MMIC Power Amplifier Puts Out 40 mW From 75 to 110 GHz; 2D/3D Visual Tracker for Rover Mast; Adding Hierarchical Objects to Relational Database General-Purpose XML-Based Information Managements; Vaporizable Scaffolds for Fabricating Thermoelectric Modules; Producing Quantum Dots by Spray Pyrolysis; Mobile Robot for Exploring Cold Liquid/Solid Environments; System Would Acquire Core and Powder Samples of Rocks; Improved Fabrication of Lithium Films Having Micron Features; Manufacture of Regularly Shaped Sol-Gel Pellets; Regulating Glucose and pH, and Monitoring Oxygen in a Bioreactor; Satellite Multiangle Spectropolarimetric Imaging of Aerosols; Interferometric System for Measuring Thickness of Sea Ice; Microscale Regenerative Heat Exchanger Protocols for Handling Messages Between Simulation Computers Statistical Detection of Atypical Aircraft Flights NASA's Aviation Safety and Modeling Project Multimode-Guided-Wave Ultrasonic Scanning of Materials Algorithms for Maneuvering Spacecraft Around Small Bodies Improved Solar-Radiation-Pressure Models for GPS Satellites Measuring Attitude of a Large, Flexible, Orbiting Structure
Advances in image compression and automatic target recognition; Proceedings of the Meeting, Orlando, FL, Mar. 30, 31, 1989
Various papers on image compression and automatic target recognition are presented. Individual topics addressed include: target cluster detection in cluttered SAR imagery, model-based target recognition using laser radar imagery, Smart Sensor front-end processor for feature extraction of images, object attitude estimation and tracking from a single video sensor, symmetry detection in human vision, analysis of high resolution aerial images for object detection, obscured object recognition for an ATR application, neural networks for adaptive shape tracking, statistical mechanics and pattern recognition, detection of cylinders in aerial range images, moving object tracking using local windows, new transform method for image data compression, quad-tree product vector quantization of images, predictive trellis encoding of imagery, reduced generalized chain code for contour description, compact architecture for a real-time vision system, use of human visibility functions in segmentation coding, color texture analysis and synthesis using Gibbs random fields.
NASA Tech Briefs, February 2010
Topics covered include: Insulation-Testing Cryostat With Lifting Mechanism; Optical Testing of Retroreflectors for Cryogenic Applications; Measuring Cyclic Error in Laser Heterodyne Interferometers; Self-Referencing Hartmann Test for Large-Aperture Telescopes; Measuring a Fiber-Optic Delay Line Using a Mode-Locked Laser; Reconfigurable Hardware for Compressing Hyperspectral Image Data; Spatio-Temporal Equalizer for a Receiving-Antenna Feed Array; High-Speed Ring Bus; Nanoionics-Based Switches for Radio-Frequency Applications; Lunar Dust-Tolerant Electrical Connector; Compact, Reliable EEPROM Controller; Quad-Chip Double-Balanced Frequency Tripler; Ka-Band Waveguide Two-Way Hybrid Combiner for MMIC Amplifiers; Radiation-Hardened Solid-State Drive; Use of Nanofibers to Strengthen Hydrogels of Silica, Other Oxides, and Aerogels; Two Concepts for Deployable Trusses; Concentric Nested Toroidal Inflatable Structures; Investigating Dynamics of Eccentricity in Turbomachines; Improved Low-Temperature Performance of Li-Ion Cells Using New Electrolytes; Integrity Monitoring of Mercury Discharge Lamps; White-Light Phase-Conjugate Mirrors as Distortion Correctors; Biasable, Balanced, Fundamental Submillimeter Monolithic Membrane Mixer; ICER-3D Hyperspectral Image Compression Software; and Context Modeler for Wavelet Compression of Spectral Hyperspectral Images.
Performance of Lempel-Ziv compressors with deferred innovation
The noiseless data-compression algorithms introduced by Lempel and Ziv (LZ) parse an input data string into successive substrings each consisting of two parts: The citation, which is the longest prefix that has appeared earlier in the input, and the innovation, which is the symbol immediately following the citation. In extremal versions of the LZ algorithm the citation may have begun anywhere in the input; in incremental versions it must have begun at a previous parse position. Originally the citation and the innovation were encoded, either individually or jointly, into an output word to be transmitted or stored. Subsequently, it was speculated that the cost of this encoding may be excessively high because the innovation contributes roughly 1g(A) bits, where A is the size of the input alphabet, regardless of the compressibility of the source. To remedy this excess, it was suggested to store the parsed substring as usual, but encoding for output only the citation, leaving the innovation to be encoded as the first symbol of the next substring. Being thus included in the next substring, the innovation can participate in whatever compression that substring enjoys. This strategy is called deferred innovation. It is exemplified in the algorithm described by Welch and implemented in the C program compress that has widely displaced adaptive Huffman coding (compact) as a UNIX system utility. The excessive expansion is explained, an implicit warning is given against using the deferred innovation compressors on nearly incompressible data.
Multiresolution With Super-Compact Wavelets
The solution data computed from large scale simulations are sometimes too big for main memory, for local disks, and possibly even for a remote storage disk, creating tremendous processing time as well as technical difficulties in analyzing the data. The excessive storage demands a corresponding huge penalty in I/O time, rendering time and transmission time between different computer systems. In this paper, a multiresolution scheme is proposed to compress field simulation or experimental data without much loss of important information in the representation. Originally, the wavelet based multiresolution scheme was introduced in image processing, for the purposes of data compression and feature extraction. Unlike photographic image data which has rather simple settings, computational field simulation data needs more careful treatment in applying the multiresolution technique. While the image data sits on a regular spaced grid, the simulation data usually resides on a structured curvilinear grid or unstructured grid. In addition to the irregularity in grid spacing, the other difficulty is that the solutions consist of vectors instead of scalar values. The data characteristics demand more restrictive conditions. In general, the photographic images have very little inherent smoothness with discontinuities almost everywhere. On the other hand, the numerical solutions have smoothness almost everywhere and discontinuities in local areas (shock, vortices, and shear layers). The wavelet bases should be amenable to the solution of the problem at hand and applicable to constraints such as numerical accuracy and boundary conditions. In choosing a suitable wavelet basis for simulation data among a variety of wavelet families, the supercompact wavelets designed by Beam and Warming provide one of the most effective multiresolution schemes. Supercompact multi-wavelets retain the compactness of Haar wavelets, are piecewise polynomial and orthogonal, and can have arbitrary order of approximation. The advantages of the multiresolution algorithm are that no special treatment is required at the boundaries of the interval, and that the application to functions which are only piecewise continuous (internal boundaries) can be efficiently implemented. In this presentation, Beam's supercompact wavelets are generalized to higher dimensions using multidimensional scaling and wavelet functions rather than alternating the directions as in the 1D version. As a demonstration of actual 3D data compression, supercompact wavelet transforms are applied to a 3D data set for wing tip vortex flow solutions (2.5 million grid points). It is shown that high data compression ratio can be achieved (around 50:1 ratio) in both vector and scalar data set.
VLSI Universal Noiseless Coder
Proposed universal noiseless coder (UNC) compresses stream of data signals for efficient transmission in channel of limited bandwidth. Noiseless in sense original data completely recoverable from output code. System built as very-large-scale integrated (VLSI) circuit, compressing data in real time at input rates as high as 24 Mb/s, and possibly faster, depending on specific design. Approach yields small, lightweight system operating reliably and consuming little power. Constructed as single, compact, low-power VLSI circuit chip. Design of VLSI circuit chip made specific to code algorithms. Entire UNC fabricated in single chip, worst-case power dissipation less than 1 W.
PDF turbulence modeling and DNS
The problem of time discontinuity (or jump condition) in the coalescence/dispersion (C/D) mixing model is addressed in probability density function (pdf). A C/D mixing model continuous in time is introduced. With the continuous mixing model, the process of chemical reaction can be fully coupled with mixing. In the case of homogeneous turbulence decay, the new model predicts a pdf very close to a Gaussian distribution, with finite higher moments also close to that of a Gaussian distribution. Results from the continuous mixing model are compared with both experimental data and numerical results from conventional C/D models. The effect of Coriolis forces on compressible homogeneous turbulence is studied using direct numerical simulation (DNS). The numerical method used in this study is an eight order compact difference scheme. Contrary to the conclusions reached by previous DNS studies on incompressible isotropic turbulence, the present results show that the Coriolis force increases the dissipation rate of turbulent kinetic energy, and that anisotropy develops as the Coriolis force increases. The Taylor-Proudman theory does apply since the derivatives in the direction of the rotation axis vanishes rapidly. A closer analysis reveals that the dissipation rate of the incompressible component of the turbulent kinetic energy indeed decreases with a higher rotation rate, consistent with incompressible flow simulations (Bardina), while the dissipation rate of the compressible part increases; the net gain is positive. Inertial waves are observed in the simulation results.