Results of adaptive predictor studies, II
Conditional expectation predictor applied to data compression, and computer simulation of Tiros cloud cover picture data
TIROS SATELLITE↗
SEARCH · Search NASA
Results for “Data Compression”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Generation and Performance of Automated Jarosite Mineral Detectors for Vis/NIR Spectrometers at Mars
Sulfate salt discoveries at the Eagle and Endurance craters in Meridiani Planum by the Mars Exploration Rover Opportunity have proven mineralogically the existence and involvement of water in Mars past. Visible and near infrared spectrometers like the Mars Express OMEGA, the Mars Reconnaissance Orbiter CRISM and the 2009 Mars Science Laboratory Rover cameras are powerful tools for the identification of water-bearing salts and other high priority minerals at Mars. The increasing spectral resolution and rover mission lifetimes represented by these missions currently necessitate data compression in order to ease downlink restrictions. On board data processing techniques can be used to guide the selection, measurement and return of scientifically important data from relevant targets, thus easing bandwidth stress and increasing scientific return. We have developed an automated support vector machine (SVM) detector operating in the visible/near-infrared (VisNIR, 300-2500 nm) spectral range trained to recognize the mineral jarosite (typically KFe3(SO4)2(OH)6), positively identified by the Mossbauer spectrometer at Meridiani Planum. Additional information is included in the original extended abstract.
Gilmore, M. S.↗
Nisar L-band Digital Electronics Subsystem
The NASA-ISRO Synthetic Aperture Radar (NISAR) L-band SAR instrument employs multiple digital channels to optimize resolution while keeping a large swath on a single pass. High-speed digitization with fine synchronization and digital beam forming are necessary in order to facilitate this new technique called SweepSAR. An architecture employing multiple FPGA based digital signal processors has been conceived to facilitate digital calibration on an individual channel basis as well as digital signal processing to optimize the receive signal. On-board processing and data compression has been implemented to reduce the volume of data in order to satisfy the operational requirements of near global coverage for the desired science targets. A novel command and timing architecture was developed to manage this complex system to meet the challenging project requirements. The NISAR L-band Digital Electronics Subsystem is the combination of the hardware, firmware and software components architected and implemented to operate this radar and return the desired quantity and quality of data for the science community.
SweepSAR↗
NISAR L-SAR Digital Electronics Subsystem - A Multichannel Distributed Processing System with Synchronous Timing Control for Digital Beam Forming and Multiple Echo Tracking
The NASA-ISRO Synthetic Aperture Radar (NISAR) L-band SAR instrument employs multiple digital channels to optimize resolution while keeping a large swath on a single pass. High-speed digitization with fine synchronization and digital beam forming are necessary in order to facilitate this new technique called SweepSAR. An architecture employing multiple FPGA based digital signal processors has been conceived to facilitate digital calibration on an individual channel basis as well as digital signal processing to optimize the receive signal. On-board processing and data compression has been implemented to reduce the volume of data in order to satisfy the operational requirements of near global coverage for the desired science targets. A novel command and timing architecture was developed to manage this complex system while providing detailed control of individual channel receive window timing required for digital beam forming. The NISAR L-band Digital Electronics Subsystem is the combination of the hardware, firmware and software components architected and implemented to operate this radar and return the desired quantity and quality of data for the science community.
Chuang, Chung-Lun↗
Adaptive coding of MSS imagery
A number of adaptive data compression techniques are considered for reducing the bandwidth of multispectral data. They include adaptive transform coding, adaptive DPCM, adaptive cluster coding, and a hybrid method. The techniques are simulated and their performance in compressing the bandwidth of Landsat multispectral images is evaluated and compared using signal-to-noise ratio and classification consistency as fidelity criteria.
Habibi, A.↗
Compressive Sensing Based Data Acquisition Architecture for Transient Stellar Events in Crowded Star Fields
Compressive sensing is a mathematical technique for simultaneous data acquisition and compression. In this work, we show a CS based architecture for acquiring and reconstructing transient stellar events. This architecture recon-structs a differenced image itself, eliminating the need for any sparse domain transforms, otherwise required for traditional CS reconstruction. The resulting reconstructed differenced image is of critical importance as the information required for generating a time-series photometric light curve is obtained only from a differenced image. Hence, reconstructing a crowded star spatial image, followed by differencing is wasteful. This architecture eliminates the need to 1.) transform an image to a sparse domain, 2.) Reconstruct a dense field, and then apply differencing on the image to obtain the image of critical value. We study the case of microlensing to depict a star source experiencing magnification in time. Our results show that this architecture is able to reconstruct star source magnitudes with magnification factors greater than 1 for clean images with error less than 2% using only 10% of the Nyquist rate samples.
Compressive Sensing, Data acquisition, Image diffe↗
Study of efficient video compression algorithms for space shuttle applications
Results are presented of a study on video data compression techniques applicable to space flight communication. This study is directed towards monochrome (black and white) picture communication with special emphasis on feasibility of hardware implementation. The primary factors for such a communication system in space flight application are: picture quality, system reliability, power comsumption, and hardware weight. In terms of hardware implementation, these are directly related to hardware complexity, effectiveness of the hardware algorithm, immunity of the source code to channel noise, and data transmission rate (or transmission bandwidth). A system is recommended, and its hardware requirement summarized. Simulations of the study were performed on the improved LIM video controller which is computer-controlled by the META-4 CPU.
Poo, Z.↗
Design of a digital compression technique for shuttle television
The determination of the performance and hardware complexity of data compression algorithms applicable to color television signals, were studied to assess the feasibility of digital compression techniques for shuttle communications applications. For return link communications, it is shown that a nonadaptive two dimensional DPCM technique compresses the bandwidth of field-sequential color TV to about 13 MBPS and requires less than 60 watts of secondary power. For forward link communications, a facsimile coding technique is recommended which provides high resolution slow scan television on a 144 KBPS channel. The onboard decoder requires about 19 watts of secondary power.
Habibi, A.↗
Performance of Lempel-Ziv compressors with deferred innovation
The noiseless data-compression algorithms introduced by Lempel and Ziv (LZ) parse an input data string into successive substrings each consisting of two parts: The citation, which is the longest prefix that has appeared earlier in the input, and the innovation, which is the symbol immediately following the citation. In extremal versions of the LZ algorithm the citation may have begun anywhere in the input; in incremental versions it must have begun at a previous parse position. Originally the citation and the innovation were encoded, either individually or jointly, into an output word to be transmitted or stored. Subsequently, it was speculated that the cost of this encoding may be excessively high because the innovation contributes roughly 1g(A) bits, where A is the size of the input alphabet, regardless of the compressibility of the source. To remedy this excess, it was suggested to store the parsed substring as usual, but encoding for output only the citation, leaving the innovation to be encoded as the first symbol of the next substring. Being thus included in the next substring, the innovation can participate in whatever compression that substring enjoys. This strategy is called deferred innovation. It is exemplified in the algorithm described by Welch and implemented in the C program compress that has widely displaced adaptive Huffman coding (compact) as a UNIX system utility. The excessive expansion is explained, an implicit warning is given against using the deferred innovation compressors on nearly incompressible data.
Cohn, Martin↗
Automatic commanding of the Mars Observer Camera
Mars Observer, launched in September 1992, was intended to be a 'survey-type' mission that acquired global coverage of Mars from a low, circular, near-polar orbit during an entire Martian year. As such, most of its instruments had fixed data rates, wide fields of view, and relatively low resolution, with fairly limited requirements for commanding. An exception is the Mars Observer Camera, or MOC. The MOC consists of a two-color Wide Angle (WA) system that can acquire both global images at low resolution (7.5 km/pixel) and regional images at commandable resolutions up to 250 m/pixel. Complementing the WA is the Narrow Angle (NA) system, that can acquire images at 8 resolutions from 12 m/pixel to 1.5 m/pixel, with a maximum crosstrack dimension of 3 km. The MOC also provides various forms of data compression (both lossless and lossy), and is designed to work at data rates from 700 bits per second (bps) to over 80k bps. Because of this flexibility, developing MOC command sequences is much more difficult than the routine mode-changing that characterizes other instrument operations. Although the MOC cannot be pointed (the spacecraft is fixed nadir-pointing and has no scan platform), the timing, downlink stream allocation, compression type and parameters, and image dimensions of each image must be commanded from the ground, subject to the constraints inherent in the MOC and the spacecraft. To minimize the need for a large operations staff, the entire command generation process has been automated within the MOC Ground Data System. Following the loss of the Mars Observer spacecraft in August 1993, NASA intends to launch a new spacecraft, Mars Global Surveyor (MGS), in late 1996. This spacecraft will carry the MOC flight spare (MOC 2). The MOC 2 operations plan will be largely identical to that developed for MOC, and all of the algorithms described here are applicable to it.
Caplinger, Michael↗
Digital ultrasonics signal processing: Flaw data post processing use and description
A modular system composed of two sets of tasks which interprets the flaw data and allows compensation of the data due to transducer characteristics is described. The hardware configuration consists of two main units. A DEC LSI-11 processor running under the RT-11 sngle job, version 2C-02 operating system, controls the scanner hardware and the ultrasonic unit. A DEC PDP-11/45 processor also running under the RT-11, version 2C-02, operating system, stores, processes and displays the flaw data. The software developed the Ultrasonics Evaluation System, is divided into two catagories; transducer characterization and flaw classification. Each category is divided further into two functional tasks: a data acquisition and a postprocessor ask. The flaw characterization collects data, compresses its, and writes it to a disk file. The data is then processed by the flaw classification postprocessing task. The use and operation of a flaw data postprocessor is described.
Buel, V. E.↗
Block adaptive quantization of Magellan SAR data
A report is presented on a data compression scheme that will be used to reduce the SAR data rate on the NASA Magellan mission to Venus. The spacecraft has only one scientific instrument, a radar system for imaging the surface, for altimetric profiling of the planet topography, and for measuring radiation from the planet surface. A straightforward implementation of the scientific requirements of the mission results in a data rate higher than can be accommodated by the available system bandwidth. A data-rate-reduction scheme which includes operation of the radar in burst mode and block-adaptive quantization of the SAR data is selected to satisfy the scientific requirements. Descriptions of the quantization scheme and its hardware implementation are given. Burst-mode SAR operation is also briefly discussed.
Kwok, Ronald↗
Electron trapping data storage system and applications
The advent of digital information storage and retrieval has led to explosive growth in data transmission techniques, data compression alternatives, and the need for high capacity random access data storage. Advances in data storage technologies are limiting the utilization of digitally based systems. New storage technologies will be required which can provide higher data capacities and faster transfer rates in a more compact format. Magnetic disk/tape and current optical data storage technologies do not provide these higher performance requirements for all digital data applications. A new technology developed at the Optex Corporation out-performs all other existing data storage technologies. The Electron Trapping Optical Memory (ETOM) media is capable of storing as much as 14 gigabytes of uncompressed data on a single, double-sided 54 inch disk with a data transfer rate of up to 12 megabits per second. The disk is removable, compact, lightweight, environmentally stable, and robust. Since the Write/Read/Erase (W/R/E) processes are carried out 100 percent photonically, no heating of the recording media is required. Therefore, the storage media suffers no deleterious effects from repeated Write/Read/Erase cycling.
Brower, Daniel↗
HP upgrade operational streamlining
New computer technology and resources must be successfully integrated into CDSLR station operations to manage new complex operational tracking requirements, support the on site production of new data products, support ongoing station performance improvements, and to support new station communication requirements. The NASA CDSLR Network is in the process of upgrading station computer resources with HP UNIX workstations, designed to automate a wide range of operational station requirements. The primary HP upgrade objective was to relocate computer intensive data system tasks from the controller computer to a new advanced computer environment designed to meet the new data system requirements. The HP UNIX environment supports fully automated real time data communications, data management, data processing, and data quality control. Automated data compression procedures are used to improve the efficiency of station data communications. In addition, the UNIX environment supports a number of semi-automated technical and administrative operational station tasks. The x window user interface generates multiple simultaneous color graphics displays, providing direct operator visibility and control over a wide range of operational station functions.
Edge, David R.↗
The Kepler Science Operations Center Pipeline Framework Extensions
The Kepler Science Operations Center (SOC) is responsible for several aspects of the Kepler Mission, including managing targets, generating on-board data compression tables, monitoring photometer health and status, processing the science data, and exporting the pipeline products to the mission archive. We describe how the generic pipeline framework software developed for Kepler is extended to achieve these goals, including pipeline configurations for processing science data and other support roles, and custom unit of work generators that control how the Kepler data are partitioned and distributed across the computing cluster. We describe the interface between the Java software that manages the retrieval and storage of the data for a given unit of work and the MATLAB algorithms that process these data. The data for each unit of work are packaged into a single file that contains everything needed by the science algorithms, allowing these files to be used to debug and evolve the algorithms offline.
Klaus, Todd C.↗
Neural-Based Compression Scheme for Solar Image Data
Studying the solar system and especially the Sun relies on the data gathered daily from space missions. These missions are data-intensive and compressing this data to make them efficiently transferable to the ground station is a twofold decision to make. Stronger compression methods, by distorting the data, can increase data throughput at the cost of accuracy which could affect scientific analysis of the data. On the other hand, preserving subtle details in the compressed data requires a high amount of data to be transferred, reducing the desired gains from compression. In this work, we propose a neural network-based lossy compression method to be used in NASA’s data-intensive imagery missions. We chose NASA’s Solar Dynamics Observatory (SDO) mission which transmits 1.4 terabytes of data each day as a proof of concept for the proposed algorithm. In this work, we propose an adversarially trained neural network, equipped with local and non-local attention modules to capture both the local and global structure of the image resulting in a better trade-off in rate-distortion (RD) compared to conventional hand-engineered codecs. The RD variational autoencoder used in this work is jointly trained with a channel-dependent entropy model as a shared prior between the analysis and synthesis transforms to make the entropy coding of the latent code more effective. We also studied how optimizing perceptual losses could help our neural compressor to preserve high-frequency details of the data in the reconstructed compressed image. Our neural image compression algorithm outperforms currently-in-use and state-of-the-art codecs such as JPEG and JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet (EUV) data. As a proof of concept for use of this algorithm in SDO data analysis, we have performed coronal hole (CH) detection using our compressed images, and generated consistent segmentations, even at a compression rate of ∼ 0.1 bits per pixel (compared to 8 bits per pixel on the original data) using EUV data from SDO.
Image coding↗
On-board data management study for EOPAP
The requirements, implementation techniques, and mission analysis associated with on-board data management for EOPAP were studied. SEASAT-A was used as a baseline, and the storage requirements, data rates, and information extraction requirements were investigated for each of the following proposed SEASAT sensors: a short pulse 13.9 GHz radar, a long pulse 13.9 GHz radar, a synthetic aperture radar, a multispectral passive microwave radiometer facility, and an infrared/visible very high resolution radiometer (VHRR). Rate distortion theory was applied to determine theoretical minimum data rates and compared with the rates required by practical techniques. It was concluded that practical techniques can be used which approach the theoretically optimum based upon an empirically determined source random process model. The results of the preceding investigations were used to recommend an on-board data management system for (1) data compression through information extraction, optimal noiseless coding, source coding with distortion, data buffering, and data selection under command or as a function of data activity, (2) for command handling, (3) for spacecraft operation and control, and (4) for experiment operation and monitoring.
Davisson, L. D.↗
Algorithm for Compressing Time-Series Data
An algorithm based on Chebyshev polynomials effects lossy compression of time-series data or other one-dimensional data streams (e.g., spectral data) that are arranged in blocks for sequential transmission. The algorithm was developed for use in transmitting data from spacecraft scientific instruments to Earth stations. In spite of its lossy nature, the algorithm preserves the information needed for scientific analysis. The algorithm is computationally simple, yet compresses data streams by factors much greater than two. The algorithm is not restricted to spacecraft or scientific uses: it is applicable to time-series data in general. The algorithm can also be applied to general multidimensional data that have been converted to time-series data, a typical example being image data acquired by raster scanning. However, unlike most prior image-data-compression algorithms, this algorithm neither depends on nor exploits the two-dimensional spatial correlations that are generally present in images. In order to understand the essence of this compression algorithm, it is necessary to understand that the net effect of this algorithm and the associated decompression algorithm is to approximate the original stream of data as a sequence of finite series of Chebyshev polynomials. For the purpose of this algorithm, a block of data or interval of time for which a Chebyshev polynomial series is fitted to the original data is denoted a fitting interval. Chebyshev approximation has two properties that make it particularly effective for compressing serial data streams with minimal loss of scientific information: The errors associated with a Chebyshev approximation are nearly uniformly distributed over the fitting interval (this is known in the art as the "equal error property"); and the maximum deviations of the fitted Chebyshev polynomial from the original data have the smallest possible values (this is known in the art as the "min-max property").
Hawkins, S. Edward, III↗