SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Consider the problem of compressing a uniformly quantized source. A traditional approach is to assign variable length codewords to the quantizer output symbols or groups of symbols. Here an alternate solution is proposed: assign a fixed length binary codeword to each output symbol in such a way that a zero is more likely than a one in every codeword bit position. The encoding procedure is described.
The increase of computer processing speed is significantly outpacing improvements in network and storage bandwidth, leading to the big data challenge in modern science, where scientific applications can quickly generate much more data than that can be transferred and stored. As a result, big scientific data must be reduced by a few orders of magnitude while the accuracy of the reduced data needs to be guaranteed for further scientific explorations. Moreover, scientists are often interested in some specific spatial/temporal regions in their data, where higher accuracy is required. The locations of the regions requiring high accuracy can sometimes be prescribed based on application knowledge, while other times they must be estimated based on general spatial/temporal variation. In this paper, we develop a novel multilevel approach which allows users to impose region-wise compression error bounds. Our method utilizes the byproduct of a multilevel compressor to detect regions where details are rich and we provide the theoretical underpinning for region-wise error control. With spatially varying precision preservation, our approach can achieve significantly higher compression ratios than single-error bounded compression approaches and control errors in the regions of interest.We conduct the evaluations on two climate use cases – one targeting small-scale, node features and the other focusing on long, areal features. For both use cases, the locations of the features were unknown ahead of the compression. By selecting approximately 16% of the data based on multi-scale spatial variations and compressing those regions with smaller error tolerances than the rest, our approach improves the accuracy of post-analysis by approximately 2 × compared to single-error-bounded compression at the same compression ratio. Using the same error bound for the region of interest, our approach can achieve an increase of more than 50% in overall compression ratio.
Digital data coders/decoders are used extensively in video transmission. A digitally encoded video signal is separated into subbands. Separating the video into subbands allows transmission at low data rates. Once the data is separated into these subbands it can be coded and then decoded by statistical coders such as the Lempel-Ziv based coder.
Today, most X-ray pixel detectors used at light sources transmit raw pixel data off the detector ASIC. With the availability of more advanced ASIC technology nodes for scientific application, more digital functionalities from the computing domains (e.g., compression) can be integrated directly into a detector ASIC to increase data velocity. In this paper, we describe a lightweight, user-configurable detector ASIC digital architecture with on-chip compression which can be implemented in 130 nm technologies in a reasonable area on the ASIC periphery. In addition, we present a design to efficiently handle the variable data from the stream of parallel compressors. The architecture includes user-selectable lossy and lossless compression blocks. The impact of lossy compression algorithms is evaluated on simulated and experimental X-ray ptychography datasets. This architecture is a practical approach to increase pixel detector frame rates towards the continuous 1 MHz regime for not only coherent imaging techniques such as ptychography, but also for other diffraction techniques at X-ray light sources.
Knowledge of the composition and certain statistical characteristics of the output signals of the scientific instruments on board a space vehicle allows determination of possible signal processing algorithms in the system and to evaluate the prospects for utilization of data reduction. A method is presented to estimate the compression factor of an on-board data collection and processing system with known mean activities of the sensors used and unknown mutual correlation of their output signals.
We present a new convolution layer for deep learning architectures which we call QuadConv — an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. Here, we show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.
A lightweight, low-power, real-time data compressor design for the Earth Observing System (Eos) onboard synthetic aperture radar (SAR) processor is presented. The implementation is based on VLSI design of a pipelined binary tree-searched vector quantizer (VQ), utilizing space-qualifiable 1.25-micron CMOS technology. The implementation exploits VLSI system design principles such as the modularity, regular data flow, simple interconnection, localized communication, simple global control, and parallel/pipelined processing. The overall system requires 30 chips with only one VLSI processing element design. The total weight is about 1.2 lbs, with an estimated power dissipation of approximately 4 watts operating at the maximum input data rate. The projected throughput rate exceeds 5 MHz.
Analog-to-digital converter-pulse height analyzer for data reduction from balloon measurements
Frog retina layered model, stereoscopic system and decision/control system in connection with robot data reduction in Voyager missions
Digital computer for approximation of wavefront propagation to obtain medial point transformation of two dimensional data
Transform coding and block quantization techniques are applied to multispectral aircraft scanner data, and digitized satellite imagery. The multispectral source is defined and an appropriate mathematical model proposed. The Karhunen-Loeve, Fourier, and Hadamard encoders are considered and are compared to the rate distortion function for the equivalent Gaussian source and to the performance of the single sample PCM encoder.
Long-duration, on-orbit microgravity experiments require a combination of high resolution and high frame rate video data acquisition. The digitized high-rate video stream presents a difficult data storage problem. Data produced at rates of several hundred million bytes per second may require a total mission video data storage requirement exceeding one terabyte. A NASA-designed, VLSI-based, highly parallel digital state machine generates a digital trigger signal at the onset of a video event. High capacity random access memory storage coupled with newly available fuzzy logic devices permits the monitoring of a video image stream for long term (DC-like) or short term (AC-like) changes caused by spatial translation, dilation, appearance, disappearance, or color change in a video object. Pre-trigger and post-trigger storage techniques are then adaptable to archiving only the significant video images.
Efficient onboard data compression can reduce the data volume from hyperspectral imagers on NASA and DoD spacecraft in order to return as much imagery as possible through constrained downlink channels. Lossless compression is important for signature extraction, object recognition, and feature classification capabilities. To provide onboard data compression, a hardware implementation of a lossless hyperspectral compression algorithm was developed using a field programmable gate array (FPGA). The underlying algorithm is the Fast Lossless (FL) compression algorithm reported in Fast Lossless Compression of Multispectral- Image Data (NPO-42517), NASA Tech Briefs, Vol. 30, No. 8 (August 2006), p. 26 with the modification reported in Lossless, Multi-Spectral Data Comressor for Improved Compression for Pushbroom-Type Instruments (NPO-45473), NASA Tech Briefs, Vol. 32, No. 7 (July 2008) p. 63, which provides improved compression performance for data from pushbroom-type imagers. An FPGA implementation of the unmodified FL algorithm was previously developed and reported in Fast and Adaptive Lossless Onboard Hyperspectral Data Compression System (NPO-46867), NASA Tech Briefs, Vol. 36, No. 5 (May 2012) p. 42. The essence of the FL algorithm is adaptive linear predictive compression using the sign algorithm for filter adaption. The FL compressor achieves a combination of low complexity and compression effectiveness that exceeds that of stateof- the-art techniques currently in use. The modification changes the predictor structure to tolerate differences in sensitivity of different detector elements, as occurs in pushbroom-type imagers, which are suitable for spacecraft use. The FPGA implementation offers a low-cost, flexible solution compared to traditional ASIC (application specific integrated circuit) and can be integrated as an intellectual property (IP) for part of, e.g., a design that manages the instrument interface. The FPGA implementation was benchmarked on the Xilinx Virtex IV LX25 device, and ported to a Xilinx prototype board. The current implementation has a critical path of 29.5 ns, which dictated a clock speed of 33 MHz. The critical path delay is end-to-end measurement between the uncompressed input data and the output compression data stream. The implementation compresses one sample every clock cycle, which results in a speed of 33 Msample/s. The implementation has a rather low device use of the Xilinx Virtex IV LX25, making the total power consumption of the implementation about 1.27 W.
Voice bandwidth compression techniques were investigated to anticipate link margin difficulties in the shuttle S-band communication system. It was felt that by reducing the data rate on each voice channel from the baseline 24 (or 32) Kbps to 8 Kbps, additional margin could be obtained. The feasibility of such an alternate voice transmission system was studied. Several factors of prime importance that were addressed are: (1) achieving high quality voice at 8 Kbps; (2) performance in the presence of the anticipated shuttle cabin environmental noise; (3) performance in the presence of the anticipated channel error statistics; and (4) minimal increase in size, weight, and power over the current baseline voice processor.
The National Oceanic and Atmospheric Administration (NOAA) receives high quality infrared weather images from each of its two geostationary weather satellites at an average data rate of 57 kilobits/second. These images are currently distributed to field stations over 3 kilohertz analog phone lines. The resulting loss in image quality renders the images unacceptable for proposed digital image processing. This paper documents the study leading to a current effort to implement a microprocessor-based universal noiseless coder/decoder to satisfy NOAA's requirements of high quality, good coverage and timely transmission of its infrared images.
Bit-wise arithmetic coding is data-compression scheme intended especially for use with uniformly quantized data from source with Gaussian, Laplacian, or similar probability distribution function. Code words of fixed length, and bits treated as being independent. Scheme serves as means of progressive transmission or of overcoming buffer-overflow or rate constraint limitations sometimes arising when data compression used.
Telemetry enhancement techniques used by Voyager-2 to reduce telemetry transmission rates by over 50 percent compared to those used at Saturn, with negligible loss in information return, are described. The use of the Reed-Solomon encoder is discussed, and the principles and implementation of an Image Data Compressor algorithm for noiseless coding techniques are addressed. Parallel operation of the redundant Flight Data Subsystem processors is discussed.
The photometric and astrometric accuracy of the Minnesota automated dual-plate scanner is summarized. The relation between magnitude and isophotometric image diameter is linear on Schmidt plates over a large useful range. Results of a study of centroid measurement accuracy and image classification ability of parameters derived from both densitometric and isophotometric scanning techniques are presented. The data analyzed for this purpose are synthesized ideal star and galaxy brightness distributions with appropriate random noise added to simulate plate grain and sky background effects. A preliminary study of image classification using real plates and single isophote data finds that a parameter describing the raggedness of the isophote may be useful in distinguising stars from galaxies.