Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Shock compression of fused silica: An impedance matching standard

The properties of silica (SiO 2 ) at extreme conditions have important applications for planetary processes and for high pressure research. Here, we report the results of 125 plate impact shock compression experiments on fused silica spanning 200–1100 GPa using the Z machine at Sandia National Laboratories. Additionally, we present a complementary set of density functional theory based molecular dynamics calculations based on an amorphous reference state that extend the Hugoniot to 2500 GPa. We find good agreement between the Z data, extant laser driven shock compression experiment data, and computational results over most of the pressure range. With these results, fused silica can be used as a new impedance matching standard for shock compression experiments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A data-driven approach to sampling matrix selection for compressive sensing

Sampling is a fundamental aspect of any implementation of compressive sensing. Typically, the choice of sampling method is guided by the reconstruction basis. However, this approach can be problematic with respect to certain hardware constraints and is not responsive to domain-specific context. We propose a method for defining an order for a sampling basis that is optimal with respect to capturing variance in data, thus allowing for meaningful sensing at any desired level of compression. We focus on the Walsh-Hadamard sampling basis for its relevance to hardware constraints, but our approach applies to any sampling basis of interest. We illustrate the effectiveness of our method on the Physical Sciences Inc. Fabry-Perot interferometer sensor multispectral dataset, the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset, and a Colorado State University Swiss Ranger depth image dataset. The spectral datasets consist of simulant experiments, including releases of chemicals such as GAA and SF6. We combine our sampling and reconstruction with the adaptive coherence estimator (ACE) and bulk coherence for chemical detection and we incorporate an algorithmic threshold for ACE values to determine the presence or absence of a chemical. We compare results across sampling methods in this context. We have successful chemical detection at a compression rate of 90%. For all three datasets, we compare our sampling approach to standard orderings of sampling basis such as random, sequency, and an analog of sequency that we term `frequency.' In one instance, the peak signal to noise ratio was improved by over 30% across a test set of depth images.

compressive sensing, hyperspectral imaging, Walsh-↗
NASA NTRS1997

Motion video compression system with neural network having winner-take-all function

A motion video data system includes a compression system, including an image compressor, an image decompressor correlative to the image compressor having an input connected to an output of the image compressor, a feedback summing node having one input connected to an output of the image decompressor, a picture memory having an input connected to an output of the feedback summing node, apparatus for comparing an image stored in the picture memory with a received input image and deducing therefrom pixels having differences between the stored image and the received image and for retrieving from the picture memory a partial image including the pixels only and applying the partial image to another input of the feedback summing node, whereby to produce at the output of the feedback summing node an updated decompressed image, a subtraction node having one input connected to received the received image and another input connected to receive the partial image so as to generate a difference image, the image compressor having an input connected to receive the difference image whereby to produce a compressed difference image at the output of the image compressor.

Fang, Wai-Chi↗
NASA NTRS1994

Numerical Simulation of Supersonic Compression Corners and Hypersonic Inlet Flows Using the RPLUS2D Code

A two-dimensional computational code, PRLUS2D, which was developed for the reactive propulsive flows of ramjets and scramjets, was validated for two-dimensional shock-wave/turbulent-boundary-layer interactions. The problem of compression corners at supersonic speeds was solved using the RPLUS2D code. To validate the RPLUS2D code for hypersonic speeds, it was applied to a realistic hypersonic inlet geometry. Both the Baldwin-Lomax and the Chien two-equation turbulence models were used. Computational results showed that the RPLUS2D code compared very well with experimentally obtained data for supersonic compression corner flows, except in the case of large separated flows resulting from the interactions between the shock wave and turbulent boundary layer. The computational results compared well with the experiment results in a hypersonic NASA P8 inlet case, with the Chien two-equation turbulence model performing better than the Baldwin-Lomax model.

Kapoor, Kamlesh↗
NASA NTRS2011

Analysis of Compression Algorithm in Ground Collision Avoidance Systems (Auto-GCAS)

Automatic Ground Collision Avoidance Systems (Auto-GCAS) utilizes Digital Terrain Elevation Data (DTED) stored onboard a plane to determine potential recovery maneuvers. Because of the current limitations of computer hardware on military airplanes such as the F-22 and F-35, the DTED must be compressed through a lossy technique called binary-tree tip-tilt. The purpose of this study is to determine the accuracy of the compressed data with respect to the original DTED. This study is mainly interested in the magnitude of the error between the two as well as the overall distribution of the errors throughout the DTED. By understanding how the errors of the compression technique are affected by various factors (topography, density of sampling points, sub-sampling techniques, etc.), modifications can be made to the compression technique resulting in better accuracy. This, in turn, would minimize unnecessary activation of A-GCAS during flight as well as maximizing its contribution to fighter safety.

Schmalz, Tyler↗
NASA NTRS1993

TOPEX/POSEIDON GPS Demonstration Experiment Data

Global Positioning Satellite (GPS) measurements obtained from the on-board TOPEX/POSEIDON demonstration flight receiver and a global network of ground receivers are available from the GPS Data Handling Facility (GDHF) archive. Ground receiver observations are derived from a six station subset of the GPS global tracking network. The six TOPEX ground stations yielded 95 percent of the expected data during the six month demonstration phase from November, 1992 to April, 1993. For the same period, 98 percent of the available P-code tracking from the TOPEX flight receiver was obtained. Raw carrier phase and P-code pseudorange measurements are provided for the ground and flight receivers along with an additional file of calibrated and compressed flight receiver data...

Guinn, J. R.↗
DOE OSTI2017

Combining Hadoop with MPI to Solve Metagenomics Problems that are both Data- and Compute-intensive

Metagenomics, the study of all microbial species cohabitants in an environment, usually produces large amount of sequence data varying from several GBs to a few TBs. Analyzing metagenomics data includes both data-intensive and compute-intensive steps, making the entire process hard to scale. Here we aim to optimize a metagenomics application that partitions the shortgun metagenomics sequences based on their species of origin. Our solution combines MapReduce-based BioPig analytic toolkit with MPI to provide scalability in respective to both data and compute. We also made some improvements to the existing BioPig toolkit by using simplified data types and compressed k-mer storage. These optimizations leads up to 193× speedup for the computing-intensive step and 9.6× speedup over the entire pipeline. Our optimized application is also capable of processing datasets that are 16 times larger on the same hardware platform. These conclusions indicate integrating heterogeneous technologies such as Hadoop and MPI is quite efficient to solve large genomics problems that are both data-intensive and compute-intensive.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

CODAS Data from Oliktok Point, Beaufort Sea, Alaska

Cryosphere/Ocean Distributed Acoustic Sensing (CODAS) data collected from the Beaufort Sea, Alaska, using ~37.4 km of dark telecommunications fiber located at Oliktok Point, Alaska. Data were collected with a Silixa iDAS, using 10 m gauge length, 2 m spatial resolution, and 1000 Hz sample rate. Provided here are the DAS-recorded time series for the rapid refreeze event described in Baker & Abbott (2022) (see link below). This covers a date range of 2021-11-10 15:00 UTC to 2021-11-11 17:00 UTC. Data have been decimated to 100 Hz and 20 m (i.e., every 10th channel for 1831 channels, total), as used in Baker & Abbott (2022). Data have been extracted from raw format into 1-hour long .sac* files and organized into directories by channel number, spanning channels 100 to 18400. Time series units are nano-strainrate (nm/m/s). For distribution, data have been compressed into .zip files containing all time series files for 100 channels. *For information on the Seismic Analysis Code (SAC) file format: https://seiscode.iris.washington.edu/projects/sac

16 TIDAL AND WAVE POWER↗
NASA NTRS2013

Human Factors Evaluation of Conflict Detection Tool for Terminal Area

A conflict detection and resolution tool, Terminal-area Tactical Separation-Assured Flight Environment (T-TSAFE), is being developed to improve the timeliness and accuracy of alerts and reduce the false alert rate observed with the currently deployed technology. The legacy system in use today, Conflict Alert, relies primarily on a dead reckoning algorithm, whereas T-TSAFE uses intent information to augment dead reckoning. In previous experiments, T-TSAFE was found to reduce the rate of false alerts and increase time between the alert to the controller and a loss of separation over the legacy system. In the present study, T-TSAFE was tested under two meteorological conditions, 1) all aircraft operated under instrument flight regimen, and 2) some aircraft operated under mixed operating conditions. The tool was used to visually alert controllers to predicted Losses of separation throughout the terminal airspace, and show compression errors, on final approach. The performance of T-TSAFE on final approach was compared with Automated Terminal Proximity Alert (ATPA), a tool recently deployed by the FAA. Results show that controllers did not report differences in workload or situational awareness between the T-TSAFE and ATPA cones but did prefer T-TSAFE features over ATPA functionality. T-TSAFE will provide one tool that shows alerts in the data blocks and compression errors via cones on the final approach, implementing all tactical conflict detection and alerting via one tool in TRACON airspace.

Verma, Savita Arora↗
DOE OSTI2019

Effect of Carbon on the Volume of Solid Iron at High Pressure: Implications for Carbon Substitution in Iron Structures and Carbon Content in the Earth’s Inner Core

Understanding the effect of carbon on the density of hcp (hexagonal-close-packed) Fe-C alloys is essential for modeling the carbon content in the Earth’s inner core. Previous studies have focused on the equations of state of iron carbides that may not be applicable to the solid inner core that may incorporate carbon as dissolved carbon in metallic iron. Carbon substitution in hcp-Fe and its effect on the density have never been experimentally studied. We investigated the compression behavior of Fe-C alloys with 0.31 and 1.37 wt % carbon, along with pure iron as a reference, by in-situ X-ray diffraction measurements up to 135 GPa for pure Fe, and 87 GPa for Fe-0.31C and 109 GPa for Fe-1.37C. The results show that the incorporation of carbon in hcp-Fe leads to the expansion of the lattice, contrary to the known effect in body-centered cubic (bcc)-Fe, suggesting a change in the substitution mechanism or local environment. The data on axial compressibility suggest that increasing carbon content could enhance seismic anisotropy in the Earth’s inner core. The new thermoelastic parameters allow us to develop a thermoelastic model to estimate the carbon content in the inner core when carbon is incorporated as dissolved carbon hcp-Fe. The required carbon contents to explain the density deficit of Earth’s inner core are 1.30 and 0.43 wt % at inner core boundary temperatures of 5000 K and 7000 K, respectively.

58 GEOSCIENCES↗
DOE OSTI2017

X-Ray Reflectometry of DMPS Monolayers on a Water Substrate

The molecular structure of dimyristoyl phosphatidylserine (DMPS) monolayers on a water substrate in different phase states has been investigated by X-ray reflectometry with a photon energy of ~8 keV. According to the experimental data, the transition from a two-dimensional expanded liquid state to a solid gel state (liquid crystal) accompanied by the ordering of the hydrocarbon tails C{sub 14}H{sub 27} of the DMPS molecule occurs in the monolayer as the surface pressure rises. The monolayer thickness is 20 ± 3 and 28 ± 2 Å in the liquid and solid phases, respectively, with the deflection angle of the molecular tail axis from the normal to the surface in the gel phase being 26° ± 8°. At least a twofold decrease in the degree of hydration of the polar lipid groups also occurs under two-dimensional monolayer compression. The reflectometry data have been analyzed using two approaches: under the assumption about the presence of two layers with different electron densities in the monolayer and without any assumptions about the transverse surface structure. Both approaches demonstrate satisfactory agreement between themselves in describing the experimental results.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗
DOE OSTI2025

A surprising proliferation of detwinning in β -tin at extreme loading rates

Integrating data from dynamic compression experiments of condensed matter across three national laboratories has led to insight and quantitative calibration of materials strength over decades of loading rate. For many materials, a single strength model (such as PTW) is sufficient to capture the flow-stress strain rate relationship which is monotonic. Here, we show here that β -tin, a tetragonal metal, exhibits dramatic deviations from this behavior. Naive fitting to a single PTW model is insufficient to capture the behavior; indeed, such resulting inferred flow stress versus strain exhibits a non-monotonic behavior. We suggest a resolution to this by proposing that in β -tin there are important Bauschinger effects arising from favorable conditions for twinning and detwinning. A simple yield surface model when paired with PTW hardening captures the experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Evaluation of Graph Analytics Frameworks Using the GAP Benchmark Suite

The analysis of connected data is an increasingly important application in high-performance computing. Such analyses can reveal fraudulent patterns in financial transactions, optimize telecommunications networks, predict information flow in social networks, etc. However, the landscape of graph analytics is highly diverse. Graph algorithms stress processor architectures differently, and no one graph can represent all topologies. Consequently, no single approach or framework is expected to be optimal for all graph analytics problems. To help make sense of this diverse landscape, we evaluated four approaches to graph analytics: GraphBLAS, Galois, BGL17, GraphIt; and compare them against hand-tuned implementations that take advantage of hardware features on our test platform. Graph- BLAS formulates graph analytics as sparse linear algebra. Galois provides syntactic constructs for data parallelism over irregular data structures. BGL17 is a generic C++ template library for implementing graph algorithms. GraphIt provides a domain- specific language to describe and optimize graph algorithms. We use the GAP Benchmark Suite to establish baseline performance and guide the side-by-side evaluation of each framework. GAP consists of 30 tests: six graph analytics algorithms (breadth- first search, single-source shortest path, PageRank, betweenness centrality, connected components, and triangle counting) run on five graphs, each with different topological characteristics (e.g., high diameter, skewed degree distribution, high average degree). High-performance reference implementations are included for each benchmark algorithm. Because a graph can be loaded into memory a number of ways (e.g., flat file on disk, compressed sparse format, data frames, retrieved from SQL or NoSQL databases), our evaluation focused on computational performance rather than I/O. Our results show the relative strengths of each framework.

Graph algorithms, Benchmarking, shared-memory prog↗
NASA NTRS2017

Hierarchical Data Formats (HDF) Update

In this presentation, we will talk about the latest releases of HDF4 and HDF5 software and tools, new features available in HDF5, and roadmap for the HDF software. We will also solicit feedback from the users of HDF data and HDF application developers on new features and new tools. The talk will cover: Difference between 1.8 and 1.10 releases and how and when to move to the latest release Features of the recent HDF5 1.8.19, 1.10.1 and HDF 4.2.13 Overview of HDF View 3.0 and other enhancements to tools Supported compilers and systems Open discussion of new requirements and wish list of the HDF features Compression library for interoperability with h5py and Pandas and better floating-point data compression.

HDFView↗
DOE OSTI2025

High-performance data format for scientific data storage and analysis

Here, in this article, we present the High-Performance Output (HiPO) data format developed at Jefferson Laboratory for storing and analyzing data from Nuclear Physics experiments. The format was designed to efficiently store large amounts of experimental data, utilizing modern fast compression algorithms. The purpose of this development was to provide organized data in the output, facilitating access to relevant information within the large data files. The HiPO data format has features that are suited for storing raw detector data, reconstruction data, and the final physics analysis data efficiently, eliminating the need to do data conversions through the lifecycle of experimental data. The HiPO data format is implemented in C++ and JAVA, and provides bindings to FORTRAN, Python, and Julia, providing users with the choice of data analysis frameworks to use. In this paper, we will present the general design and functionalities of the HiPO library and compare the performance of the library with more established data formats used in data analysis in High Energy and Nuclear Physics (such as ROOT and Parquete). In columnar data analysis, HiPO surpasses established data formats in performance and can be effectively applied to data analysis in other scientific fields.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1993

A browse facility for Earth science remote sensing data: Center director's discretionary fund final report

An image data visual browse facility is developed for a UNIX platform using the X Windows 11 system. It allows one to visually examine reduced resolution image data to determine which data are applicable for further research. Links with a relational data base manager then allow one to extract not only the full resolution image data, but any other ancillary data related to the case study. Various techniques are examined for compression of the image data in order to reduce data storage requirements and time necessary to transmit the data on the internet. Data used were from the WetNet project.

Meyer, P. J.↗
NASA NTRS2010

NASA Tech Briefs, February 2010

Topics covered include: Insulation-Testing Cryostat With Lifting Mechanism; Optical Testing of Retroreflectors for Cryogenic Applications; Measuring Cyclic Error in Laser Heterodyne Interferometers; Self-Referencing Hartmann Test for Large-Aperture Telescopes; Measuring a Fiber-Optic Delay Line Using a Mode-Locked Laser; Reconfigurable Hardware for Compressing Hyperspectral Image Data; Spatio-Temporal Equalizer for a Receiving-Antenna Feed Array; High-Speed Ring Bus; Nanoionics-Based Switches for Radio-Frequency Applications; Lunar Dust-Tolerant Electrical Connector; Compact, Reliable EEPROM Controller; Quad-Chip Double-Balanced Frequency Tripler; Ka-Band Waveguide Two-Way Hybrid Combiner for MMIC Amplifiers; Radiation-Hardened Solid-State Drive; Use of Nanofibers to Strengthen Hydrogels of Silica, Other Oxides, and Aerogels; Two Concepts for Deployable Trusses; Concentric Nested Toroidal Inflatable Structures; Investigating Dynamics of Eccentricity in Turbomachines; Improved Low-Temperature Performance of Li-Ion Cells Using New Electrolytes; Integrity Monitoring of Mercury Discharge Lamps; White-Light Phase-Conjugate Mirrors as Distortion Correctors; Biasable, Balanced, Fundamental Submillimeter Monolithic Membrane Mixer; ICER-3D Hyperspectral Image Compression Software; and Context Modeler for Wavelet Compression of Spectral Hyperspectral Images.

Source record↗