Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62
All sourcesNASA NTRSDOE OSTI
NASA NTRS1960

Correlation of the Buckling Strength of Pressurized Cylinders in Compression or Bending with Structural Parameters

The data on nonpressurized cylinders in bending of NACA TN 3735 and the data on pressurized cylinders in compression and bending of NASA TN D-360 are correlated with structural parameters by using small-deflection buckling theory and reduced values for the extensional stiffness of the cylinder wall. The correlating procedure should prove useful in future shell-buckling investigations by reducing the number of tests required.

Peterson, James P.↗
DOE OSTI2023

HarDWR - Water Management Area (WMA) Shapefiles

For a detailed description of the database of which this record is only one part, please see the HarDWR meta-record. Borders of all Water Management Areas (WMAs) across the 11 Western-most states of the continental United States are available filtered through a single source. The legal name for this set of boundaries varies state-by-state. The data is provided as two compressed shapefiles. One, stateWMAs, contains data for all 11 states. For 10 of those states, Arizona being the exception, the polygons represent the legal management boundaries used by those states to manage their surface and groundwater resources respectively. Arizona is unique among this collection of states in that surface and groundwater resources are managed using two separate sets of boundaries. During our followup analysis, due to technical reasons we decided to focus on one set of boundaries, those for surface water. Due to this, the Arizona surface WMAs are included within stateWMAs. The Arizona groundwater WMAs are provided as the second file azGroundWMAs as a companion to the first file for completeness and general reference, although the boundaries were no longer referenced in the analysis (Grogan et al., in review) past a certain point. The columns for both shapefiles are: basinNum: the state provided unique numerical ID basinName: the state provided English name of the area, where applicable state: the state name uniID: a unique identifier we created by concatenating the state name, and underscore, and the state numerical ID

Economics↗
DOE OSTI2020

HarDWR - Water Management Area (WMA) Shapefiles

A dataset within the Harmonized Database of Western U.S. Water Rights (HarDWR). For a detailed description of the database, please see the meta-record v2.0. Changelog v2.0 - No changes v1.0 - Initial public release Description Borders of all Water Management Areas (WMAs) across the 11 western-most states of the coterminous United States are available filtered through a single source. The legal name for this set of boundaries varies state-by-state. The data is provided as two compressed shapefiles. One, stateWMAs, contains data for all 11 states. For 10 of those states, Arizona being the exception, the polygons represent the legal management boundaries used by those states to manage their surface and groundwater resources respectively. WMAs refer to the set of boundaries a particular state uses to manage its water resources. Each set of boundaries was collected from the states individually, and then merged into one spatial layer. The merging process included renaming some columns to enable merging with all other source layers, as well as removing columns deemed not required for followup analysis. The retained columns for each boundary are: basinNum - the state provided unique numerical ID; basinName - the state provided English name of the area, where applicable; state - the state name; and uniID - a unique identifier we created by concatenating the state name, and underscore, and the state numerical ID. Arizona is unique within this collection of states in that surface and groundwater resources are managed using two separate sets of boundaries. During our followup analysis (Grogan et al., in review) we decided to focus on one set of boundaries, those for surface water. This is due to the recommendation of our hydrologists that the surface water boundary set is a more realist representation of how water moves across the landscape, as a few of the groundwater boundaries are based on political and/or economic considerations. Therefore, the Arizona surface WMAs are included within stateWMAs. The Arizona groundwater WMAs are provided as a separate file, azGroundWMAs, as a companion to the first file for completeness and general reference. WMA spatial boundary data sources by state: Arizona: Arizona Surface Water Watersheds; Collected February, 2020; https://gisdata2016-11-18t150447874z-azwater.opendata.arcgis.com/datasets/surface-watershed/explore?location=34.158174%2C-111.970823%2C7.50 Arizona: Arizona Ground Water Basins; Collected February, 2020; https://gisdata2016-11-18t150447874z-azwater.opendata.arcgis.com/datasets/groundwater-basin-2/explore?location=34.158174%2C-111.970823%2C7.50 California: California CalWater 2.2.1; Collected February, 2020; https://www.mlml.calstate.edu/mpsl-mlml/data-center/data-entry-tools/data-tools/gis-shapefile-layers/ Colorado: Colorado Water District Boundaries; Collected February, 2020; https://www.colorado.gov/pacific/cdss/gis-data-category Idaho: Idaho Department of Water Resources (IDWR) Administrative Basins; Collected November, 2015; https://data-idwr.opendata.arcgis.com/datasets/fb0df7d688a04074bad92ca8ef74cc26_4/explore?location=45.018686%2C-113.862284%2C6.93 Montana: Collected June, 2019; Directly contacted Montana Department of Natural Resources and Conservation (DNRC) Office of Information Technology (OIT) Nevada: Nevada State Engineer Admin Basin Boundaries; Collected April, 2020 https://ndwr.maps.arcgis.com/apps/mapviewer/index.html?layers=1364d0c3a0284fa1bcd90f952b2b9f1c New Mexico: New Mexico Office of the State Engineer (OSE) Declared Groundwater Basins; Collected April, 2020 https://geospatialdata-ose.opendata.arcgis.com/datasets/ose-declared-groundwater-basins/explore?location=34.179783%2C-105.996542%2C7.51 Oregon: Oregon Water Resources Department (OWRD) Administrative Basins; Collected February, 2020; https://www.oregon.gov/OWRD/access_Data/Pages/Data.aspx Utah: Utah Adjudication Books; Collected April, 2020; https://opendata.gis.utah.gov/datasets/utahDNR::utah-adjudication-books/explore?location=39.497165%2C-111.587782%2C-1.00 Washington: Washington Water Resource Inventory Areas (WRIA); Collected June, 2017; https://ecology.wa.gov/Research-Data/Data-resources/Geographic-Information-Systems-GIS/Data Wyoming: Wyoming State Engineer's Office Board of Control Water Districts; Collected June, 2019; Directly contacted Wyoming State Engineer's Office

Economics↗
DOE OSTI2021

Data Science and Computation for Rapid and Dynamic Compression Experiment Workflows at Experimental Facilities (Workshop Overview) [Slides]

Motivation for this workshop stems from these items: Compression experiments are important to LANL's core mission. The scientific community is sometimes frustrated at the pace of discovery. Data analytics for other experimental regimes are advancing. Now is the time as facilities are upgraded and coming online. Advances in data analytics and computational resources could push compression experiment discoveries to a new level.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. These optimizations can improve the compression ratio of SOTA approaches by up to 3.3× under the same data quality loss. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

Wang, Daoce↗
DOE OSTI2021

Data Science and Computation for Rapid and Dynamic Compression Experiment Workflows at Experimental Facilities, September 8-11, 2020. Workshop Report

The application of high pressure to materials has enabled discoveries in scientific fields such as planetary science, materials science, and materials synthesis. Recent advances in X-ray user light sources and other facilities, co-location and integration of user facilities with high-pressure drivers, availability of high-performance computing (HPC) platforms, and the development of new data science techniques have created opportunities for, and challenges in, advancing data analytics for rapid and dynamic compression experiments. To address these challenges, harness the emerging technology now available, and expedite scientific discovery, Los Alamos National Laboratory (LANL) hosted a virtual workshop entitled “Data Science and Computation for Rapid and Dynamic Compression Workflows at Experimental Facilities” from September 8 to 11, 2020. The workshop included 95 registered scientists and analytics experts from 15 universities, 9 United States (US) national laboratories, 5 US and European X-ray light sources, neutron sources such as the Los Alamos Neutron Science Center (LANSCE), other big science facilities such as the National Ignition Facility (NIF), and an industry representative. The workshop included 31 invited talks and 4 lightning talks by students and postdocs.

36 MATERIALS SCIENCE↗
NASA NTRS1993

A study of the compressible flow through a diffusing S-duct

Benchmark aerodynamic data are presented for compressible flow through a representative S-duct configuration. A numerical prediction of the S-duct flow field, obtained from a subsonic parabolized Navier-Stokes algorithm, is also shown. The experimental and numerical results are compared. Measurements of the three-dimensional velocity field, total pressures, and static pressures were obtained at five cross-sectional planes. Aerodynamic data were gathered with calibrated pneumatic probes. Surface static pressure and surface flow visualization data were also acquired. All reported tests were conducted with an inlet centerline Mach number of 0.6. The Reynolds number, based on the inlet centerline velocity and duct inlet diameter, was 2.6 x 10(exp 6). Thin inlet turbulent boundary layers existed. The collected data should be beneficial to aircraft inlet designers and the measurements are suitable for the validation of computational codes. The results show that a region of streamwise flow separation occurred within the duct. Details about the separated flow region, including mechanisms which drive this complicated flow phenomenon, are discussed. Results also indicate that the duct curvature induces strong pressure driven secondary flows. The cross flows evolve into counter-rotating vortices. These vortices convect low momentum fluid of the boundary layer toward the center of the duct, degrading both the uniformity and magnitude of the total pressure profile.

Wellborn, Steven R.↗
DOE OSTI2022

Ephemeral Learning - Augmenting Triggers with Online-Trained Normalizing Flows

The large data rates at the LHC require an online trigger system to select relevant collisions. Rather than compressing individual events, we propose to compress an entire data set at once. We use a normalizing flow as a deep generative model to learn the probability density of the data online. The events are then represented by the generative neural network and can be inspected offline for anomalies or used for other analysis purposes. We demonstrate our new approach for a toy model and a correlation-enhanced bump hunt.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

On the performance of Trellis coded modulation with octal phase shift keying over the TDRSS channel

As the National Aeronautics and Space Administration moves into the 21st century with programs like Space Station Freedom, a manned mission to Mars, and the new Landsat mission, transmission demands on the Tracking and Data Relay Satellite System (TDRSS) will very likely exceed the available bandwidth. The Manual Lujan, Jr. Center for Space Telemetering and Telecommunications Systems (CSTTS) at New Mexico State University (NMSU) is studying techniques for increasing the data rate capabilities of TDRSS. These techniques include the use of advanced bandwidth efficient modulation formats to increase the data rate that can be sustained in a TDRSS transponder and the use of lossless bandwidth compression of the data to be transmitted to lower the data rate required from the user spacecraft. Based upon current technology the most promising bandwidth efficient modulation technique is Trellis Coded Modulation (TCM) operating with Octal Phase shift Keying (8PSK). Trellis Coded Modulation coding with 8PSK carrier modulation has the capability to increase the data rate which can be transmitted through the TDRSS spacecraft by a factor of 2 to 2.5 times that available with todays coded QPSK systems with only a small penalty in link performance relative to the existing systems. However, before NASA can safely employ TCM coding it is necessary to prove that this complex format can perform on the real TDRSS link as it does in labs and simulation studies. This proof-of-concept test over a live satellite channel was the objective of the construction and testing performed under this task of the NMSU NASA grant referenced above. In conjunction with NASA, NMSU's CSTTS has constructed a system to test a new candidate TDRSS modulation scheme, TCM 8PSK, that can enhance the information throughput of the TDRSS spacecraft. The test system for this project which was constructed over a period of 18 months by NMSU consisted of two racks of commercial and univeristy-designed and -built equipment. This project has included modifications of an existing White Sands Ground Terminal (WSGT) High Rate QPSK Demodulator to demodulate 8PSK as well as the construction of other support hardware. Also, two TCM codecs (coder/decoders) have been constructed to implement two levels of bandwidth efficiency. One was designed and built by the research team at NMSU while the other was created by the University of Notre Dame with the University of South Australia. The NMSU codec achieves a 2-to-1 increase in data rate per unit bandwidth with a coding gain relative to QPSK of about 3dB. The Notre Dame/South Australia codec achieves a 2.5-to-1 increase in data rate per unit of occupied bandwidth and a coding gain of about 2dB.

Osborne, William P.↗
NASA NTRS1987

A multiprocessing architecture for real-time monitoring

A multiprocessing architecture for performing real time monitoring and analysis using knowledge-based problem solving techniques is discussed. To handle asynchronous inputs and perform in real time, the system consists of three or more separate processes which run concurrently on one or more processors and communicate via a message passing scheme. The Data Management Process gathers, compresses, scales and sends the incoming telemetry data to other tasks. The Inference Process consists of a proprietary high performance inference engine that runs at 1000 rules per second using telemetry data to perform real time analysis on the state and health of the Space Telescope. The multiprocessing architecture has been interfaced to a simulator and is able to process the incoming telemetry in real time.

Laffey, Thomas J.↗
NASA NTRS1978

Applications of dynamic measurement techniques to wind-tunnel testing

The uses of laser Doppler velocimeter, hot wire, and surface hot film techniques in the study of turbulent flows are described, and data obtained in compressible flows are discussed. Applications are illustrated with measurements of wind tunnel freestream turbulence characteristics and with data obtained in transitional, turbulent, and separated shear flows. A new method which was developed for the study of time dependent and unsteady turbulent flows is also presented.

Owen, F. K.↗
NASA NTRS2005

Low-Complexity Lossless Compression of Hyperspectral Imagery via Adaptive Filtering

A low-complexity, adaptive predictive technique for lossless compression of hyperspectral data is presented. The technique relies on the sign algorithm from the repertoire of adaptive filtering. The compression effectiveness obtained with the technique is competitive with that of the best of previously described techniques with similar complexity.

Klimesh, M.↗
DOE OSTI2020

Toxicology and Biodegradability of Blendstocks for Mixing Controlled Compression Ignition Combustion: Literature Review of Available Data

The goal of the U.S. Department of Energy’s Co-Optima project is to develop an understanding of potential biomass-derived blendstocks for use in the transportation sector. This report documents the estimated environmental and toxicological impact of potential bioblendstocks for mixing controlled compression ignition (MCCI) engines. The candidate molecules were: 2-nonanol; n-undecane; 2,6,10-trimethyldodecane; 5-ethyl-4-propylnonane; hexylhexanoate; methyldecanoate; dibutoxymethane; 4-butoxyheptane; dipentylether; renewable diesel; and soy biodiesel. To provide a robust comparison, diesel surrogate molecules were also considered: α-methylnaphthalene; decahydronaphthalene; 2,2,4,4,6,8,8-heptamethylnonane; n-butylcyclohexane; n-hexadecane; tetralin; and n-dodecylbenzene. The intent of this work was not to provide absolute answers and remove any molecules from consideration, but to provide more robust data for each molecule that can be used in subsequent evaluations. Little literature data was available for many of the molecules, both potential blendstocks and diesel surrogates, so a quantitative structural activity relationship (QSAR) approach was used to provide input into the relevant models. This assessment included an evaluation of compartment partitioning in the event of an accidental chemical spill, fate and transport indicators, estimated biodegradability, and human and environmental toxicology. The higher molecular weight and longer hydrocarbon chain lengths of these MCCI molecules reduce water solubility and mobility. Generally, the oxygenated molecules improved biodegradability metrics. The hydrocarbons farnesane and 5-Et-4-PrNonane are not predicted to be biodegradable. Human acute toxicity was found to be slightly to non-toxic for the MCCI blendstocks and the diesel surrogates. 2-Nonanol, undecane, MeDecanoate, dibutoxymethane, and dipentylether were likely developmental toxicants. Although poorly water soluble, all the compounds, whether Co-Optima blendstocks or diesel surrogates, showed significant toxicity to D. magna. The oxygenated blendstocks were less likely to bioaccumulate than the non-oxygenated blendstocks, which behaved similar to the diesel surrogates. Overall, this predicted analysis showed that the MCCI blendstocks are very similar to their diesel counterparts. No significant showstoppers were found that would indicate concern moving forward with additional research into these potential fuels. The results presented were based solely on predictive models and serve to provide information going forward for continued evaluation of these molecules.

09 BIOMASS FUELS↗
DOE OSTI2025

TensorID v1.0

This Python software package includes new and efficient algorithms for satellite and core interpolative decomposition of tensor data. In general, these algorithms target high-dimensional data reduction and compression. The software is purely numerical and can be applied by others to many important sources of tensor data generated by computation or experiment.

Zhang, Yifan [Lawrence Berkeley National Laborator↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2021

A Platform-Independent X-ray Diffraction Diagnostic for Phase Transition Kinetics in Traditional and Synthetic Microstructure Materials (LDRD Project 213088 Final Report)

Pulsed-power generators using the magnetic loading technique are able to produce well-controlled continuous ramp compression of condensed matter for high-pressure equation-of-state studies. X-ray diffraction (XRD) data from dynamically compressed samples provide direct measurements of the elastic compression of the crystal lattice, onset of plastic flow, strength-strain rate dependence, structural phase transitions, and density of crystal defects such as dislocations. Here, we present a cost effective, compact X-ray source for XRD measurements on pulsed-power-driven ramp-loaded samples. This combination of magnetically-driven ramp compression of materials with single, short-pulse XRD diagnostic will be a powerful capability for the dynamic materials community. The success in fielding this new XRD diagnostic dramatically improves our predictive capability and understanding of rate-dependent behavior at or near phase transition. As Sandia plans the next-generation pulse-power driver platform, a key element needed to deliver new state-of-the-art experiments will be having the necessary diagnostic tools to probe new regimes and phenomena. These diagnostics need to be as versatile, compact, and portable as they are powerful. The development of a platform-independent XRD diagnostic gives Sandia researchers a new window to study the microstructure and phase dynamics of materials under load. This project has paved the way for phase transition research in a variety of materials with mission interest.

36 MATERIALS SCIENCE↗
NASA NTRS2009

Improving 3D Wavelet-Based Compression of Hyperspectral Images

Two methods of increasing the effectiveness of three-dimensional (3D) wavelet-based compression of hyperspectral images have been developed. (As used here, images signifies both images and digital data representing images.) The methods are oriented toward reducing or eliminating detrimental effects of a phenomenon, referred to as spectral ringing, that is described below. In 3D wavelet-based compression, an image is represented by a multiresolution wavelet decomposition consisting of several subbands obtained by applying wavelet transforms in the two spatial dimensions corresponding to the two spatial coordinate axes of the image plane, and by applying wavelet transforms in the spectral dimension. Spectral ringing is named after the more familiar spatial ringing (spurious spatial oscillations) that can be seen parallel to and near edges in ordinary images reconstructed from compressed data. These ringing phenomena are attributable to effects of quantization. In hyperspectral data, the individual spectral bands play the role of edges, causing spurious oscillations to occur in the spectral dimension. In the absence of such corrective measures as the present two methods, spectral ringing can manifest itself as systematic biases in some reconstructed spectral bands and can reduce the effectiveness of compression of spatially-low-pass subbands. One of the two methods is denoted mean subtraction. The basic idea of this method is to subtract mean values from spatial planes of spatially low-pass subbands prior to encoding, because (a) such spatial planes often have mean values that are far from zero and (b) zero-mean data are better suited for compression by methods that are effective for subbands of two-dimensional (2D) images. In this method, after the 3D wavelet decomposition is performed, mean values are computed for and subtracted from each spatial plane of each spatially-low-pass subband. The resulting data are converted to sign-magnitude form and compressed in a manner similar to that of a baseline hyperspectral- image-compression method. The mean values are encoded in the compressed bit stream and added back to the data at the appropriate decompression step. The overhead incurred by encoding the mean values only a few bits per spectral band is negligible with respect to the huge size of a typical hyperspectral data set. The other method is denoted modified decomposition. This method is so named because it involves a modified version of a commonly used multiresolution wavelet decomposition, known in the art as the 3D Mallat decomposition, in which (a) the first of multiple stages of a 3D wavelet transform is applied to the entire dataset and (b) subsequent stages are applied only to the horizontally-, vertically-, and spectrally-low-pass subband from the preceding stage. In the modified decomposition, in stages after the first, not only is the spatially-low-pass, spectrally-low-pass subband further decomposed, but also spatially-low-pass, spectrally-high-pass subbands are further decomposed spatially. Either method can be used alone to improve the quality of a reconstructed image (see figure). Alternatively, the two methods can be combined by first performing modified decomposition, then subtracting the mean values from spatial planes of spatially-low-pass subbands.

Klimesh, Matthew↗
DOE OSTI2020

Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution

To address the problem of ever-growing scientific data sizes making data movement a major hindrance to analysis, we introduce a novel encoding for scalar fields: a unified tree of resolution and precision, specifically constructed so that valid cuts correspond to sensible approximations of the original field in the precision-resolution space. Furthermore, we introduce a highly flexible encoding of such trees that forms a parameterized family of data hierarchies. We discuss how different parameter choices lead to different trade-offs in practice, and show how specific choices result in known data representation schemes such as zfp[52], idx[58], and jpeg2000 [76]. Lastly, we provide system-level details and empirical evidence on how such hierarchies facilitate common approximate queries with minimal data movement and time, using real-world data sets ranging from a few gigabytes to nearly a terabyte in size. Experiments suggest that our new strategy of combining reductions in resolution and precision is competitive with state-of-the-art compression techniques with respect to data quality, while being significantly more flexible and orders of magnitude faster, and requiring significantly reduced resources.

97 MATHEMATICS AND COMPUTING↗