Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. These optimizations can improve the compression ratio of SOTA approaches by up to 3.3× under the same data quality loss. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

Wang, Daoce↗
DOE OSTI2021

Data Science and Computation for Rapid and Dynamic Compression Experiment Workflows at Experimental Facilities, September 8-11, 2020. Workshop Report

The application of high pressure to materials has enabled discoveries in scientific fields such as planetary science, materials science, and materials synthesis. Recent advances in X-ray user light sources and other facilities, co-location and integration of user facilities with high-pressure drivers, availability of high-performance computing (HPC) platforms, and the development of new data science techniques have created opportunities for, and challenges in, advancing data analytics for rapid and dynamic compression experiments. To address these challenges, harness the emerging technology now available, and expedite scientific discovery, Los Alamos National Laboratory (LANL) hosted a virtual workshop entitled “Data Science and Computation for Rapid and Dynamic Compression Workflows at Experimental Facilities” from September 8 to 11, 2020. The workshop included 95 registered scientists and analytics experts from 15 universities, 9 United States (US) national laboratories, 5 US and European X-ray light sources, neutron sources such as the Los Alamos Neutron Science Center (LANSCE), other big science facilities such as the National Ignition Facility (NIF), and an industry representative. The workshop included 31 invited talks and 4 lightning talks by students and postdocs.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Ephemeral Learning - Augmenting Triggers with Online-Trained Normalizing Flows

The large data rates at the LHC require an online trigger system to select relevant collisions. Rather than compressing individual events, we propose to compress an entire data set at once. We use a normalizing flow as a deep generative model to learn the probability density of the data online. The events are then represented by the generative neural network and can be inspected offline for anomalies or used for other analysis purposes. We demonstrate our new approach for a toy model and a correlation-enhanced bump hunt.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Toxicology and Biodegradability of Blendstocks for Mixing Controlled Compression Ignition Combustion: Literature Review of Available Data

The goal of the U.S. Department of Energy’s Co-Optima project is to develop an understanding of potential biomass-derived blendstocks for use in the transportation sector. This report documents the estimated environmental and toxicological impact of potential bioblendstocks for mixing controlled compression ignition (MCCI) engines. The candidate molecules were: 2-nonanol; n-undecane; 2,6,10-trimethyldodecane; 5-ethyl-4-propylnonane; hexylhexanoate; methyldecanoate; dibutoxymethane; 4-butoxyheptane; dipentylether; renewable diesel; and soy biodiesel. To provide a robust comparison, diesel surrogate molecules were also considered: α-methylnaphthalene; decahydronaphthalene; 2,2,4,4,6,8,8-heptamethylnonane; n-butylcyclohexane; n-hexadecane; tetralin; and n-dodecylbenzene. The intent of this work was not to provide absolute answers and remove any molecules from consideration, but to provide more robust data for each molecule that can be used in subsequent evaluations. Little literature data was available for many of the molecules, both potential blendstocks and diesel surrogates, so a quantitative structural activity relationship (QSAR) approach was used to provide input into the relevant models. This assessment included an evaluation of compartment partitioning in the event of an accidental chemical spill, fate and transport indicators, estimated biodegradability, and human and environmental toxicology. The higher molecular weight and longer hydrocarbon chain lengths of these MCCI molecules reduce water solubility and mobility. Generally, the oxygenated molecules improved biodegradability metrics. The hydrocarbons farnesane and 5-Et-4-PrNonane are not predicted to be biodegradable. Human acute toxicity was found to be slightly to non-toxic for the MCCI blendstocks and the diesel surrogates. 2-Nonanol, undecane, MeDecanoate, dibutoxymethane, and dipentylether were likely developmental toxicants. Although poorly water soluble, all the compounds, whether Co-Optima blendstocks or diesel surrogates, showed significant toxicity to D. magna. The oxygenated blendstocks were less likely to bioaccumulate than the non-oxygenated blendstocks, which behaved similar to the diesel surrogates. Overall, this predicted analysis showed that the MCCI blendstocks are very similar to their diesel counterparts. No significant showstoppers were found that would indicate concern moving forward with additional research into these potential fuels. The results presented were based solely on predictive models and serve to provide information going forward for continued evaluation of these molecules.

09 BIOMASS FUELS↗
DOE OSTI2025

TensorID v1.0

This Python software package includes new and efficient algorithms for satellite and core interpolative decomposition of tensor data. In general, these algorithms target high-dimensional data reduction and compression. The software is purely numerical and can be applied by others to many important sources of tensor data generated by computation or experiment.

Zhang, Yifan [Lawrence Berkeley National Laborator↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2021

A Platform-Independent X-ray Diffraction Diagnostic for Phase Transition Kinetics in Traditional and Synthetic Microstructure Materials (LDRD Project 213088 Final Report)

Pulsed-power generators using the magnetic loading technique are able to produce well-controlled continuous ramp compression of condensed matter for high-pressure equation-of-state studies. X-ray diffraction (XRD) data from dynamically compressed samples provide direct measurements of the elastic compression of the crystal lattice, onset of plastic flow, strength-strain rate dependence, structural phase transitions, and density of crystal defects such as dislocations. Here, we present a cost effective, compact X-ray source for XRD measurements on pulsed-power-driven ramp-loaded samples. This combination of magnetically-driven ramp compression of materials with single, short-pulse XRD diagnostic will be a powerful capability for the dynamic materials community. The success in fielding this new XRD diagnostic dramatically improves our predictive capability and understanding of rate-dependent behavior at or near phase transition. As Sandia plans the next-generation pulse-power driver platform, a key element needed to deliver new state-of-the-art experiments will be having the necessary diagnostic tools to probe new regimes and phenomena. These diagnostics need to be as versatile, compact, and portable as they are powerful. The development of a platform-independent XRD diagnostic gives Sandia researchers a new window to study the microstructure and phase dynamics of materials under load. This project has paved the way for phase transition research in a variety of materials with mission interest.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution

To address the problem of ever-growing scientific data sizes making data movement a major hindrance to analysis, we introduce a novel encoding for scalar fields: a unified tree of resolution and precision, specifically constructed so that valid cuts correspond to sensible approximations of the original field in the precision-resolution space. Furthermore, we introduce a highly flexible encoding of such trees that forms a parameterized family of data hierarchies. We discuss how different parameter choices lead to different trade-offs in practice, and show how specific choices result in known data representation schemes such as zfp[52], idx[58], and jpeg2000 [76]. Lastly, we provide system-level details and empirical evidence on how such hierarchies facilitate common approximate queries with minimal data movement and time, using real-world data sets ranging from a few gigabytes to nearly a terabyte in size. Experiments suggest that our new strategy of combining reductions in resolution and precision is competitive with state-of-the-art compression techniques with respect to data quality, while being significantly more flexible and orders of magnitude faster, and requiring significantly reduced resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

DNTTD (Distributed Non-Negative Tensor Train Decomposition)

The era of exascale computing opens new venues for innovations and discoveries in many scientific, engineering, and commercial fields. However, with the exa flops also come the extra-large high-dimensional data generated by high performance computing. High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data by classical tensor factorization techniques. However, the classical tensor methods are not always stable or they can be exponential in their memory requirements, which makes them not suitable for high-dimensional tensors. Tensor train (TT) is a state-of-the-art tensor network introduced for factorization of high-dimensional tensors. TT transforms the initial high-dimensional tensor in a network of three-dimensional tensors that requires only a linear storage. Many real-world data, such as, density, temperature, population, probability, etc., are non-negative and for an easy interpretation, the algorithms preserving non-negativity are preferred. Here, we introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real world big datasets.

Bhattarai, Manish↗
DOE OSTI2022

A compact x-ray diffraction system for dynamic compression experiments on pulsed-power generators

Pulsed-power generators can produce well-controlled continuous ramp compression of condensed matter for high-pressure equation-of-state studies using the magnetic loading technique. X-ray diffraction (XRD) data from dynamically compressed samples provide direct measurements of the elastic compression of the crystal lattice, onset of plastic flow, strength–strain rate dependence, structural phase transitions, and density of crystal defects, such as dislocations. Here, we present a cost-effective, compact, pulsed x-ray source for XRD measurements on pulsed-power-driven ramp-loaded samples. This combination of magnetically driven ramp compression of materials with a single, short-pulse XRD diagnostic will be a powerful capability for the dynamic materials’ community to investigate in situ dynamic phase transitions critical to equation of states. Finally, we present results using this new diagnostic to evaluate lattice compression in Zr and Al and to capture signatures of phase transitions in CdS.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Efficient DNA sequence compression with neural networks

Abstract Background The increasing production of genomic data has led to an intensified need for models that can cope efficiently with the lossless compression of DNA sequences. Important applications include long-term storage and compression-based data analysis. In the literature, only a few recent articles propose the use of neural networks for DNA sequence compression. However, they fall short when compared with specific DNA compression tools, such as GeCo2. This limitation is due to the absence of models specifically designed for DNA sequences. In this work, we combine the power of neural networks with specific DNA models. For this purpose, we created GeCo3, a new genomic sequence compressor that uses neural networks for mixing multiple context and substitution-tolerant context models. Findings We benchmark GeCo3 as a reference-free DNA compressor in 5 datasets, including a balanced and comprehensive dataset of DNA sequences, the Y-chromosome and human mitogenome, 2 compilations of archaeal and virus genomes, 4 whole genomes, and 2 collections of FASTQ data of a human virome and ancient DNA. GeCo3 achieves a solid improvement in compression over the previous version (GeCo2) of $2.4\%$, $7.1\%$, $6.1\%$, $5.8\%$, and $6.0\%$, respectively. To test its performance as a reference-based DNA compressor, we benchmark GeCo3 in 4 datasets constituted by the pairwise compression of the chromosomes of the genomes of several primates. GeCo3 improves the compression in $12.4\%$, $11.7\%$, $10.8\%$, and $10.1\%$ over the state of the art. The cost of this compression improvement is some additional computational time (1.7–3 times slower than GeCo2). The RAM use is constant, and the tool scales efficiently, independently of the sequence size. Overall, these values outperform the state of the art. Conclusions GeCo3 is a genomic sequence compressor with a neural network mixing approach that provides additional gains over top specific genomic compressors. The proposed mixing method is portable, requiring only the probabilities of the models as inputs, providing easy adaptation to other data compressors or compression-based data analysis tools. GeCo3 is released under GPLv3 and is available for free download at https://github.com/cobilab/geco3.

Silva, Milton↗
DOE OSTI2024

Sound speed and Grüneisen parameter up to three terapascal in shock-compressed iron

This paper presents the first sound speed and Grüneisen parameter data for fluid iron compressed to 3 TPa (30 million atmospheres) and 20 g/cm 3 on the Hugoniot. Both the sound speed and Grüneisen parameter are derivatives of the equation of state (EOS), and thus tightly constrain the contours of the EOS surface. The sound speed data are systematically lower than expected from a simple extrapolation of previous data. The Grüneisen parameter shows a 30% drop at pressures and temperatures above the melt transition. Furthermore, while some models compare well with either the sound speed or Grüneisen parameter, none of today’s state-of-the-art models can explain both sets of data. Furthermore these new data will provide pivotal benchmarks for both future theoretical EOSs of warm dense iron and modeling planetary states and processes.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Simulations of biomass compression-screw feeding using a compressible non-Newtonian constitutive model

There is global interest in the conversion of biomass into sustainable low-carbon-footprint fuels and chemicals as an alternative to non-renewable fossil feedstocks. Feeding biomass solids into pressurized reactors is one of the key steps in biomass conversion. Predicting mechanical failure and energy requirements for this step helps avoid upstream processing bottlenecks and enables efficient operation of a biorefinery. Here, in this work, we developed a predictive computational model for biomass screw feeders that capture the highly viscous, non-Newtonian and compressible behavior of biomass slurries. Biomass compressible behavior is formulated by an equation of state and the non-Newtonian rheology is represented by a density-dependent viscosity model. Experimental data from two compression screw-feeder systems are presented as a validation for our model. Our model successfully predicted the location of the compressed biomass “plug”, biomass flow rate, and the required torque at different operating conditions for the experimental conditions studied in this work.

09 BIOMASS FUELS↗
DOE OSTI2024

Effect of Part Size, Displacement Rate, and Aging on Compressive Properties of Elastomeric Parts of Different Unit Cell Topologies Formed by Vat Photopolymerization Additive Manufacturing

Due to its ability to achieve geometric complexity at high resolution and low length scales, additive manufacturing (AM) has increasingly been used for fabricating cellular structures (e.g., foams and lattices) for a variety of applications. Specifically, elastomeric cellular structures offer tunability of compliance as well as energy absorption and dissipation characteristics. However, there are limited data available on compression properties for printed elastomeric cellular structures of different designs and testing parameters. In this work, the authors evaluate how unit cell topology, part size, the rate of compression, and aging affect the compressive response of polyurethane-based simple cubic, body-centered, and gyroid structures formed by vat photopolymerization AM. Finite element simulations incorporating hyperelastic and viscoelastic models were used to describe the data, and the simulated results compared well with the experimental data. Of the designs tested, only the parts with the body-centered unit cell exhibited differences in stress–strain responses at different part sizes. Of the compression rates tested, the highest displacement rate (1000 mm/min) often caused stiffer compressive behavior, indicating deviation from the quasi-static assumption and approaching the intermediate rate response. The cellular structures did not change in compression properties across five weeks of aging time, which is desirable for cushioning applications. This work advances knowledge on the structure–property relationships of printed elastomeric cellular materials, which will enable more predictable compressive properties that can be traced to specific unit cell designs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Autoignition of cyclohexane at low-to-intermediate temperatures: Rapid compression machine experiments and improved comprehensive chemical kinetic model

Cycloalkanes are an important class of hydrocarbons found in conventional and alternative transportation fuels. Being one of the simplest practical cycloalkanes, cyclohexane is often used as a surrogate in studies to understand the oxidation, ignition, and sooting characteristics of cycloalkanes more broadly. Although cyclohexane has been extensively studied, very limited datasets for cyclohexane oxidation at low-to-intermediate temperatures are available in the literature. Moreover, the literature autoignition data from rapid compression machines (RCMs) did not provide information on volume histories or experimental heat transfer characteristics which can significantly impact model predictions, thereby hindering the detailed validation of chemical kinetic models. It has also been observed that the cyclohexane kinetic models developed so far are not able to predict the literature data across a wide range of operating and mixture conditions. Hence, further studies are recommended to understand and improve model predictions for cyclohexane oxidation. In this study, RCM experiments were conducted using cyclohexane/air mixtures at varying compressed pressures (P C =15, 20, and 30 bar), equivalence ratios (ϕ=0.5, 1.0, and 2.0), and compressed temperatures (T C =670–960 K). The current RCM results of cyclohexane were found to complement well with the literature data, with the first-stage IDTs at high pressures and volume histories at all test conditions being first reported for better development and validation of chemical kinetic models. Additionally, a detailed chemical kinetic model for the cyclohexane oxidation has been developed and validated using both newly acquired experimental IDTs in this study and literature experimental studies (IDTs in RCMs and STs, species profiles in jet-stirred reactors, and laminar burning velocities). Unlike literature models, the proposed model was observed to provide an overall better prediction for experimental results over a wide range of conditions. Chemical kinetic analysis of the developed model was further conducted to identify the reaction pathways controlling the autoignition of cyclohexane.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Modulated Thermomechanical Analysis of Compression-Molded High-Density Polyethylene

Thermomechanical analysis (TMA) experiments conducted on high-density polyethylene (HDPE) show both reversible and irreversible dimensional changes. To further explore these reversible and irreversible processes, modulated thermomechanical analysis (MTMA) was used. Before reliable data on compression-molded HDPE was collected, a parameter optimization was performed to obtain a suitable MTMA method. Once a suitable method was obtained, several MTMA experiments were conducted on compression-molded HDPE. This work highlights the steps taken during the MTMA parameter optimization and the results obtained from MTMA experiments conducted on pristine compression-molded HDPE samples.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Technical report: Design of the accessor

As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al. Such a decoupling of memory precision and arithmetic precision requires an “accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. Such an accessor has to be general in the sense that it can handle a wide range of floating point formats and compression strategies, and still allows for further extension. At the same time, is important that the use of the accessor is not a burden for the developer. Therefore, it is important to abstract all implementation details from the user, hide the underlying storage format from the user, and make the accessor cross-platform portable. In this document we present the design of an accessor that aims at fulfilling all these requirements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Ultrafast Error-bounded Lossy Compression for Scientific Datasets

Today's scientific high-performance computing applications and advanced instruments are producing vast volumes of data across a wide range of domains, which impose a serious burden on data transfer and storage. Error-bounded lossy compression has been developed and widely used in the scientific community because it not only can significantly reduce the data volumes but also can strictly control the data distortion based on the user-specified error bound. Existing lossy compressors, however, cannot offer ultrafast compression speed, which is highly demanded by numerous applications or use cases (such as in-memory compression and online instrument data compression). In this paper, we propose a novel ultrafast error-bounded lossy compressor that can obtain fairly high compression performance on both CPUs and GPUs and with reasonably high compression ratios. The key contributions are threefold. (1) We propose a generic error-bounded lossy compression framework---called SZx---that achieves ultrafast performance through its novel design comprising only lightweight operations such as bitwise and addition/subtraction operations, while still keeping a high compression ratio. (2) We implement SZx on both CPUs and GPUs and optimize the performance according to their architectures. (3) We perform a comprehensive evaluation with six real-world production-level scientific datasets on both CPUs and GPUs. Experiments show that SZx is 2~16x faster than the second-fastest existing error-bounded lossy compressor (either SZ or ZFP) on CPUs and GPUs, with respect to both compression and decompression.

Yu, Xiaodong↗