Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Progress toward Accelogic compression in ROOT

For the last 7 years, Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom-line outcome will be to provide state-of-the-art compression algorithms --and accompanying software libraries-- able to surpass the performance of the compression engines currently available in the ROOT [7] framework. The resulting technology has the capability to enable substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4 [6].As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.In this contribution, we will present our efforts integrating our concepts of “functionally lossless compression” within the ROOT framework implementation, with the purpose of producing a basic solution readily integrated into HENP applications. We will also present our progress applying this compression through realistic examples of analysis from both the STAR and CMS experiments.

Canal, Ph.↗
NASA NTRS1995

A spatially adaptive spectral re-ordering technique for lossless coding of hyper-spectral images

In this paper, we propose a new approach, applicable to lossless compression of hyper-spectral images, that alleviates some limitations of linear prediction as applied to this problem. According to this approach, an adaptive re-ordering of the spectral components of each pixel is performed prior to prediction and encoding. This re-ordering adaptively exploits, on a pixel-by pixel basis, the presence of inter-band correlations for prediction. Furthermore, the proposed approach takes advantage of spatial correlations, and does not introduce any coding overhead to transmit the order of the spectral bands. This is accomplished by using the assumption that two spatially adjacent pixels are expected to have similar spectral relationships. We thus have a simple technique to exploit spectral and spatial correlations in hyper-spectral data sets, leading to compression performance improvements as compared to our previously reported techniques for lossless compression. We also look at some simple error modeling techniques for further exploiting any structure that remains in the prediction residuals prior to entropy coding.

Memon, Nasir D.↗
DOE OSTI2024

Transfer Tube Material Properties Obtained from Test Data

A series of tension and compression tests were performed on typical transfer tubes made of 304L corrosion-resistant steel. The results were compiled and converted to true stress and true plastic strain for comparison to numerical models. The parameters of a Johnson-Cook constitutive model and fracture criterion were calibrated to mimic the test data. Comparisons were made between the converted data and the empirical formulas in the Johnson-Cook model. Simulations were performed to evaluate the implementation of the formula in finite element analysis, and the parameters were adjusted to improve the correlation.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Enhanced Tensor Completion Based Approaches for State Estimation in Distribution Systems

Grid state estimation is essential for effective control and management of distribution systems. While weighted least squares has been the conventional method for state estimation, sparsity-aware methods have become popular due to their superior performance with limited data. Matrix completion and compressed sensing-based state estimation approaches exploit the underlying smoothness in the state variables. However, classic matrix completion methods do not take into account the temporal correlation of system states. Compressed sensing methods, on the other hand, require an appropriate choice of sparsifying basis that may not be easy to identify. This paper proposes a blocktensor completion based framework which uses an alternative approach to estimate voltage phasor, power injections and branch currents. This approach utilizes the temporal correlation of the system states in a tensor trace-norm minimization formulation with power flow equations as constraints. Herein, feature scaling is introduced in the problem formulation to benefit from the improved sensitivity of the tensor trace norm to the matrix columns in the scaled unfoldings of the tensor. Weighted tensor norm is utilized to exploit the structures of the different unfoldings of the state measurement tensor to improve the voltage estimation. The estimation accuracy is further improved by alternatively estimating the tensor columns and increasing the available data at each stage in the tensor completion process. The proposed methods are evaluated on the IEEE-33, 37 test systems and a 100- node test system. The proposed methods are shown to provide significant performance gains relative to the classic matrix and tensor completion based approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

An Efficient Approach to Collect Inlet Characterization Data

This report considers a novel process to acquire performance data for supersonic mixed compression inlets that is faster, and therefore less costly, than the previous one. In the previous process, measurements were recorded in discrete increments. The backpressuring actuator was advanced, there was a pause to allow for airflow fluctuations to diminish, and then the measurements were recorded. While this process was methodical, reliable and accurate, it consumed significant wind-on time. In general, high-speed wind tunnels are expensive to operate considering maintenance, energy and human resource requirements. Furthermore, many facilities, such as blowdown wind tunnels, are limited in the duration that they can maintain the desired test condition. Consequently, a quick method to collect the measurements is desired. A favored approach has the actuator moving in a continuous fashion with measurements recorded as the actuator continuously progresses through the increment points. This approach, identified as the dynamic inlet characteristic data acquisition procedure, uses in situ dynamic high-speed pressure sensors to measure pressure signals. While faster, continuous movement adds dynamic effects to the data. To better understand these dynamic effects and other potential influences, a study was undertaken to compare wind tunnel measurements taken at discrete actuator position points with data gathered during continuous actuator movement. Specially, plots of inlet pressure recovery versus mass capture ratio, or total pressure characteristic curves, were examined. The data were measured during experiments in the NASA Glenn Research Center Abe Silverstein 10- by 10-Foot Supersonic Wind Tunnel (SWT) with an inlet propulsion test article. In this report, the wind tunnel experiment and the study objectives are described. The processes to reduce the data and for analysis are explained. A discussion of the results along with features noted in the data is given. Finally, conclusions from this study that may be used to guide future wind tunnel experiment planning are presented.

Inlet↗
NASA NTRSDate not supplied

An Efficient Approach to Collect Inlet Characterization Data

This report considers a novel process to acquire performance data for supersonic mixed compression inlets that is faster, and therefore less costly, than the previous one. In the previous process, measurements were recorded in discrete increments. The backpressuring actuator was advanced, there was a pause to allow for airflow fluctuations to diminish, and then the measurements were recorded. While this process was methodical, reliable and accurate, it consumed significant wind-on time. In general, high-speed wind tunnels are expensive to operate considering maintenance, energy and human resource requirements. Furthermore, many facilities, such as blowdown wind tunnels, are limited in the duration that they can maintain the desired test condition. Consequently, a quick method to collect the measurements is desired. A favored approach has the actuator moving in a continuous fashion with measurements recorded as the actuator continuously progresses through the increment points. This approach, identified as the dynamic inlet characteristic data acquisition procedure, uses in situ dynamic high-speed pressure sensors to measure pressure signals. While faster, continuous movement adds dynamic effects to the data. To better understand these dynamic effects and other potential influences, a study was undertaken to compare wind tunnel measurements taken at discrete actuator position points with data gathered during continuous actuator movement. Specially, plots of inlet pressure recovery versus mass capture ratio, or total pressure characteristic curves, were examined. The data were measured during experiments in the NASA Glenn Research Center Abe Silverstein 10- by 10-Foot Supersonic Wind Tunnel (SWT) with an inlet propulsion test article. In this report, the wind tunnel experiment and the study objectives are described. The processes to reduce the data and for analysis are explained. A discussion of the results along with features noted in the data is given. Finally, conclusions from this study that may be used to guide future wind tunnel experiment planning are presented.

Data Acquisition↗
NASA NTRS1985

On the use of wall functions as boundary conditions for two-dimensional separated compressible flows

A new and improved wall function method for compressible turbulent flows has been developed and tested. This method is applicable to attached and separated flows, to both high- and low-Reynolds number flows, and to flows with adiabatic and nonadiabatic surfaces. This wall function method has been applied to the Launder-Spalding k-epsilon two-equation model of turbulence. The tests consist of comparisons of calculated and experimental results for: (1) an axisymmetrical transonic shock-wave/boundary-wave interaction flow at low Reynolds number in an adiabatic tube, (2) an axisymmetrical high-Reynolds number transonic flow over a nonadiabatic bump, and (3) a two-dimensional supersonic high-Reynolds number flow on a nonadiabatic deflected flap. Each of these experiments had significant regions of flow separation. The calculations are performed with an implicit algorithm that solves the Reynolds-averaged Navier-Stokes equations. It is shown that the results obtained agree very well with the data for the complex compressible flows tested.

Viegas, J. R.↗
DOE OSTI2022

SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Here, error-bounded lossy compression has been considered one of the most effective solutions to the above problem. In practice, however, the best-fit compression method often needs to be customized or optimized in particular because of diverse characteristics in different datasets and various user requirements on the compression quality and performance. In this paper, we address this issue with a novel modular, composable compression framework named SZ3. Our contributions are four-folds. (1) We develop SZ3 which features an innovative modular abstraction for the prediction-based compression framework, such that compression modules can be plugged in easily to create new compressors based on characteristics of data and user requirements. (2) We create a new compression pipeline by SZ3 for GAMESS data, which significantly improves the compression ratios over state-of-the-art compressors. (3) We develop an adaptive compression pipeline by SZ3 for APS data with minimal efforts, which leads to the best rate-distortion among all existing error-bounded lossy compressors for any bit-rate. (4) We compare the sustainability of SZ3 with leading error-bounded prediction-based compressors, and then demonstrate the necessity of diverse pipelines by integrating and evaluating several compression pipelines on diverse scientific datasets from multiple disciplines. Experiments show that SZ3 incurs very limited overhead in compressor integration and our customized compression pipelines lead to up to 20% improvement in compression ratios under the same data distortion, when compared with the best existing approach.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Compressed/reconstructed test images for CRAF/Cassini

A set of compressed, then reconstructed, test images submitted to the Comet Rendezvous Asteroid Flyby (CRAF)/Cassini project is presented as part of its evaluation of near lossless high compression algorithms for representing image data. A total of seven test image files were provided by the project. The seven test images were compressed, then reconstructed with high quality (root mean square error of approximately one or two gray levels on an 8 bit gray scale), using discrete cosine transforms or Hadamard transforms and efficient entropy coders. The resulting compression ratios varied from about 2:1 to about 10:1, depending on the activity or randomness in the source image. This was accomplished without any special effort to optimize the quantizer or to introduce special postprocessing to filter the reconstruction errors. A more complete set of measurements, showing the relative performance of the compression algorithms over a wide range of compression ratios and reconstruction errors, shows that additional compression is possible at a small sacrifice in fidelity.

Dolinar, S.↗
NASA NTRS2025

Influence of Actuator Movement Rate During Acquisition of Inlet Characterization Data

This report considers using a process to acquire performance data for supersonic mixed-compression inlets that is faster and less costly than the previous process. In the previous process for acquiring data during wind tunnel experiments, actuators for backpressuring an inlet moved in discrete increments, paused for dynamics to dampen, measured data for 5 s, recorded the data, and then reduced the data to graphic plots of pressure recovery versus mass capture ratio. This discrete data acquisition procedure was methodical, reliable, and accurate. However, acquiring data to characterize the inlet consumed significant wind-on time. Wind tunnel operation was costly when considering maintenance, energy, and human resources (operators and research team); therefore, the price for each data point was high. This report presents a quicker method to obtain all measurements, one which can also be used in blowdown wind tunnels. This approach, identified as the dynamic inlet characteristic data acquisition procedure, uses in situ dynamic high-speed pressure sensors to measure pressure signals. The backpressuring actuator moves continuously, and measurements from high-speed pressure sensors are recorded as the actuator progresses to and past the incremental points. Despite its advantages, the approach raised the following questions: (1) Will the new process be as accurate as the previous process? and (2) Are there any hidden costs? These concerns are addressed in this report. This report reviews a study that was undertaken to compare wind tunnel experiment data taken using the previous discrete data acquisition procedure with data recorded using the dynamic inlet characterization data acquisition procedure. In this report, the wind tunnel experiment and the study objectives are described. The processes to reduce the data for analysis are explained, and a discussion of the results is given. Finally, conclusions from this study that may guide further work in this area are presented.

Inlet Characterization Curves↗
DOE OSTI2023

TDD LoRa and Delta Encoding in Low-Power Networks of Environmental Sensor Arrays for Temperature and Deformation Monitoring: Supporting Data

This dataset was used to assess the performance of a compression algorithm for Soil Deformation and Temperature Monitoring data. The development of this technology is presented in the article named time division duplex (TDD) "TDD LoRa and Delta Encoding in Low-Power Networks of Environmental Sensor Arrays for Temperature and Deformation Monitoring" and published in Springer's Journal on Signal Processing Systems. [Time Division Duplex (TDD); Long Range (LoRa)]Field experiment data includes: 1) soil temperature measurements, 2) snow temperature data, 3) soil temperature and deformation data. In these files, the first column contains the time (UTC), followed by the sensor's battery voltage and temperature and acceleration values (X, Y, Z) in subsequent columns. The measurements were acquired every 15 minutes. Lists included for probe, datalogger, and packaging components. The two temperature data sets were acquired at a site at the East River Watershed, Colorado, while the temperature/deformation data were collected at the Teller Road (mile 27) site near Nome, Alaska. The package includes 8 comma-separated files (.csv) and one .pdf file.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1991

A two-equation turbulence model for compressible reacting flows

A two-equation turbulence model (k-epsilon) has been modified to model compressible reacting flows. A compressibility correction model based on modeling the dilatational terms in the Reynolds stress equations has been used. Computations are carried out using the Navier-Stokes solver SPARK which includes a finite rate chemistry model for hydrogen-air combustion. Initially, planar compressible mixing layers are simulated. Comparisons of the predictions with available experimental data and the predictions of a compressible Reynolds stress closure indicate that the model is well suited for the study of such flows. The decrease in the growth rate of the mixing layer with increasing convective Mach number is well predicted. The model is then tested on a three-dimensional supersonic flow problem where solid boundaries are present. An axisymmetric nonreacting mixing layer flow involving dissimilar gases is evaluated next using the model and the predictions are compared with available experimental data. Finally, two reacting turbulent flow configurations have been studied and comparison of predictions and experimental data has been done. The comparisons indicate that the model predicts the overall features of the test cases very well.

Narayan, J. R.↗
NASA NTRS2013

Towards Efficient Scientific Data Management Using Cloud Storage

A software prototype allows users to backup and restore data to/from both public and private cloud storage such as Amazon's S3 and NASA's Nebula. Unlike other off-the-shelf tools, this software ensures user data security in the cloud (through encryption), and minimizes users operating costs by using space- and bandwidth-efficient compression and incremental backup. Parallel data processing utilities have also been developed by using massively scalable cloud computing in conjunction with cloud storage. One of the innovations in this software is using modified open source components to work with a private cloud like NASA Nebula. Another innovation is porting the complex backup to- cloud software to embedded Linux, running on the home networking devices, in order to benefit more users.

He, Qiming↗
DOE OSTI2021

cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions

Lossy compression is becoming an indispensable technique for the success of today's extreme-scale high-performance computing projects that produce vast volumes of data during scientific simulations or instrument data acquisitions. Comprehensively understanding the compression quality and performance of different lossy compressors is critical to selecting the best-fit compressors and using them properly and efficiently in practice. A few lossy compression assessment tools (e.g., Z-checker) have been developed, but none of them support the execution in a GPU environment. This is a significant gap because many recent extreme-scale applications and lossy compressors (e.g., cuSZ) can run entirely within GPUs. In this work, we develop an efficient lossy compression measuring system (called cuZ-Checker) on the GPU platform, which aims to perform the lossy compression quality and performance assessment completely within the GPU environment. Our contribution is threefold. (1) We develop a novel GPU-based lossy compression measuring framework using a computation pattern-based design approach. This approach classifies the computing-intensive metrics into three categories based on their patterns which creates large opportunities for kernel fusion and data reuse. (2) For each pattern in cuZ-Checker, we develop a CUDA kernel and provide fine-grained optimizations to boost its performance. (3) We thoroughly evaluate our cuZ-checker on a V100 GPU using four real-world scientific application datasets. Experiments show that cuZ-Checker can significantly accelerate the overall lossy compression assessment performance by 23X similar to 31X compared with the OpenMP-based multithreading CPU performance. To the best of our knowledge, this is the first lossy compression measuring system designed for GPU devices.

GPU↗
DOE OSTI2017

Fast Quasi-Static Time-Series (QSTS) for yearlong PV impact studies using vector quantization

The rapidly growing penetration levels of distributed photovoltaic (PV) systems requires more comprehensive studies to understand their impact on distribution feeders. IEEE P.1547 highlights the need for Quasi-Static Time Series (QSTS) simulation in conducting distribution impact studies for distributed resource interconnection. Unlike conventional scenario-based simulation, the time series simulation can realistically assess time-dependent impacts such as the operation of various controllable elements (e.g. voltage regulating tap changers) or impacts of power fluctuations. However, QSTS simulations are still not widely used in the industry because of the computational burden associated with running yearlong simulations at a 1-s granularity, which is needed to capture device controller effects responding to PV variability. This study presents a novel algorithm that reduces the number of times that the non-linear 3-phase unbalanced AC power flow must be solved by storing and reassigning power flow solutions as it progresses through the simulation. Each unique power flow solution is defined by a set of factors affecting the solution that can easily be queried. We demonstrate a computational time reduction of 98.9% for a yearlong simulation at 1-s resolution with minimal errors for metrics including: number of tap changes, capacitor actions, highest and lowest voltage on the feeder, line losses, and ANSI voltage violations. Finally, the key contribution of this work is the formulation of an algorithm capable of: (i) drastically reducing the computational time of QSTS simulations, (ii) accurately modeling distribution system voltage-control elements with hysteresis, and (iii) efficiently compressing result time series data for post-simulation analysis.

14 SOLAR ENERGY↗
DOE OSTI2020

X-ray diffraction at the National Ignition Facility

We report details of an experimental platform implemented at the National Ignition Facility to obtain in situ powder diffraction data from solids dynamically compressed to extreme pressures. Thin samples are sandwiched between tamper layers and ramp compressed using a gradual increase in the drive-laser irradiance. Pressure history in the sample is determined using high-precision velocimetry measurements. Up to two independently timed pulses of x rays are produced at or near the time of peak pressure by laser illumination of thin metal foils. The quasi-monochromatic x-ray pulses have a mean wavelength selectable between 0.6 Å and 1.9 Å depending on the foil material. The diffracted signal is recorded on image plates with a typical 2 θ x-ray scattering angle uncertainty of about 0.2° and resolution of about 1°. Analytic expressions are reported for systematic corrections to 2 θ due to finite pinhole size and sample offset. A new variant of a nonlinear background subtraction algorithm is described, which has been used to observe diffraction lines at signal-to-background ratios as low as a few percent. Variations in system response over the detector area are compensated in order to obtain accurate line intensities; this system response calculation includes a new analytic approximation for image-plate sensitivity as a function of photon energy and incident angle. This experimental platform has been used up to 2 TPa (20 Mbar) to determine the crystal structure, measure the density, and evaluate the strain-induced texturing of a variety of compressed samples spanning periods 2–7 on the periodic table.

47 OTHER INSTRUMENTATION↗