Search NASA⌕ Search

SEARCH · Search NASA

Results for “compression algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Fast Algorithms for Scientific Data Compression

Many scientific simulations and experiments generate terabytes to petabytes of data daily, necessitating data compression techniques. Unlike video and image compression, scientists require methods that accurately preserve primary data (PD) and derived quantities of interest (QoIs). In our previous work, we demonstrated the effectiveness of hybrid compression techniques that combine machine learning with traditional approaches. This paper presents innovative computational techniques aimed at expediting the compression pipeline. Our experiments, conducted on two distinct platforms with a large-scale XGC-based fusion simulation, demonstrate that the overhead incurred by these new approaches is less than one percent of the computational resources needed for the simulation.

Banerjee, Tania↗
DOE OSTI2015

Dynamic metamaterial aperture for microwave imaging

We present a dynamic metamaterial aperture for use in computational imaging schemes at microwave frequencies. The aperture consists of an array of complementary, resonant metamaterial elements patterned into the upper conductor of a microstrip line. Each metamaterial element contains two diodes connected to an external control circuit such that the resonance of the metamaterial element can be damped by application of a bias voltage. Through applying different voltages to the control circuit, select subsets of the elements can be switched on to create unique radiation patterns that illuminate the scene. Spatial information of an imaging domain can thus be encoded onto this set of radiation patterns, or measurements, which can be processed to reconstruct the targets in the scene using compressive sensing algorithms. We discuss the design and operation of a metamaterial imaging system and demonstrate reconstructed images with a 10:1 compression ratio. Dynamic metamaterial apertures can potentially be of benefit in microwave or millimeter wave systems such as those used in security screening and through-wall imaging. In addition, feature-specific or adaptive imaging can be facilitated through the use of the dynamic aperture.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Lifting MGARD: Construction of (pre)wavelets on the interval using polynomial predictors of arbitrary order

MGARD (MultiGrid Adaptive Reduction of Data) is an algorithm for compressing and refactoring scientific data, based on the theory of multigrid methods. The core algorithm is built around stable multilevel decompositions of conforming piecewise linear $C^0$ finite element spaces, enabling accurate error control in various norms and derived quantities of interest. In this work, we extend this construction to arbitrary order Lagrange finite elements $\mathbb{Q}_p$, $p \geq 0$, and propose a reformulation of the algorithm as a lifting scheme with polynomial predictors of arbitrary order. Additionally, a new formulation using a compactly supported wavelet basis is discussed, and an explicit construction of the proposed wavelet transform for uniform dyadic grids is described.

Reshniak, Viktor [Oak Ridge National Laboratory (O↗
DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Reverse-Mode Algorithmic Differentiation of an OpenMP-Parallel Compressible Flow Solver

y Reverse-mode algorithmic differentiation (AD) is an established method for obtaining adjoint derivatives of computer simulation applications. In computational fluid dynamics (CFD), adjoint derivatives of a cost function output such as drag or lift with respect to design parameters such as surface coordinates or geometry control points are a key ingredient for shape optimization, uncertainty quantification and flow control. The computational cost of CFD applications and their derivatives makes it essential to use high-performance computing hardware efficiently, including multi- and many-core architectures. Nevertheless, OpenMP is not supported in most AD tools, and previously shown methods achieve poor scalability of the derivative code. We present the AD of an OpenMP-parallelized finite volume compressible flow solver for unstructured meshes. Our approach enables us to reuse the parallelization of the original code in the computation of adjoint derivatives. The method works by identifying code segments that can be differentiated in reverse-mode without changing their memory access pattern. The OpenMP parallelization is integrated into the derivative code during the build process in a way that is robust to modifications of the original code and independent of the OpenMP support of the differentiation tool. We show the scalability of our adjoint CFD solver on test cases ranging from thousands to millions of finite volume mesh cells on CPUs with up to 16 threads as well as on an Intel XeonPhi card with 236 threads. We demonstrate that our approach is more practical to implement for production-sized CFD codes and produces more efficient adjoint derivative code than previously shown AD methods.

Huckelheim, Jan↗
DOE OSTI2012

Advances in compact proton spectrometers for inertial-confinement fusion and plasma nuclear science

Compact wedge-range-filter proton spectrometers cover proton energies {approx}3-20 MeV. They have been used at the OMEGA laser facility for more than a decade for measuring spectra of primary D{sup 3}He protons in D{sup 3}He implosions, secondary D{sup 3}He protons in DD implosions, and ablator protons in DT implosions; they are now being used also at the National Ignition Facility. The spectra are used to determine proton yields, shell areal density at shock-bang time and compression-bang time, fuel areal density, and implosion symmetry. There have been changes in fabrication and in analysis algorithms, resulting in a wider energy range, better accuracy and precision, and better robustness for survivability with indirect-drive inertial-confinement-fusion experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

A Well-Posed Two Phase Flow Model and its Numerical Solutions for Reactor Thermal-Fluids Analysis

A 7-equation two-phase flow model and its numerical implementation is presented for reactor thermal-fluids applications. The equation system is well-posed and treats both phases as compressible flows. The numerical discretization of the equation system is based on the finite element formalism. The numerical algorithm is implemented in the next generation RELAP-7 code (Idaho National Laboratory (INL)’s thermal-fluids code) built on top of an other INL’s product, the massively parallel multi-implicit multi-physics object oriented code environment (MOOSE). Some preliminary thermal-fluids computations are presented.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2014

Numerical simulation of the dynamics of a liquid crystal in the case of plane strain using GPUs

Parallel computational algorithm is worked out for numerical implementation of two-dimensional dynamic model of a liquid crystal, which takes into account the mutual influence of three physical effects – transfer of acoustic energy due to translational motion, viscoelastic rotation of the particles under the action of tangential stresses, anisotropic thermal expansion and compression. The algorithm is based on the Godunov gap decay method, the Ivanov scheme with controlled dissipation of energy and the splitting method with respect to spatial variables. The CUDA technology for computer systems with graphic accelerators is used. Results of computations demonstrating the efficiency of proposed method and algorithm are represented.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

CEAZ: Accelerating Parallel I/O Via Hardware-Algorithm Co-Designed Adaptive Lossy Compression

As supercomputers continue to grow to exa-scale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the I/O performance. However, little work has been done for effectively offloading lossy compression onto FPGA-based SmartNICs to reduce the compression overhead. In this paper, we propose a hardware-algorithm co-design of efficient and adaptive lossy compressor for scientific data on FPGAs (called CEAZ) to accelerate parallel I/O. Our contribution is fourfold: (1) We propose an efficient Huffman coding approach that can adaptively update Huffman codewords online based on codewords generated offline (from a variety of representative scientific datasets). (2) We derive a theoretical analysis to support a precise control of compression ratio under an error-bounded compression mode, enabling accurate offline Huffman codewords generation. This also help us create a fixed-ratio compression mode for consistent throughput. (3) We develop an efficient compression pipeline by adopting cuSZ’s dual-quantization algorithm to our hardware use case. (4) We evaluate CEAC on five real-world datasets with both a single FPGA board and 256 nodes from Bridges2 supercomputer. Experiments show that CEAZ outperforms the second-best FPGA-based lossy compressor by 2× of throughput and 9.6× of compression ratio. It also improves MPI_File_write and MPI_Gather throughputs by up to 32.7× and 31.4×, respectively.

Zhang, Chengming↗
DOE OSTI2025

Thermal Radiation Transport with Tensor Trains

We present a novel tensor network algorithm to solve the time-dependent, gray thermal radiation transport equation. The method invokes a tensor train (TT) decomposition for the specific intensity. The efficiency of this approach is dictated by the rank of the decomposition. When the solution is “low rank,” the memory footprint of the specific intensity solution vector may be significantly compressed. The algorithm, following a step-then-truncate approach of a traditional discrete ordinates method, operates directly on the compressed state vector, thereby enabling large speedups for low-rank solutions. To achieve these speedups, we rely on a recently developed rounding approach based on the Gram-SVD. We detail how familiar S N algorithms for (gray) thermal transport can be mapped to this TT framework and present several numerical examples testing both the optically thick and thin regimes. The TT framework finds low-rank structure and supplies up to ≃60× speedups and ≃1000× compressions for problems demanding large angle counts, thereby enabling previously intractable SN calculations and supplying a promising avenue to mitigate ray effects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Resource-aware compression

Systems, apparatuses, and methods for implementing a multi-tiered approach to cache compression are disclosed. A cache includes a cache controller, light compressor, and heavy compressor. The decision on which compressor to use for compressing cache lines is made based on certain resource availability such as cache capacity or memory bandwidth. This allows the cache to opportunistically use complex algorithms for compression while limiting the adverse effects of high decompression latency on system performance. To address the above issue, the proposed design takes advantage of the heavy compressors for effectively reducing memory bandwidth in high bandwidth memory (HBM) interfaces as long as they do not sacrifice system performance. Accordingly, the cache combines light and heavy compressors with a decision-making unit to achieve reduced off-chip memory traffic without sacrificing system performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression

We present memory-efficient and scalable algorithms for kernel methods used in machine learning. Using hierarchical matrix approximations for the kernel matrix the memory requirements, the number of floating point operations, and the execution time are drastically reduced compared to standard dense linear algebra routines. We consider both the general H matrix hierarchical format as well as Hierarchically Semi-Separable (HSS) matrices. Furthermore, we investigate the impact of several preprocessing and clustering techniques on the hierarchical matrix compression. Effective clustering of the input leads to a ten-fold increase in efficiency of the compression. The algorithms are implemented using the STRUMPACK solver library. These results confirm that - with correct tuning of the hyperparameters - classification using kernel ridge regression with the compressed matrix does not lose prediction accuracy compared to the exact - not compressed - kernel matrix and that our approach can be extended to O(1M) datasets, for which computation with the full kernel matrix becomes prohibitively expensive. We present numerical experiments in a distributed memory environment up to 1,024 processors of the NERSC's Cori supercomputer using well-known datasets to the machine learning community that range from dimension 8 up to 784.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Genetic algorithm-based geometry calibration for dynamic compression x-ray diffraction experiments

An important component of dynamic compression x-ray diffraction (XRD) experiment analysis is geometry calibration: proper data interpretation requires knowledge of the precise detector position and orientation and, if the experiment involves a single-crystal sample, knowledge of the lattice orientation. The determination of these parameters in the arbitrary three-dimensional (3D) scattering geometries often present in dynamic compression facilities is challenging, as the associated optimization problem can be highly nonlinear, nonsmooth, and discontinuous. We present a genetic algorithm-based approach for performing dynamic compression XRD calibrations that overcomes these obstacles. We provide details regarding the image processing, algorithm implementation, and open-source software deployment and demonstrate the capability of the approach to calibrate the detector and crystal parameters in 3D geometries. Notably, we demonstrate the solver’s capacity to find the crystal orientation without a priori rotation constraints.

Brown, Nathan P. [Sandia National Laboratories (SN↗
DOE OSTI2022

Algebraic compression of quantum circuits for Hamiltonian evolution

Here unitary evolution under a time-dependent Hamiltonian is a key component of simulation on quantum hardware. Synthesizing the corresponding quantum circuit is typically done by breaking the evolution into small time steps, also known as Trotterization, which leads to circuits the depth of which scales with the number of steps. When the circuit elements are limited to a subset of SU(4) - or equivalently, when the Hamiltonian may be mapped onto free fermionic models - several identities exist that combine and simplify the circuit. Based on this, we present an algorithm that compresses the Trotter steps into a single block of quantum gates using algebraic relations between adjacent circuit elements. This results in a fixed depth time evolution for certain classes of Hamiltonians. We explicitly show how this algorithm works for several spin models, and demonstrate its use for adiabatic state preparation of the transverse field Ising model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

A Compressed Sensing-based Image Reconstruction Algorithm for Solar Flare X-Ray Observations

One way of imaging X-ray emission from solar flares is to measure Fourier components of the spatial X-ray source distribution. We present a new compressed sensing-based algorithm named VIS-CS, which reconstructs the spatial distribution from such Fourier components. We demonstrate the application of the algorithm on synthetic and observed solar flare X-ray data from the Reuven Ramaty High Energy Solar Spectroscopic Imager satellite and compare its performance with existing algorithms. VIS-CS produces competitive results with accurate photometry and morphology, without requiring any algorithm- and X-ray-source-specific parameter tuning. Its robustness and performance make this algorithm ideally suited for the generation of quicklook images or large image cubes without user intervention, such as for imaging spectroscopy analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Compressive sensing adaptation for polynomial chaos expansions

Basis adaptation in Homogeneous Chaos spaces rely on a suitable rotation of the underlying Gaussian germ. Several rotations have been proposed in the literature resulting in adaptations with different convergence properties. Here, we present a new adaptation mechanism that builds on compressive sensing algorithms, resulting in a reduced polynomial chaos approximation with optimal sparsity. The developed adaptation algorithm consists of a two-step optimization procedure that computes the optimal coefficients and the input projection matrix of a low dimensional chaos expansion with respect to an optimally rotated basis. Finally, we demonstrate the attractive features of our algorithm through several numerical examples including the application on Large-Eddy Simulation (LES) calculations of turbulent combustion in a HIFiRE scramjet engine.

42 ENGINEERING↗