Search NASA⌕ Search

SEARCH · Search NASA

Results for “compression algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Reverse-Mode Algorithmic Differentiation of an OpenMP-Parallel Compressible Flow Solver

y Reverse-mode algorithmic differentiation (AD) is an established method for obtaining adjoint derivatives of computer simulation applications. In computational fluid dynamics (CFD), adjoint derivatives of a cost function output such as drag or lift with respect to design parameters such as surface coordinates or geometry control points are a key ingredient for shape optimization, uncertainty quantification and flow control. The computational cost of CFD applications and their derivatives makes it essential to use high-performance computing hardware efficiently, including multi- and many-core architectures. Nevertheless, OpenMP is not supported in most AD tools, and previously shown methods achieve poor scalability of the derivative code. We present the AD of an OpenMP-parallelized finite volume compressible flow solver for unstructured meshes. Our approach enables us to reuse the parallelization of the original code in the computation of adjoint derivatives. The method works by identifying code segments that can be differentiated in reverse-mode without changing their memory access pattern. The OpenMP parallelization is integrated into the derivative code during the build process in a way that is robust to modifications of the original code and independent of the OpenMP support of the differentiation tool. We show the scalability of our adjoint CFD solver on test cases ranging from thousands to millions of finite volume mesh cells on CPUs with up to 16 threads as well as on an Intel XeonPhi card with 236 threads. We demonstrate that our approach is more practical to implement for production-sized CFD codes and produces more efficient adjoint derivative code than previously shown AD methods.

Huckelheim, Jan↗
DOE OSTI2022

CEAZ: Accelerating Parallel I/O Via Hardware-Algorithm Co-Designed Adaptive Lossy Compression

As supercomputers continue to grow to exa-scale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the I/O performance. However, little work has been done for effectively offloading lossy compression onto FPGA-based SmartNICs to reduce the compression overhead. In this paper, we propose a hardware-algorithm co-design of efficient and adaptive lossy compressor for scientific data on FPGAs (called CEAZ) to accelerate parallel I/O. Our contribution is fourfold: (1) We propose an efficient Huffman coding approach that can adaptively update Huffman codewords online based on codewords generated offline (from a variety of representative scientific datasets). (2) We derive a theoretical analysis to support a precise control of compression ratio under an error-bounded compression mode, enabling accurate offline Huffman codewords generation. This also help us create a fixed-ratio compression mode for consistent throughput. (3) We develop an efficient compression pipeline by adopting cuSZ’s dual-quantization algorithm to our hardware use case. (4) We evaluate CEAC on five real-world datasets with both a single FPGA board and 256 nodes from Bridges2 supercomputer. Experiments show that CEAZ outperforms the second-best FPGA-based lossy compressor by 2× of throughput and 9.6× of compression ratio. It also improves MPI_File_write and MPI_Gather throughputs by up to 32.7× and 31.4×, respectively.

Zhang, Chengming↗
DOE OSTI2025

Thermal Radiation Transport with Tensor Trains

We present a novel tensor network algorithm to solve the time-dependent, gray thermal radiation transport equation. The method invokes a tensor train (TT) decomposition for the specific intensity. The efficiency of this approach is dictated by the rank of the decomposition. When the solution is “low rank,” the memory footprint of the specific intensity solution vector may be significantly compressed. The algorithm, following a step-then-truncate approach of a traditional discrete ordinates method, operates directly on the compressed state vector, thereby enabling large speedups for low-rank solutions. To achieve these speedups, we rely on a recently developed rounding approach based on the Gram-SVD. We detail how familiar S N algorithms for (gray) thermal transport can be mapped to this TT framework and present several numerical examples testing both the optically thick and thin regimes. The TT framework finds low-rank structure and supplies up to ≃60× speedups and ≃1000× compressions for problems demanding large angle counts, thereby enabling previously intractable SN calculations and supplying a promising avenue to mitigate ray effects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Resource-aware compression

Systems, apparatuses, and methods for implementing a multi-tiered approach to cache compression are disclosed. A cache includes a cache controller, light compressor, and heavy compressor. The decision on which compressor to use for compressing cache lines is made based on certain resource availability such as cache capacity or memory bandwidth. This allows the cache to opportunistically use complex algorithms for compression while limiting the adverse effects of high decompression latency on system performance. To address the above issue, the proposed design takes advantage of the heavy compressors for effectively reducing memory bandwidth in high bandwidth memory (HBM) interfaces as long as they do not sacrifice system performance. Accordingly, the cache combines light and heavy compressors with a decision-making unit to achieve reduced off-chip memory traffic without sacrificing system performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression

We present memory-efficient and scalable algorithms for kernel methods used in machine learning. Using hierarchical matrix approximations for the kernel matrix the memory requirements, the number of floating point operations, and the execution time are drastically reduced compared to standard dense linear algebra routines. We consider both the general H matrix hierarchical format as well as Hierarchically Semi-Separable (HSS) matrices. Furthermore, we investigate the impact of several preprocessing and clustering techniques on the hierarchical matrix compression. Effective clustering of the input leads to a ten-fold increase in efficiency of the compression. The algorithms are implemented using the STRUMPACK solver library. These results confirm that - with correct tuning of the hyperparameters - classification using kernel ridge regression with the compressed matrix does not lose prediction accuracy compared to the exact - not compressed - kernel matrix and that our approach can be extended to O(1M) datasets, for which computation with the full kernel matrix becomes prohibitively expensive. We present numerical experiments in a distributed memory environment up to 1,024 processors of the NERSC's Cori supercomputer using well-known datasets to the machine learning community that range from dimension 8 up to 784.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Genetic algorithm-based geometry calibration for dynamic compression x-ray diffraction experiments

An important component of dynamic compression x-ray diffraction (XRD) experiment analysis is geometry calibration: proper data interpretation requires knowledge of the precise detector position and orientation and, if the experiment involves a single-crystal sample, knowledge of the lattice orientation. The determination of these parameters in the arbitrary three-dimensional (3D) scattering geometries often present in dynamic compression facilities is challenging, as the associated optimization problem can be highly nonlinear, nonsmooth, and discontinuous. We present a genetic algorithm-based approach for performing dynamic compression XRD calibrations that overcomes these obstacles. We provide details regarding the image processing, algorithm implementation, and open-source software deployment and demonstrate the capability of the approach to calibrate the detector and crystal parameters in 3D geometries. Notably, we demonstrate the solver’s capacity to find the crystal orientation without a priori rotation constraints.

Brown, Nathan P. [Sandia National Laboratories (SN↗
DOE OSTI2022

Algebraic compression of quantum circuits for Hamiltonian evolution

Here unitary evolution under a time-dependent Hamiltonian is a key component of simulation on quantum hardware. Synthesizing the corresponding quantum circuit is typically done by breaking the evolution into small time steps, also known as Trotterization, which leads to circuits the depth of which scales with the number of steps. When the circuit elements are limited to a subset of SU(4) - or equivalently, when the Hamiltonian may be mapped onto free fermionic models - several identities exist that combine and simplify the circuit. Based on this, we present an algorithm that compresses the Trotter steps into a single block of quantum gates using algebraic relations between adjacent circuit elements. This results in a fixed depth time evolution for certain classes of Hamiltonians. We explicitly show how this algorithm works for several spin models, and demonstrate its use for adiabatic state preparation of the transverse field Ising model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

A Compressed Sensing-based Image Reconstruction Algorithm for Solar Flare X-Ray Observations

One way of imaging X-ray emission from solar flares is to measure Fourier components of the spatial X-ray source distribution. We present a new compressed sensing-based algorithm named VIS-CS, which reconstructs the spatial distribution from such Fourier components. We demonstrate the application of the algorithm on synthetic and observed solar flare X-ray data from the Reuven Ramaty High Energy Solar Spectroscopic Imager satellite and compare its performance with existing algorithms. VIS-CS produces competitive results with accurate photometry and morphology, without requiring any algorithm- and X-ray-source-specific parameter tuning. Its robustness and performance make this algorithm ideally suited for the generation of quicklook images or large image cubes without user intervention, such as for imaging spectroscopy analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Compressive sensing adaptation for polynomial chaos expansions

Basis adaptation in Homogeneous Chaos spaces rely on a suitable rotation of the underlying Gaussian germ. Several rotations have been proposed in the literature resulting in adaptations with different convergence properties. Here, we present a new adaptation mechanism that builds on compressive sensing algorithms, resulting in a reduced polynomial chaos approximation with optimal sparsity. The developed adaptation algorithm consists of a two-step optimization procedure that computes the optimal coefficients and the input projection matrix of a low dimensional chaos expansion with respect to an optimally rotated basis. Finally, we demonstrate the attractive features of our algorithm through several numerical examples including the application on Large-Eddy Simulation (LES) calculations of turbulent combustion in a HIFiRE scramjet engine.

42 ENGINEERING↗
DOE OSTI2020

Smart Scattering Scanning Near-Field Optical Microscopy

Scattering scanning near-field optical microscopy (s-SNOM) provides spectroscopic imaging from molecular to quantum materials with few nanometer deep subdiffraction limited spatial resolution. However, in its conventional implementation s-SNOM is slow to effectively acquire a series of spatio-spectral images, especially with large fields of view. This problem is further exacerbated for weak resonance contrast or when using light sources with limited spectral irradiance. Indeed, the generally limited signal-to-noise ratio prevents sampling a weak signal at the Nyquist sampling rate. Here, we demonstrate how acquisition time and sampling rate can be significantly reduced by using compressed sampling, matrix completion, and adaptive random sampling, while maintaining or even enhancing the physical or chemical image content. We use fully sampled real data sets of molecular, biological, and quantum materials as ground-truth physical data and show how deep under-sampling with a corresponding reduction of acquisition time by 1 order of magnitude or more retains the core s-SNOM image information. We demonstrate that a sampling rate of up to 6× smaller than the Nyquist criterion can be applied, which would provide a 30-fold reduction in the data required under typical experimental conditions. Furthermore, our smart s-SNOM approach is generally applicable and provides systematic full spatio-spectral s-SNOM imaging with a large field of view at high spectral resolution and reduced acquisition time.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Data-driven analysis of relight variability of jet fuels induced by turbulence

For safety purposes, reliable reignition of aircraft engines in the event of flame blow-out is a critical requirement. Typically, an external ignition source in the form of a spark is used to achieve a stable flame in the combustor. However, such forced turbulent ignition may not always successfully relight the combustor, mainly because the state of the combustor cannot be precisely determined. Uncertainty in the turbulent flow inside the combustor, inflow conditions, and spark discharge characteristics can lead to variability in sparking outcomes even for nominally identical operating conditions. Prior studies have shown that of all the uncertain parameters, turbulence is often dominant and can drastically alter ignition behavior. For instance, even when different fuels have similar ignition delay times, their ignition behavior in practical systems can be completely different. In practical operating conditions, it is challenging to understand why ignition fails and how much variation in outcomes can be expected. The focus of this work is to understand relight variability induced by turbulence for two different aircraft fuels, namely Jet-A and a variant named C1. A detailed, previously developed simulation approach is used to generate a large number of successful and failed ignition events. Using this data, the cause of misfire is evaluated based on a discriminant analysis that delineates the difference between turbulent initial conditions that lead to ignition or failure. From the discriminant analysis, a compressed sensing algorithm is then applied to help pinpoint the locations of relevant turbulent features. Findings from the discriminant analysis are confirmed with the time history of near kernel properties. Next, a clustering strategy is used to identify ignition and misfire modes. With this approach, it was determined that the cause of ignition failure is different for the two fuels. While it was found that Jet-A is influenced by fuel entrainment, C1 was found to be more sensitive to small scale turbulence features. Finally, a larger variability is found in the ignition modes of C1, which can be subject to extreme events induced by kernel breakdown.

42 ENGINEERING↗
DOE OSTI2021

4D Imaging of ZnO-Coated Nanoporous Al 2 O 3 Aerogels by Chemically Sensitive Ptychographic Tomography: Implications for Designer Catalysts

The 3D chemical structure (4D spectromicroscopy) of nanoporous Al 2 O 3 aerogels coated with ZnO by atomic layer deposition (ALD) was studied by multienergy scanning transmission X-ray microscopy. These materials are representative of a class of designer catalysts in which the nanoporous support is prepared separately from the active catalyst material, which is subsequently introduced by ALD, thereby allowing independent optimization of the morphology, chemistry, and spatial distribution of the support and catalyst. The samples studied were prepared by Ga ion and Xe plasma focused ion beam (FIB) milling as well as drop casting from water suspension. Zn L and Al K edge spectra of six samples with three different ZnO loadings were measured to investigate how loading and different sample preparation methods affect the 3D distribution of the ZnO and Al 2 O 3 . Scanning transmission X-ray microscopy (STXM) and ptychographic imaging at two energies each at the Zn L 3 and Al K edge were measured. The ptychography data were analyzed by using the SHARP reconstruction code to generate quantitative 2D chemical maps of the Al 2 O 3 and the ZnO. The STXM and ptychography maps were then measured at a sequence of tilt angles, covering up to 160° of rotation. The 3D structure of the ZnO and Al 2 O 3 was derived from the tilt series data by tomographic reconstruction using a compressed sensing algorithm. A twodimensional spatial resolution (half-period) of 6 nm, measured by Fourier ring correlation, and a 3D spatial resolution (half-period) of 9 nm, measured by Fourier shell correlation, were achieved when using the COSMIC beamline at the Advanced Light Source (ALS). The results show that for all of the ZnO loadings studied there is nonuniform coverage of the ZnO on the Al 2 O 3 aerogel framework. In addition, we found that both FIB methods create sample artifacts, although the distortion was less with Xe plasma than Ga ion FIB.

36 MATERIALS SCIENCE↗
DOE OSTI2020

3D Experimental Measurements of Evolution of Force Chains in Natural Silica Sand

The mechanisms of force transmission in granular materials is a classic physics problem that has been addressed since the 19th century, when Heinrich Rudolf Hertz investigated the interaction between two similar objects that were in contact under compression. However, the study of force transmission mechanisms in assemblies of more particles has proven to be a formidable problem due to the complex nature of granular materials. In recent years, synchrotron microcomputed tomography (SMT) and three-dimensional X-ray diffraction microscopy (3DXRD) have been employed to study the mechanics of granular materials experimentally. Combining SMT and 3DXRD offers unique three-dimensional (3D) experimental measurements of the internal structure, kinematics (such as rotation and translation), and lattice strains of individual sand particles. In this paper, in situ SMT and 3DXRD scans were acquired at multiple load steps for a specimen composed of 2,705 natural Ottawa sand particles that were subjected to one-dimensional (1D) confined compression. An algorithm was developed to combine SMT images and 3DXRD lattice strain measurements and used to characterize the constitutive behavior of sand particles. The results were used to identify the crystal structure and the evolution of the stresses and lattice strains of individual sand particles. Another algorithm was developed to characterize the force structures within the specimen. Force structures were identified, and their properties (such as length) and evolution through the experiment were examined. The contact number of particles is a particle-scale property that affects the mechanics of granular materials. The effect of the contact number of the sand particles on the onset and evolution of the force structures was also investigated and discussed.

3D x-ray diffraction microscopy↗
DOE OSTI2020

Where did the tumor start? An inverse solver with sparse localization for tumor growth models

In this work, we present a numerical scheme for solving an inverse problem for parameter estimation in tumor growth models for glioblastomas, a form of aggressive primary brain tumor. The growth model is a reaction–diffusion partial differential equation (PDE) for the tumor concentration. We use a PDE-constrained optimization formulation for the inverse problem. The unknown parameters are the reaction coefficient (proliferation), the diffusion coefficient (infiltration), and the initial condition field for the tumor PDE. Segmentation of magnetic resonance imaging (MRI) scans drive the inverse problem where segmented tumor regions serve as partial observations of the tumor concentration. Like most cases in clinical practice, we use data from a single time snapshot. Moreover, the precise time relative to the initiation of the tumor is unknown, which poses an additional difficulty for inversion. We perform a frozen-coefficient spectral analysis and show that the inverse problem is severely ill-posed. We introduce a biophysically motivated regularization on the structure and magnitude of the tumor initial condition. In particular, we assume that the tumor starts at a few locations (enforced with a sparsity constraint on the initial condition of the tumor) and that the initial condition magnitude in the maximum norm is equal to one. We solve the resulting optimization problem using an inexact quasi-Newton method combined with a compressive sampling algorithm for the sparsity constraint. Our implementation uses PETSc and AccFFT libraries. We conduct numerical experiments on synthetic and clinical images to highlight the improved performance of our solver over a previously existing solver that uses standard two-norm regularization for the calibration parameters. The existing solver is unable to localize the initial condition. Our new solver can localize the initial condition and recover infiltration and proliferation. In clinical datasets (for which the ground truth is unknown), our solver results in qualitatively different solutions compared to the two-norm regularized solver.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Solving protein structure from sparse serial microcrystal diffraction data at a storage-ring synchrotron source

In recent years, the success of serial femtosecond crystallography and the paucity of beamtime at X-ray free-electron lasers have motivated the development of serial microcrystallography experiments at storage-ring synchrotron sources. However, especially at storage-ring sources, if a crystal is too small it will have suffered significant radiation damage before diffracting a sufficient number of X-rays into Bragg peaks for peak-indexing software to determine the crystal orientation. As a consequence, the data frames of small crystals often cannot be indexed and are discarded. Introduced here is a method based on the expand–maximize–compress (EMC) algorithm to solve protein structures, specifically from data frames for which indexing methods fail because too few X-rays are diffracted into Bragg peaks. The method is demonstrated on a real serial microcrystallography data set whose signals are too weak to be indexed by conventional methods. In spite of the daunting background scatter from the sample-delivery medium, it was still possible to solve the protein structure at 2.1 Å resolution. The ability of the EMC algorithm to analyze weak data frames will help to reduce sample consumption. Furthermore, it will also allow serial microcrystallography to be performed with crystals that are otherwise too small to be feasibly analyzed at storage-ring sources.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2022

Unsupervised learning approaches to characterizing heterogeneous samples using X-ray single-particle imaging

One of the outstanding analytical problems in X-ray single-particle imaging (SPI) is the classification of structural heterogeneity, which is especially difficult given the low signal-to-noise ratios of individual patterns and the fact that even identical objects can yield patterns that vary greatly when orientation is taken into consideration. Proposed here are two methods which explicitly account for this orientation-induced variation and can robustly determine the structural landscape of a sample ensemble. The first, termed common-line principal component analysis (PCA), provides a rough classification which is essentially parameter free and can be run automatically on any SPI dataset. The second method, utilizing variation auto-encoders (VAEs), can generate 3D structures of the objects at any point in the structural landscape. Both these methods are implemented in combination with the noise-tolerant expand–maximize–compress (EMC) algorithm and its utility is demonstrated by applying it to an experimental dataset from gold nanoparticles with only a few thousand photons per pattern. Both discrete structural classes and continuous deformations are recovered. These developments diverge from previous approaches of extracting reproducible subsets of patterns from a dataset and open up the possibility of moving beyond the study of homogeneous sample sets to addressing open questions on topics such as nanocrystal growth and dynamics, as well as phase transitions which have not been externally triggered.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Highly multicolored light-emitting arrays for compressive spectroscopy

Miniaturized, multicolored light-emitting device arrays are promising for applications in sensing, imaging, computing, and more, but the range of emission colors achievable by a conventional light-emitting diode is limited by material or device constraints. In this work, we demonstrate a highly multicolored light-emitting array with 49 different, individually addressable colors on a single chip. The array consists of pulsed-driven metal-oxide-semiconductor capacitors, which generate electroluminescence from microdispensed materials spanning a diverse range of colors and spectral shapes, enabling facile generation of arbitrary light spectra across a broad wavelength range (400 to 1400 nm). When combined with compressive reconstruction algorithms, these arrays can be used to perform spectroscopic measurements in a compact manner without diffractive optics. As an example, we demonstrate microscale spectral imaging of samples using a multiplexed electroluminescent array in conjunction with a monochrome camera.

42 ENGINEERING↗