Search NASA⌕ Search

SEARCH · Search NASA

Results for “compression algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Pattern Dictionary Method for Anomaly Detection

In this paper, we propose a compression-based anomaly detection method for time series and sequence data using a pattern dictionary. The proposed method is capable of learning complex patterns in a training data sequence, using these learned patterns to detect potentially anomalous patterns in a test data sequence. The proposed pattern dictionary method uses a measure of complexity of the test sequence as an anomaly score that can be used to perform stand-alone anomaly detection. We also show that when combined with a universal source coder, the proposed pattern dictionary yields a powerful atypicality detector that is equally applicable to anomaly detection. The pattern dictionary-based atypicality detector uses an anomaly score defined as the difference between the complexity of the test sequence data encoded by the trained pattern dictionary (typical) encoder and the universal (atypical) encoder, respectively. We consider two complexity measures: the number of parsed phrases in the sequence, and the length of the encoded sequence (codelength). Specializing to a particular type of universal encoder, the Tree-Structured Lempel–Ziv (LZ78), we obtain a novel non-asymptotic upper bound, in terms of the Lambert W function, on the number of distinct phrases resulting from the LZ78 parser. This non-asymptotic bound determines the range of anomaly score. As a concrete application, we illustrate the pattern dictionary framework for constructing a baseline of health against which anomalous deviations can be detected.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

A learning-based measurement framework for traffic matrix inference in software defined networks

Here, we propose an intelligent framework for Traffic Matrix (TM) inference in Software Defined Networks (SDN) where the Ternary Content Addressable Memory (TCAM) entries of switches are partitioned into two parts to: (1) effectively aggregate part of incoming flows for aggregate measurements, and (2) de-aggregate and directly measure the most informative flows for per-flow measurements. These measurements are then processed to effectively estimate the size of network flows. Under hard resource constraints of limited TCAM sizes, we show how to design the optimal and efficient-compressed flow aggregation matrices. We propose an optimal Multi-Armed Bandit (MAB) based algorithm to adaptively measure the most rewarding flows. We evaluate the performance of our framework using real traffic traces from different network environments and by considering two main applications: TM estimation and Heavy Hitter (HH) detection. Moreover, we have implemented a prototype of our framework in Mininet to demonstrate its effectiveness.

42 ENGINEERING↗
DOE OSTI2022

$\mathrm{SIM}$-$\mathrm{STEM}$ $\mathrm{Lab}$: Incorporating Compressed Sensing Theory for Fast STEM Simulation

We report recently it has been shown that precise dose control and an increase in the overall acquisition speed of atomic resolution scanning transmission electron microscope (STEM) images can be achieved by acquiring only a small fraction of the pixels in the image experimentally and then reconstructing the full image using an inpainting algorithm. In this paper, we apply the same inpainting approach (a form of compressed sensing) to simulated, sub-sampled atomic resolution STEM images. We find that it is possible to significantly sub-sample the area that is simulated, the number of g-vectors contributing the image, and the number of frozen phonon configurations contributing to the final image while still producing an acceptable fit to a fully sampled simulation. Here we discuss the parameters that we use and how the resulting simulations can be quantifiably compared to the full simulations. As with any Compressed Sensing methodology, care must be taken to ensure that isolated events are not excluded from the process, but the observed increase in simulation speed provides significant opportunities for real time simulations, image classification and analytics to be performed as a supplement to experiments on a microscope to be developed in the future

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Fast wide-field upconversion luminescence lifetime thermometry enabled by single-shot compressed ultrahigh-speed imaging

Photoluminescence lifetime imaging of upconverting nanoparticles is increasingly featured in recent progress in optical thermometry. Despite remarkable advances in photoluminescent temperature indicators, existing optical instruments lack the ability of wide-field photoluminescence lifetime imaging in real time, thus falling short in dynamic temperature mapping. Here, we report video-rate upconversion temperature sensing in wide field using single-shot photoluminescence lifetime imaging thermometry (SPLIT). Developed from a compressed-sensing ultrahigh-speed imaging paradigm, SPLIT first records wide-field luminescence intensity decay compressively in two views in a single exposure. Then, an algorithm, built upon the plug-and-play alternating direction method of multipliers, is used to reconstruct the video, from which the extracted lifetime distribution is converted to a temperature map. Using the core/shell NaGdF 4 :Er 3+ ,Yb 3+ /NaGdF 4 upconverting nanoparticles as the lifetime-based temperature indicators, we apply SPLIT in longitudinal wide-field temperature monitoring beneath a thin scattering medium. SPLIT also enables video-rate temperature mapping of a moving biological sample at single-cell resolution.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

HBMax: Optimizing Memory Efficiency for Parallel Influence Maximization on Multicore Architectures

The goal of influence maximization is to select k most-influential vertices or seeds in a network, where influence is defined by a given diffusion process. The problem has a number of important applications such as viral marketing, information spread, and epidemic control. Although computing optimal seed set is NP-Hard, due to the submodular nature of the problem efficient approximation algorithms exist. However, even state-of-the-art parallel implementations are limited by a sampling step that incurs large memory footprints. This in turn limits the problem size reach and approximation quality. In this work, we study the memory footprint of the sampling process collecting reverse reachability information in the IMM algorithm over large real-world social networks. We present an adaptive and memory-efficient optimization approach for a state-of-the-art multi-threaded parallel influence maximization algorithm. Our approach,HuffMax, uses a portion of the reverse reachable (RR) sets collected by the algorithm to learn the characteristics of the graph. Then, it compresses the intermediate reverse reachability information with Huffman coding, and queries directly on the compressed data to preserve the memory savings obtained through compression. We also propose an efficient sampling strategy based on the distribution of RR sets, which can further reduce the computation time for typical social networks with long-tail distributions. Considering a NUMA architecture, we scale up our solution on 128-core CPUs and reduce the memory footprint by up to 45.7% with negligible time overhead (or even faster) and without perceivable loss of accuracy.

Chen, Xinyu↗
DOE OSTI2019

Asynchronous Navier-Stokes Solver on 3D Unstructured Grids for the Exascale Era

This project has developed multiple fluid dynamics solvers for complex 3D flows using fully asynchronous distributed-memory task-parallel algorithms on top of the Charm++ runtime system. The algorithms solve the Euler or Navier-Stokes equations of compressible flows using unstructured tetrahedron meshes with optional solution-adaptive mesh-, and polynomial-order refinement. We have demonstrated excellent strong scaling up 50K compute cores and the benefits of Charm++’s automatic load balancing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Lookup Table Based Low Resource Usage Waveform Data Compression Block in FPGA

aA lookup table based lossless data compression scheme suitable for FPGA implementation, with very low silicon resource usage, is developed. Different algorithms can be implemented by simply changing the contents of the lookup tables in the functional block while keeping the logic circuits unchanged. It exhibits performance comparable to the zip compression software in Linux distributions. The compression block can fit into almost any existing FPGA used in the online data acquisition systems of high energy physics experiments. The decompression process can be performed in either software or in FPGA firmware, allowing users to process compressed data flexibly.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

LLNL FESP Theory Highlights: October 2024

I. Novikau, I. Y. Dodin, E. A. Startsev, I. Joseph, Quantum algorithms for simulating dissipative linear and nonlinear dynamics of plasmas. Invited talk at the 66th Annual Meeting of the APS Division of Plasma Physics, Atlanta, Georgia. Novikau I., Dodin I.Y., Startsev E.A., Encoding of linear kinetic plasma problems in quantum circuits via data compression, Journal of Plasma Physics. 2024;90(4):805900401, doi:10.1017/S0022377824000795. We propose an algorithm for encoding linear kinetic plasma problems in quantum circuits. The focus is on modelling electrostatic linear waves in a one-dimensional Maxwellian electron plasma. The waves are described by the linearized Vlasov–Ampère system with a spatially localized external current that drives plasma oscillations. This system is formulated as a boundary-value problem and cast in the form of a linear vector equation to be solved by using the quantum signal processing algorithm. The latter requires encoding of a matrix in a quantum circuit as a sub-block of a unitary matrix. We propose how to encode in a circuit in a compressed form and discuss how the resulting circuit scales with the problem size and the desired precision.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Transported PDF Modeling of Compressible Turbulent Reactive Flows by using the Eulerian Monte Carlo Fields Method

Although the transported probability density function (PDF) method has been developed for decades, its application has been mainly focused on the low-Mach number flow problems. This work extends the transported PDF method to compressible flow problems. The Eulerian Monte Carlo fields (EMCF) solution method is employed to solve the transported PDF equation for compressible flow problems. A pseudo stagnation enthalpy is introduced and its stochastic partial differential equation is derived to ensure total energy conservation numerically. A new mixing model called interaction by partial exchange with mean (IPEM) is introduced to expand the available choices of mixing models for the EMCF method. The consistency of the EMCF method is examined for solving the transported PDF equation. Numerical implementation details are discussed, such as the density coupling between the compressible flow solver and the EMCF solver, discretization schemes for the mixing terms and the stochastic terms. The implemented compressible flow solver coupled with the EMCF solver is verified and validated in a series of test cases with increasing level of complexity, ranging from a statistically one-dimensional turbulent mixing layer to a self-excited resonance model rocket combustor. It is observed that in general with the increase of compressibility, there is an increase in the sensitivity of the modeling results to the different models and algorithms. This makes it necessary to develop a thorough understanding of the model sensitivity in order to develop a robust and accurate simulation solver for highly compressible turbulent reactive flows. The thermo-acoustic instability inside the model rocket combustor case is captured reasonably, which demonstrates the overall capability of the developed compressible turbulent combustion solver based on the transported PDF method.

42 ENGINEERING↗
DOE OSTI2025

Bridging material models across scales: An integrated approach to equation of state and molecular dynamics modeling of copper

New uncertainty-aware equation of state (EOS) and electrical conductivity (EC) models for copper have been developed. The multiphase EOS/EC models are fit to experimental solid/liquid EC isobar measurements as well as density-functional theory molecular dynamics (DFT-MD) EC calculations in both expanded and compressed regimes (0.1–16 g/ cm 3 ⁠). The liquid and solid EOS phases were fit to available experimental data along with additional DFT-MD data over the same range as the EC. Leveraging the DFT-MD data, a corresponding machine-learned interatomic potential (MLIAP) for copper was trained using genetic-algorithm optimization. The copper MLIAP was constrained by EOS shock points at high compressions. The final EOS bounded MLIAP proves to be stable over a large density range (approximately 0.1–20 g/ cm 3 ) with good agreement to an isothermal compression curve, shock Hugoniot, and liquid speed of sound measurements at high pressures (100s of GPa).

Acoustic measurements and instrumentation↗
DOE OSTI2023

Sparse Symmetric Format for Tucker Decomposition

Tensor-based methods are receiving renewed attention in recent years due to their prevalence in diverse real-world applications. There is considerable literature on tensor representations and algorithms for tensor decompositions, both for dense and sparse tensors. Many applications in hypergraph analytics, machine learning, psychometry, and signal processing result in tensors that are both sparse and symmetric, making them an important class for further study. Similar to the critical Tensor Times Matrix chain operation (TTM c ) in general sparse tensors, the $\underline{S}$ parse $\underline{S}$ ymmetric $\underline{T}$ ensor $\underline{T}$ imes $\underline{S}$ ame $\underline{M}$ atrix $\underline{c}$ hain (S 3 TTM c ) operation is compute and memory intensive due to high tensor order and the associated factorial explosion in the number of non-zeros. We present the novel Compressed Sparse Symmetric (CSS) format for sparse symmetric tensors, along with an efficient parallel algorithm for the S 3 TTM c operation. We theoretically establish that S 3 TTM c on CSS achieves a better memory versus run-time trade-off compared to state-of-the-art implementations, and visualize the variation of the performance gap over the parameter space. We demonstrate experimental findings that confirm these results and achieve up to 2.72× speedup on synthetic and real datasets. The scaling of the algorithm on different test architectures is also showcased to highlight the effect of machine characteristics on algorithm performance.

42 ENGINEERING↗
DOE OSTI2019

Phase retrieval for Bragg coherent diffraction imaging at high x-ray energies

Coherent x-ray beams with energies ≥ 50 keV can potentially enable three-dimensional imaging of atomic lattice distortion fields within individual crystallites in bulk polycrystalline materials through Bragg coherent diffraction imaging (BCDI). However, the undersampling of the diffraction signal due to Fourier-space compression at high x-ray energies renders conventional phase-retrieval algorithms unsuitable for three-dimensional reconstruction. To address this problem, we utilize a phase-retrieval method with a Fourier constraint specifically tailored for undersampled diffraction data measured with coarse-pitched detector pixels that bin the underlying signal. With our approach, we show that it is possible to reconstruct three-dimensional strained crystallites from an undersampled Bragg diffraction data set subject to pixel-area integration without having to physically upsample the diffraction signal. Using simulations and experimental results, we demonstrate that explicit modeling of Fourier-space compression during phase retrieval provides a viable means by which to invert high-energy BCDI data, which is otherwise intractable.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2018

A hybrid adaptive low-Mach number/compressible method: Euler equations

Flows in which the primary features of interest do not rely on high-frequency acoustic effects, but in which long-wavelength acoustics play a nontrivial role, present a computational challenge. Integrating the entire domain with low-Mach-number methods would remove all acoustic wave propagation, while integrating the entire domain with the fully compressible equations can in some cases be prohibitively expensive due to the CFL time step constraint. For example, simulation of thermoacoustic instabilities might require fine resolution of the fluid/chemistry interaction but not require fine resolution of acoustic effects, yet one does not want to neglect the long-wavelength wave propagation and its interaction with the larger domain. The present paper introduces a new multi-level hybrid algorithm to address these types of phenomena. In this new approach, the fully compressible Euler equations are solved on the entire domain, potentially with local refinement, while their low-Mach-number counterparts are solved on subregions of the domain with higher spatial resolution. The finest of the compressible levels communicates inhomogeneous divergence constraints to the coarsest of the low-Mach-number levels, allowing the low-Mach-number levels to retain the long-wavelength acoustics. Here, the performance of the hybrid method is shown for a series of test cases, including results from a simulation of the aeroacoustic propagation generated from a Kelvin-Helmholtz instability in low-Mach-number mixing layers. It is demonstrated that compared to a purely compressible approach, the hybrid method allows time-steps two orders of magnitude larger at the finest level, leading to an overall reduction of the computational time by a factor of 8.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

BoBa

BoBa is a C++ software library for working with large matrices, tensors, and tensor decompositions. The library provides tools for dense matrix and tensor operations, tensor decompositions, and tensor decomposition methods that support modern CPU and GPU architectures. It includes portable abstractions for linear algebra, tensor algebra, and multidimensional computation. BoBa is intended for scientific computing applications that involve large multidimensional data sets or high dimensional mathematical models. Its capabilities support tasks such as data compression, linear algebra, efficient numerical computation, and the development of scalable algorithms for heterogeneous hardware. Tutorials, tests, and example applications are included to help users learn and apply the library.

Yao, Jin [Lawrence Livermore National Laboratory (↗
DOE OSTI2017

IoGET: Internet of Geophysical and Environmental Things

The objective of this project is to provide novel and fast reduced-order models for onboard computation at sensor nodes for real-time analysis. The approach will require that LANL perform high-fidelity numerical simulations, construct simple reduced-order models (ROMs) using machine learning and signal processing algorithms, and use real-time data analysis for ROMs and compressive sensing at sensor nodes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Efficient Parallel Sparse Symmetric Tucker Decomposition for High-Order Tensors

Tensor based methods are receiving renewed attention in recent years due to their prevalence in diverse real-world applications. There is considerable literature on tensor representations and algorithms for tensor decompositions, both for dense and sparse tensors. Many applications in hypergraph analytics, machine learning, psychometry, and signal processing result in tensors that are both sparse and symmetric, making it an important class for further study. Similar to the critical Tensor Times Matrix chain operation (TTMc) in general sparse tensors, the Sparse Symmetric Tensor Times Same Matrix chain (S3TTMc) operation is compute and memory intensive due to high tensor order and the associated factorial explosion in the number of non-zeros. In this work, we present a novel compressed storage format CSS for sparse symmetric tensors, along with an efficient parallel algorithm for the S3TTMc operation. We theoretically establish that S3TTMc on CSS achieves a better memory versus run-time trade-off compared to state-of-the-art implementations. We demonstrate experimental findings that confirm these results and achieve up to 2.9× speedup on synthetic and real datasets.

Shivakumar, Shruti↗
DOE OSTI2021

An automated procedure built on MTEX for reconstructing deformation twin hierarchies from electron backscattered diffraction datasets of heavily twinned microstructures

Here we present a set of algorithms built on the MTEX and MATLAB graph toolboxes for automatic reconstruction of deformation twin hierarchies from Electron Backscatter Diffraction (EBSD) datasets with a focus on developing methods for heavily twinned microstructures (twin fractions >0.5). The algorithms address key issues arising at large strains, mainly: missing twin relationships, grouping of heavily deformed grain fragments into families of similar orientation originating from a single initial grain, identification of parent fragments for large twin volume fractions, and classification of families having twin relationships with multiple families. To facilitate the development of these algorithms, large-grained ultra-high purity α-Ti deformed in compression along two directions is investigated. Graphs are utilized to handle non-local geometric merging and to represent relationships throughout the reconstruction process. When determining if a grain fragment is from the undeformed microstructure, the combined metrics of the fragment's orientation volume fraction in the initial texture and the directed graph centrality measure of out-closeness (the number of nodes reached in a graph from a given node) are essential. To address automation in reconstructing the sequence of twinning and relating fragments originating from a single grain in the initial microstructure, the twin family tree is formulated as a minimum spanning tree emanating from the initial grain family. A scheme constructing the distances associated with twin relationship comprising the spanning tree is developed, and a novel quasi-directional Prim spanning tree algorithm is used to determine the twin family tree. The procedure is demonstrated to significantly improve the level of automation in reconstructing twin hierarchies in heavily twinned microstructure compared to other methodologies in literature. The procedure can readily be applied to analyses of twinning in metals, as well as provide an approach for routinely extracting twin statistics at larger deformation levels than previously possible. Significantly, the procedure is demonstrated to be capable of identifying third generation twinning in α-Ti microstructures.

36 MATERIALS SCIENCE↗