Search NASA⌕ Search

SEARCH · Search NASA

Results for “compression algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

MrHyDE v.1.0

SAND2024-01324O MrHyDE, which stands for Multi-resolution Hybridized Differential Equations, is a general-purpose C++ package for the solution of coupled multiphysics and multiscale systems on massively parallel computing systems. MrHyDE is designed to enable moving beyond forward simulation for multiscale applications which includes optimization, control, uncertainty quantification, and stochastic inversion. The framework provides interfaces to several packages within the Trilinos framework and leverages automatic differentiation to enable adjoint capabilities for large-scale, gradient-based optimization. MrHyDE provides automated multiscale capabilities through a subgrid model interface and multiscale Dirichlet-to-Neumann maps. For extreme-scale applications, MrHyDE provides in situ data-compression algorithms to reduce memory requirements while maintaining performance. MrHyDE is a general-purpose, computational framework for the solution of multiscale and multiphysics applications. It uses a combination of structure-preserving, physics-compatible discretizations, fully implicit methods, multi-resolution schemes, or fully explicit methods. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗

The Center for Momentum Transport and Flow Organization in Plasmas - Final Scientific Report

Overview of University of Colorado Efforts: The University of Colorado group has focused on two primary fronts during the grant period: development of a variety of multi-point diagnostic and/or imaging analysis techniques, and momentum-transport related experiments on a variety of devices (NSTX at PPPL, CSDX at UCSD, LAPD at UCLA, DIII-D at GA). Experimental work has taken advantage of several diagnostic instruments, including fast-framing cameras for imaging of electron density fluctuations (either directly or using injected gas puffs), ECEI for imaging of electron temperature fluctuations, and multi-tipped Langmuir and magnetic probes for corroborating measurements of Reynolds and Maxwell stresses. Mode Characterization in CSDX: We have performed a series of experiments at the CSDX linear device at UCSD, in collaboration with Center PI G. Tynan's group. The experiments included a detailed study of velocity estimation techniques, including direct comparisons between Langmuir probes and image-based velocimetry from fast-framing camera data. We used the camera data in a second set of studies to identify the spatial and spectral structure of coherent modes, which illuminates wave behavior to a level of detail previously unavailable, and enables direct comparison of dispersion curves to theoretical estimates. In another CSDX study, similar techniques were used to demonstrate a controlled transition from nonlinearly coupled discrete eigenmodes to fully developed broadband turbulence. The axial magnetic field was varied from 40-240 mT, which drove the transition. At low magnetic fields, the plasma is dominated by drift waves. As the magnetic field is increased, a strong potential gradient at the edge introduces an ExB shear-driven instability. At the transition, another mode with signatures of a rotation-induced Rayleigh–Taylor instability appears at the central plasma region. Concurrently, large axial velocities were found in the plasma core. For larger magnetic fields, all of the instabilities co-exist, leading to rich plasma dynamics and fully developed broadband turbulence. Edge-Turbulence and Flow Experiments in NSTX: A series of Gas Puff Imaging (GPI) observations on NSTX revealed a quasi-periodic oscillation in the plasma edge preceding the L-H transition in a limited set of neutral beam heated plasmas. These ~3 kHz flow oscillations exhibit both long wavelength and long correlation lengths, suggesting they are zonal-flow-like. The flow oscillations are strongly correlated with modulations of the level of edge turbulence, thus the system appears to undergo a predator--prey-type limit-cycle preceding the L-H transition. However, a clear trigger for the L-H transition was not observed. Reynolds stress profiles were obtained directly from image velocimetry for L-mode periods ELM-Precursor Studies in NSTX: A separate study based on NSTX-GPI data captured the two-dimensional evolution of edge-localized mode (ELM) precursors. Precursor events were observed preceding ELMs and ELM-induced H–L back-transitions in radio-frequency heated H-mode plasmas, and the growth of the precursor mode through the ELM filamentation was imaged in the plane perpendicular to the local B-field. Strong edge intensity modulations appeared to propagate in the electron diamagnetic direction while steadily drifting radially outwards. Intensity fluctuations were observed at frequencies around 20 kHz and wavenumbers of 0.05-0.2 cm -1 . Upon growing to a trigger point, precursor fluctuations were seen to form filamentary structures and move into the scrape-off layer (SOL) explosively with radial velocities peaking at 8 km/s. Once in the SOL, filaments reverse their propagation direction and travel in the ion diamagnetic direction. Edge intensity fluctuations were strongly correlated with magnetic signals from Mirnov coils, and toroidally distributed coils estimated toroidal mode numbers of n=5-10. Quantitatively similar precursors have been observed in ohmic H-mode plasmas as well, though significantly fewer events are seen in the ohmic cases and none were observed in the near-threshold NBI H-modes studied. Development of Velocimetry and Image Analysis Techniques: Along with the experiments listed above, the Colorado group has continued to explore various velocimetry techniques and their range of validity. We have developed a “linear optical flow” code, which calculates smooth velocity maps while accurately assessing local regions of high curl. This is critical for separating spatial scales of velocity behavior, and thus transport. This code has recently come on-line, and we are currently using it to revisit a number of older datasets. Additionally, we have worked on developing pattern-recognition techniques for imaging diagnostics, based on established digital image compression algorithms. This has the potential to open the analysis of turbulent plasma behavior beyond the well-trodden Fourier and wavelet approaches. Finally, we have extended several of these image-analysis routines to multiple other diagnostic sets, including GPI datasets in NSTX and ECEI imaging on DIII-D. In one study, correlation maps were used to measure the full 2-D mode structure of drift-wave level structures using ECEI for the first time. This enabled direct comparison to gyrokinetic simulations using the GEM code, from which is was determined that TEM modes were being measured in DIII-D.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

TDD LoRa and Delta Encoding in Low-Power Networks of Environmental Sensor Arrays for Temperature and Deformation Monitoring: Supporting Data

This dataset was used to assess the performance of a compression algorithm for Soil Deformation and Temperature Monitoring data. The development of this technology is presented in the article named time division duplex (TDD) "TDD LoRa and Delta Encoding in Low-Power Networks of Environmental Sensor Arrays for Temperature and Deformation Monitoring" and published in Springer's Journal on Signal Processing Systems. [Time Division Duplex (TDD); Long Range (LoRa)]Field experiment data includes: 1) soil temperature measurements, 2) snow temperature data, 3) soil temperature and deformation data. In these files, the first column contains the time (UTC), followed by the sensor's battery voltage and temperature and acceleration values (X, Y, Z) in subsequent columns. The measurements were acquired every 15 minutes. Lists included for probe, datalogger, and packaging components. The two temperature data sets were acquired at a site at the East River Watershed, Colorado, while the temperature/deformation data were collected at the Teller Road (mile 27) site near Nome, Alaska. The package includes 8 comma-separated files (.csv) and one .pdf file.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

STRUMPACK -- STRUctured Matrices PACKage

STRUMPACK - STRUctured Matrices PACKage - is a package for computations with sparse and dense structured matrix, i.e., matrices that exhibit some kind of low-rank property, in particular Hierarchically Semi Separable structure (HSS). Such matrices appear in many applications, e.g., FEM, BEM, Integral equations. etc. Exploiting this structure using certain compression algorithms allow for fast solution of linear systems and/or fast computation of matrix-vector products, which are the two main building blocks of matrix computations. STRUMPACK has presently two main components: a distributed-memory dense matrix computations package and a shared-memory sparse direct solver.

Source record↗

Streaming Compression of Scientific Data via Weak-SINDy

Here, in this paper, a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing a stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical “offline” compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data “online” while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.

97 MATHEMATICS AND COMPUTING↗

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

We present regression and compression algorithms for lattice QCD data utilizing the efficient binary optimization ability of quantum annealers. In the regression algorithm, we encode the correlation between the input and output variables into a sparse coding machine learning algorithm. The trained correlation pattern is used to predict lattice QCD observables of unseen lattice configurations from other observables measured on the lattice. In the compression algorithm, we define a mapping from lattice QCD data of floating-point numbers to the binary coefficients that closely reconstruct the input data from a set of basis vectors. Since the reconstruction is not exact, the mapping defines a lossy compression, but, a reasonably small number of binary coefficients are able to reconstruct the input vector of lattice QCD data with the reconstruction error much smaller than the statistical fluctuation. In both applications, we use D-Wave quantum annealers to solve the NP-hard binary optimization problems of the machine learning algorithms.

79 ASTRONOMY AND ASTROPHYSICS↗

Real-Time Lossless Compression for Ultra-High-Density Synchrophasor and Point on Wave Data

Modern advanced Phasor Measurement Units (PMUs) are developed with ultra-high reporting rates to meet the demand for monitoring the power systems dynamics in detail. Due to the large volume of data, the communication and storage systems are seriously challenged with the presence of Ultra-High-Density (UHD) synchrophasor and Point on Wave (POW) data. Therefore, it is an urgent task to compress the UHD data for more efficient communication and data storage. This paper proposes several methods to compress the synchrophasor and POW data in a lossless manner. First, an Improved-Time-Series-Special Compression (ITSSC) method is proposed to compress the UHD frequency data. Second, a Delta-difference Huffman method is combined with the TSSC algorithm to compress the UHD phase angle data. Finally, a cyclical high-order delta modulation method is proposed to compress the UHD POW data. The proposed models are extensively tested and compared with different existing lossless compression algorithms using the field-collected synchrophasor and POW data at different reporting rates. The results indicate that the proposed algorithms are efficient in performing lossless compression for the UHD synchrophasor and POW data in real time.

42 ENGINEERING↗

An added-mass partition algorithm for fluid–structure interactions of compressible fluids and nonlinear solids

We describe an added-mass partitioned (AMP) algorithm for solving fluid–structure interaction (FSI) problems involving inviscid compressible fluids interacting with nonlinear solids that undergo large rotations and displacements. The computational approach is a mixed Eulerian–Lagrangian scheme that makes use of deforming composite grids (DCG) to treat large changes in the geometry in an accurate, flexible, and robust manner. The current work extends the AMP algorithm developed in Banks et al. [1] for linearly elasticity to the case of nonlinear solids. To ensure stability for the case of light solids, the new AMP algorithm embeds an approximate solution of a nonlinear fluid–solid Riemann (FSR) problem into the interface treatment. The solution to the FSR problem is derived and shown to be of a similar form to that derived for linear solids: the state on the interface being fundamentally an impedance-weighted average of the fluid and solid states. Numerical simulations demonstrate that the AMP algorithm is stable even for light solids when added-mass effects are large. The accuracy and stability of the AMP scheme is verified by comparison to an exact solution using the method of analytical solutions and to a semi-analytical solution that is obtained for a rotating solid disk immersed in a fluid. The scheme is applied to the simulation of a planar shock impacting a light elliptical-shaped solid, and comparisons are made between solutions of the FSI problem for a neo-Hookean solid, a linearly elastic solid, and a rigid solid. The ability of the approach to handle large deformations is demonstrated for a problem of a high-speed flow past a light, thin, and flexible solid beam.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Impacts of swinging door lossy compression of synchrophasor data

As the number of operational Phasor Measurement Units (PMUs) is increasing, compression is becoming crucial for efficiently transferring and storing enormous amount of synchrophasor data continuously measured from a power system. Some utilities in the U.S. have already adopted a nonlinear compression technique, called Swinging Door Algorithm (SDA), assuming compressed data meets technical and regulatory requirements without resulting in any degradation of the measurements. This paper studies impacts of this compression algorithm on field measured and simulated PMU data, and presents results from spectral analyses of raw and decompressed measurements. The results show blind application of the SDA compression alters frequency content in the measurements and introduces false harmonics and aliasing. These can lead to false conclusions being drawn from the measurements about the power system condition.

42 ENGINEERING↗

A Comparison of Compressed Sensing and Sparse Recovery Algorithms Applied to Simulation Data

The move toward exascale computing for scientific simulations is placing new demands on compression techniques. It is expected that the I/O system will not be able to support the volume of data that is expected to be written out. To enable quantitative analysis and scientific discovery, we are interested in techniques that compress high-dimensional simulation data and can provide perfect or near-perfect reconstruction. In this paper, we explore the use of compressed sensing (CS) techniques to reduce the size of the data before they are written out. Using large-scale simulation data, we investigate how the sufficient sparsity condition and the contrast in the data affect the quality of reconstruction and the degree of compression. Also, we provide suggestions for the practical implementation of CS techniques and compare them with other sparse recovery methods. Finally, our results show that despite longer times for reconstruction, compressed sensing techniques can provide near perfect reconstruction over a range of data with varying sparsity.

97 MATHEMATICS AND COMPUTING↗

Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model Compression

With the development of the Internet of Things, the application of computer vision on mobile phones is becoming more and more extensive and people have higher and higher requirements for the timeliness of the recognition results returned and the processing capabilities of the mobile phone for image recognition. However, the processing capability and storage capability of the user terminal equipment cannot meet the needs of identifying and storing a large number of pictures, and the data transmission process will cause high energy consumption of the terminal equipment. At the same time, multisource deep transfer learning has outstanding performance in computer vision and image classification. However, due to the huge amount of calculation of the deep network model, it is impossible to use the existing excellent network model to realize image recognition and classification on the mobile terminal. In order to solve the abovementioned problems, we propose a multisource mobile transfer learning algorithm based on dynamic model compression, this algorithm considers the realization of multisource transfer learning computing in the case of multiple mobile device computing source domains, and the method also guarantees data privacy and security for each device (origin domain). Meanwhile, extensive experiments show that our method can achieve remarkable results in popular image classification datasets.

Gao, Peng↗

A fourth-order adaptive mesh refinement algorithm for the multicomponent, reacting compressible Navier–Stokes equations

In this study, we present a fourth-order in space and time block-structured adaptive mesh refinement algorithm for the compressible multicomponent reacting Navier–Stokes equations. The algorithm uses a finite-volume approach that incorporates a fourth-order discretisation of the convective terms. The time-stepping algorithm is based on a multi-level spectral deferred corrections method that enables explicit treatment of advection and diffusion coupled with an implicit treatment of reactions. The temporal scheme is embedded in a block-structured adaptive mesh refinement algorithm that includes subcycling in time with spectral deferred correction sweeps applied on levels. Here we present the details of the multi-level scheme paying particular attention to the treatment of coarse–fine boundaries required to maintain fourth-order accuracy in time. We then demonstrate the convergence properties of the algorithm on several test cases including both non-reacting and reacting flows. Finally we present simulations of a vitiated dimethyl ether jet in 2D and a turbulent hydrogen jet in 3D, both with detailed kinetics and transport.

97 MATHEMATICS AND COMPUTING↗

System for outputting a spectrum of light of a scene having optical detectors to receive light of different spectral transmittance from respective filters

Technologies pertaining to a compressive sensing snapshot spectrometer are described herein. Light is focused through an array of filters onto an array of optical detectors by way of an optical objective. Each detector in the array receives light from a single respective filter in the filters. Each filter in the filters has a spectral transmittance function that overlaps a spectral transmittance function of at least one other filter. A compressive sensing algorithm is executed over outputs of the detectors based upon the known spectral transmittance functions of the filters. Based upon the execution of the compressive sensing algorithm, intensity values of the light are identified for a number of spectral bins that is greater than a number of detectors in the array.

Shields, Eric A.↗

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor train decomposition of a stream of tensor data. This new algorithm, called the tensor train incremental core expansion (TT-ICE) improves upon the current state-of-the-art algorithms for compressing in tensor train format by developing a new adaptive approach that incurs significantly slower rank growth and guarantees compression accuracy. This capability is achieved by limiting the number of new vectors appended to the TT-cores of an existing accumulation tensor after each data increment. These vectors represent directions orthogonal to the span of existing cores and are limited to those needed to represent a newly arrived tensor to a target accuracy. We provide two versions of the algorithm: TT-ICE and TT-ICE accelerated with heuristics (TT-ICE*). Here, we provide a proof of correctness for TT-ICE and empirically demonstrate the performance of the algorithms in compressing large-scale video and scientific simulation datasets. Compared to existing approaches that also use rank adaptation, TT-ICE* achieves 57× higher compression and up to 95% reduction in computational time.

97 MATHEMATICS AND COMPUTING↗

Integrating Online Compression to Accelerate Large-Scale Data Analytics Applications

Abstract—With growing dataset sizes, and as computing cycles are increasing faster than storage and wide-area bandwidths, compression appears like a promising approach for improving the performance of large-scale data analytics applications. In this context, this paper makes the following contributions. First, we develop a new compression methodology, which exploits the similarities between spatial and/or temporal neighbors in a simulation dataset, and enables high compression ratios and low decompression costs. Second, we have developed a framework that can be used to incorporate a variety of compression and decompression algorithms. This framework also supports a simple API to allow integration with an existing application or data processing middleware. Once a compression algorithm is implemented, this framework can allow multi-threaded retrieval, multi-threaded data decompression, and use of informed prefetching and caching. By integrating this framework with a data-intensive middleware, we have applied our compression methodology and framework to three applications over two datasets, including a GCRM climate model dataset. We obtained an average compression ratio of 51.68%, and up to 53.27% improvement in execution time of data analysis applications.

Big Data, I/O, HPC↗

Compression Analytics for Classification and Anomaly Detection within Network Communication

Here, the flexibility of network communication within Internet protocols is fundamental to network function, yet this same flexibility permits the possibility of malicious use. In particular, malicious behavior can masquerade as benign traffic, thus evading systems designed to catch misuse of network resources. However, perfect imitation of benign traffic is difficult, meaning that small unintentional deviations from normal can occur. Identifying these deviations requires that the defenders know what features reveal malicious behavior. Herein we present an application of compression-based analytics to network communication that can reduce the need for defenders to know a priori what features they need to examine. Motivating the approach is the idea that compression relies on the ability to discover and make use of predictable elements in information, thereby highlighting any deviations between expected and received content. We introduce a so-called “slice compression” score to identify malicious or anomalous communication in two ways. First, we apply normalized compression distances (NCDs) to classification problems and discuss methods for reducing the noise by excising application content (as opposed to protocol features) using slice compression. Second, we present a new technique for anomaly detection, referred to as slice compression for anomaly detection (SCADe). A diverse collection of datasets are analyzed to illustrate the efficacy of the proposed approaches. While our focus is network communication, other types of data are also considered to illustrate the generality of the method.

97 MATHEMATICS AND COMPUTING↗