Search NASA⌕ Search

SEARCH · Search NASA

Results for “ZFP”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

ZFP Hardware Implementation

ZFP is a floating-point compression format gaining traction in high-performance computing applications. A software implementation has demonstrated the ability to reduce data movement across communication channels and to reduce the footprint of floating-point arrays in memory. Nevertheless, any benefit in performance is limited to the spare compute cycles available before reaching bandwidth limitations. A hardware implementation of ZFP has the potential to substantially improve HPC application performance. Hardware compression speedup over software alone on a single CPU core ranges from 15x for 1-D arrays to over 200x for 3-D arrays. The implementation of ZFP is sourced in SystemC to facilitate its evaluation in various architectures. The encode pipeline consists of several modules. Uncompressed blocks of floating-point numbers in IEEE format flow into the pipeline in a stream and a compressed bitstream flows out from the pipeline. The decode pipeline is the inverse of the encode pipeline. A modular design enables formats other than IEEE (such as posits) to be considered with minor adaptations. Best performance will be realized from the hardware ZFP unit when batches of blocks are processed at a time. The implementation supports 1-D, 2- D and 3-D blocks of floating-point numbers. The hardware implementation of ZFP has been validated with the software implementation of ZFP. C++ template parameters are used to specify the bit width of floating-point numbers and the array dimension of the encoder. A test bench program has been created with several test cases, some with continuous data that is ideal for ZFP, and others with extreme cases containing numbers near or at the maximum or minimum values supported by the number format.

Lloyd, GS↗
DOE OSTI2019

Error Analysis of ZFP Compression for Floating-Point Data

Compression of floating-point data will play an important role in high-performance computing as data bandwidth and storage become dominant costs. Lossy compression of floating-point data is powerful, but theoretical results are needed to bound its errors when used to store look-up tables, simulation results, or even the solution state during the computation. In this paper, we analyze the round-off error introduced by ZFP, a lossy compression algorithm. Here, the stopping criteria for ZFP depends on the compression mode specified by the user: fixed rate, fixed accuracy, or fixed precision [P. Lindstrom, ZFP 0.5.3 Documentation, 2018]. While most of our discussion is focused on the fixed precision mode of ZFP, we establish a bound on the error introduced by all three compression modes. In order to tightly capture the error, first we introduce a vector space that allows us to work with binary representations of components. Under this vector space, we define operators that implement each step of the ZFP compression and decompression to establish a bound on the error caused by ZFP. To conclude, numerical tests are provided to demonstrate the accuracy of the established bounds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by high-performance computing applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are two leading BSD licensed open source C/C++ libraries for compressed floating-point arrays that support high throughput read and write random access. However, their performance is not consistent across different data sets and across different fields of some data sets, which raises the need for an automatic online (during compression) selection between SZ and ZFP, with minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. Here, we implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70 percent with the same level of data distortion because of very accurate selection (around 99 percent) of the bestfit compressor, with little overhead (less than 7 percent in the experiments).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ZFP: A compressed array representation for numerical computations

HPC trends favor algorithms and implementations that reduce data motion relative to FLOPS. We investigate the use of lossy compressed data arrays in place of traditional IEEE floating point arrays to store the primary data of calculations. Simulation is fundamentally an exercise in controlled approximation, and error introduced by finite-precision arithmetic (or lossy compression) is just one of several sources of error that need to be managed to ensure sufficient accuracy in a computed result. We describe ZFP, a compressed numerical format designed for in-memory storage of multidimensional arrays, and summarize theoretical results that demonstrate that the error of repeated lossy compression can be bounded and controlled. Furthermore, we establish a relationship between grid resolution and compression-induced errors and show that, contrary to conventional floating point, ZFP reduces finite-difference errors with finer grids. We present example calculations that demonstrate data reduction by 4x or more with negligible impact on solution accuracy. Our results further demonstrate several orders-of-magnitude increase in accuracy using ZFP over IEEE floating point and Posits for the same storage budget.

Lindstrom, Peter↗
DOE OSTI2019

ECP Milestone Memo WBS 2.3.4.13 ECP/VTK-m FY19Q1 [MS-19/01-03] ZFP / Release / Clip STDA05-17

The STDA05-17 milestone comprises the following 3 deliverables. VTK-m Release 2 We will provide a release of VTK-m software and associated documentation. The source code repository will be tagged at a stable state, and, at a minimum, tarball captures of the source code will be made available from the web site. A version of the VTK-m User's Guide documenting this release will also be made available. Productionize zfp compression The "ZFP: Compressed Floating-Point Arrays" project (WBS 1.3.4.13) is creating an implementation of ZFP compression in VTK-m. Their implementation will be focused on operating in CUDA. The VTK-m project will assist by generalizing the implementation to other devices (such as multi-core CPUs). We will also assist in productionizing the code such that it can be used by external projects and products. Clip Clip operations intersect meshes with implicit functions. It is the foundation of spatial subsetting algorithms, such as "box," and the foundation of data-based subsetting, such as "isovolume." The algorithm requires considering thousands of possible cases, and is thus quite difficult to implement. This milestone will implement clipping to be sufficient for Visit's and ParaView's needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

ZFP Hardware Implementation

As core counts increase in new HPC systems with comparatively little increase in memory bandwidth, the trend is an effective decrease in memory bandwidth per core. Other bandwidth limitations in HPC systems exist between CPU and GPU memory, between system nodes, and between node memory and storage. Compression of floating-point data has the potential to reduce data movement and pressure across these communication channels. Furthermore, it has the potential to reduce the footprint of floating-point arrays stored in memory. ZFP, implemented in software, is gaining traction as an effective method in floating-point compression; however, performance gains are limited to the spare compute cycles available before reaching the bandwidth limitations of the communication channel. A hardware implementation of ZFP has the potential to raise the bar on performance. From the inception of ZFP, it was designed to accommodate a hardware implementation.

42 ENGINEERING↗
DOE OSTI2017

ZFP compression plugin (filter) for HDF5

H5Z-ZFP is a compression plugin (filter) for the HDF5 library based upon the ZFP-0.5.0 compression library. It supports 4- or 8-byte integer or floating point HDF5 datasets of any dimension but partitioned in 1, 2, or 3 dimensional chunks. It supports ZFP's four fundamental modes of operation; rate, precision, accuracy or expert. It is a lossy compression plugin.

Miller, Mark C.↗
DOE OSTI2020

Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods

Currently, the dominating constraint in many high performance computing applications is data capacity and bandwidth, in both internode communications and even moreso in intranode data motion. A new approach to address this limitation is to make use of data compression in the form of a compressed data array. Storing data in a compressed data array and converting to standard IEEE-754 types as needed during a computation can reduce the pressure on bandwidth and storage. However, repeated conversions (lossy compression and decompression) introduce additional approximation errors, which need to be shown to not significantly affect the simulation results. Here, we extend recent work that analyzed the error of a single use of compression and decompression of the ZFP compressed data array representation to the case of time-stepping and iterative schemes, where an advancement operator is repeatedly applied in addition to the conversions. We show that the accumulated error for iterative methods involving fixed-point and time evolving iterations is bounded under standard constraints. An upper bound is established on the number of additional iterations required for the convergence of stationary fixed-point iterations. An additional analysis of traditional forward and backward error of stationary iterative methods using ZFP compressed arrays is also presented. The results of several 1D, 2D, and 3D test problems are provided to demonstrate the correctness of the theoretical bounds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Supporting Special Values in ZFP

This white paper outlines potential approaches to supporting special values in the ZFP numerical compressor without breaking backwards compatibility. Other than infinities and NaNs, special values are often used to indicate the absence of data, where no value is defined, for example by designating finite but extreme “fill values” as special. Such fill values are commonly used in earth system science, among other applications, but if left as is during compression lead to artifacts and loss of precision in nearby true values. Multiple candidate solutions that would allow ZFP to recognize special values are here proposed. Until such support is available, we also sketch available workarounds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2023

zPerf: A Statistical Gray-Box Approach to Performance Modeling and Extrapolation for Scientific Lossy Compression

With the scaling up of simulation-based scientific discovery on high-performance computing systems, the disparity between compute and I/O has increased, forcing domain scientists to save only a small amount of simulation data to persistent storage. This can result in the loss of essential physics fields that are needed for data analysis. While error-bounded lossy compression has made tremendous progress in bridging the gap between compute and I/O, the lack of understanding of compression performance remains a key hurdle to its wide adoption. Here, in this work, we present zPerf, a statistical gray-box performance modeling approach for scientific lossy compression. Our contributions are threefold: 1) We develop zPerf to estimate the performance of lossy compression techniques, based on in-depth understanding and statistical modeling for data features and core compression metrics; 2) We demonstrate the in-detailed implementation of zPerf using two case studies, where we derive the performance modeling for SZ and ZFP, two leading lossy compressors; 3) We evaluate the effectiveness of zPerf on real-world datasets across various domains. Based on the evaluation, we demonstrate the efficacy of the zPerf performance model; 4) We further discuss three case studies where zPerf is applied to extrapolate the compression ratio of SZ and ZFP with alternative encoding schemes as well as ZFP with an alternative transform scheme. Through the case studies, we demonstrate the potential of zPerf for exploring the design space of lossy compression, which has hardly been studied in the literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

ZHW hardware library

Hardware library implementation of the open source ZFP CODEC. ZHP provides a sub-set of the ZFP library compression and decompression functionality, but in the SystemC language rather than the C/C++ and Fortran languages used in the ZFP implementation.

Barrow, MichaelJ↗
DOE OSTI2016

Effect of the linkers between the zinc fingers in zinc finger protein 809 on gene silencing and nuclear localization

Zinc finger protein 809 (ZFP809) belongs to the Kruppel-associated box-containing zinc finger protein (KRAB-ZFP) family and functions in repressing the expression of Moloney murine leukemia virus (MoMLV). ZFP809 binds to the primer-binding site (PBS)located downstream of the MoMLV-long terminal repeat (LTR) and induces epigenetic modifications at integration sites, such as repressive histone modifications and de novo DNA methylation. KRAB-ZFPs contain consensus TGEKP linkers between C2H2 zinc fingers. The phosphorylation of threonine residues within linkers leads to the inactivation of zinc finger binding to target sequences. ZFP809 also contains consensus linkers between zinc fingers. However, the function of ZFP809 linkers remains unknown. In the present study, we constructed ZFP809 proteins containing mutated linkers and examined their ability to silence transgene expression driven by MLV, binding ability to MLV PBS, and cellular localization. The results of the present study revealed that the linkers affected the ability of ZFP809 to silence transgene expression. Furthermore, this effect could be partly attributed to changes in the localization of ZFP809 proteins containing mutated linkers. Further characterization of ZFP809 linkers is required for understanding the functions and features of KRAB-ZFP-containing linkers. - Highlights: • ZFP809 has three consensus linkers between the zinc fingers. • Linkers are required for ZFP809 to silence transgene expression driven by MLV-LTR. • Linkers affect the precise nuclear localization of ZFP809.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Phytochrome-Dependent Regulation of ZFP6 and ZFPH Impacts Photomorphogenesis in Arabidopsis thaliana

Phytochromes (phy) are key regulators of photomorphogenesis in plants. Among the different phys characterized in higher plants (i.e., phyA to phyE), phyA and phyB primarily regulate phenotypic responses in plants under far-red (FR) and red (R) conditions, respectively. Recent findings suggest that some zinc finger proteins (ZFPs) are involved in plant light-modulated morphogenesis. However, the interaction(s) between phyA, phyB and ZFP homologs potentially involved in photomorphogenesis, as well as their phenotypic and molecular effects in Arabidopsis seedlings exposed to R and FR light remain to be elucidated fully. Prior analyses with phytochrome chromophore deficient lines indicated that ZFP6 expression is misregulated compared to levels in Col-0 wild type (WT). Here, we used plants with phytochrome chromophore or apoprotein (specifically phyA and phyB) deficiencies, lines with mutations in ZFP6 and ZFP6 HOMOLOG ( ZFPH ) genes, and plants overexpressing ZFP6 to examine regulatory interactions between phytochromes, ZFP6, and ZFPH. Our results indicate that phytochromes are required for downregulation of ZFP6 and ZFPH and suggest a role for light-regulated control of ZFP levels in phytochrome-dependent photomorphogenesis. Conversely, PHYB is downregulated in zfp6 mutants under R light. Analyses of a zfp6zfph double mutant confirmed disruption in photomorphogenic phenotypes, including the regulation of hypocotyl elongation in seedlings grown under FR light. In addition, PIF3 and PIF4 levels are transcriptionally regulated by ZFP6 and ZFPH in a gibberellic acid-dependent manner. ZFP6 overexpression resulted in opposite phenotypic responses to those observed in the zfp6 and zfph mutants grown in FR and R light, as well as a reduction in the rosette size of mature ZFP6 OX plants relative to WT under white light. Based on these observations, we provide insight into how phy and ZFPs interact to regulate specific aspects of light-dependent processes in Arabidopsis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A Framework for Compressing Unstructured Scientific Data via Serialization

We present a general framework for compressing unstructured scientific data with known local connectivity. A common application is simulation data defined on arbitrary finite element meshes. The framework employs a greedy topology preserving reordering of original nodes which allows for seamless integration into existing data processing pipelines. This reordering process depends solely on mesh connectivity and can be performed offline for optimal efficiency. However, the algorithm’s greedy nature also supports on-the-fly implementation. The proposed method is compatible with any compression algorithm that leverages spatial correlations within the data. The effectiveness of this approach is demonstrated on a large-scale real dataset using several compression methods, including MGARD, SZ, and ZFP.

Reshniak, Viktor [ORNL] (ORCID:0000000315454462)↗