Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression (Computer science)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Supporting Special Values in ZFP

This white paper outlines potential approaches to supporting special values in the ZFP numerical compressor without breaking backwards compatibility. Other than infinities and NaNs, special values are often used to indicate the absence of data, where no value is defined, for example by designating finite but extreme “fill values” as special. Such fill values are commonly used in earth system science, among other applications, but if left as is during compression lead to artifacts and loss of precision in nearby true values. Multiple candidate solutions that would allow ZFP to recognize special values are here proposed. Until such support is available, we also sketch available workarounds.

97 MATHEMATICS AND COMPUTING↗

Equations of State in Computational Physics: What, Why, and How

Summary: 1. Equations of state furnish thermodynamic properties of a material. 2. Usually involve P, p (or V), and T, or P, r, and E 3. Necessary ingredient to solve compressible flow problems. 4. Varieties: i. Analytical expressions (ideal gas, Grüneisen) – fast, convenient for verification and test problems ii. Graphs and charts – good for hand calculations iii. Tables and databases – generally most accurate, require interpolation i. Steam tables ii. NIST database iii. SESAME EOS Library 5. Constructed by using semi-empirical models from Statistical Mechanics and fitting to available data

36 MATERIALS SCIENCE↗

Laboratory-Based Micro-X-ray Computed Tomography of Energy Materials at Idaho National Laboratory

Abstract The Idaho National Laboratory (INL) has implemented laboratory-based micro-X-ray computed tomography in a laboratory equipped for the examination of highly radioactive samples. This capability provides nondestructive three-dimensional volumetric information on samples to inform subsequent traditional destructive examinations as well as real-world inputs for high-fidelity scientific modeling. Samples can be imaged with spatial resolutions ranging from several hundred nm/voxel up to ~ 100 µm/voxel. The best usable spatial resolution achieved to date is 384 nm/voxel with this instrument, while the highest radiological dose rate of a sample imaged is ~ 60 R/h β/γ on contact. Advanced data analysis, including custom tomographic reconstruction and segmentation methods, have also been developed to support this capability. In addition to traditional digital X-ray radiography and tomography, this instrument is also able to visualize in situ tensile and compression testing as well as perform diffraction contrast tomography. This work describes the X-ray computed tomography post-irradiation examination capabilities at INL, as well as detailing a variety of applications this instrument has examined.

36 MATERIALS SCIENCE↗

Transparent in Situ Data Transformations in ADIOS

Though an abundance of novel "data transformation" technologies have been developed (such as compression, level-of-detail, layout optimization, and indexing), there remains a notable gap in the adoption of such services by scientific applications. In response, we develop an in situ data transformation framework in the ADIOS I/O middleware with a "plug in" interface, thus greatly simplifying both the deployment and use of data transform services in scientific applications. Our approach ensures user-transparency, runtime-configurability, compatibility with existing I/O optimizations, and the potential for exploiting read-optimizing transforms (such as level-of-detail) to achieve I/O reduction. We demonstrate use of our framework with the QLG simulation at up to 8,192 cores on the leadership-class Titan supercomputer, showing negligible overhead. We also explore the read performance implications of data transforms with respect to parameters such as chunk size, access pattern, and the "opacity" of different transform methods including compression and level-of-detail.

Computer Science↗

Scaling kinetic Monte-Carlo simulations of grain growth with combined convolutional and graph neural networks

Graph neural networks (GNN) have emerged as a promising machine learning method for microstructure simulations such as grain growth. However, accurate modeling of realistic grain boundary networks requires large simulation cells, which GNN has difficulty scaling up to. To alleviate the computational costs and memory footprint of GNN, we suggest a hybrid architecture combining a convolutional neural network (CNN) based bijective autoencoder to compress the spatial dimensions, and a GNN that evolves the microstructure in the latent space of reduced spatial sizes. Our results demonstrate that the new design significantly reduces computational costs with using fewer message passing layer (from 12 down to 3) compared with GNN alone. The reduction in computational cost becomes more pronounced as the spatial size increases, indicating strong computational scalability. For the largest mesh evaluated (160 3 ), our method reduces memory usage and runtime in inference by 117× and 115×, respectively, compared with GNN-only baseline. More importantly, it shows higher accuracy and stronger spatiotemporal capability than the GNN-only baseline, especially in long-term testing. Such combination of scalability and accuracy is essential for simulating realistic material microstructures over extended time scales. The improvements can be attributed to the bijective autoencoder’s ability to compress information losslessly from spatial domain into a high dimensional feature space, thereby producing more expressive latent features for the GNN to learn from, while also contributing its own spatiotemporal modeling capability. Training data are generated from stochastic grain growth simulations, providing realistic variability for learning robust microstructure evolution. Comprehensive system validation confirms that the model is accurate, robust, and scalable.

36 MATERIALS SCIENCE↗

Adaptive Instrument Module: Space Instrument Controller "Brain" through Programmable Logic Devices

The Adaptive Instrument Module (AIM) will be the first true demonstration of reconfigurable computing with field-programmable gate arrays (FPGAs) in space, enabling the 'brain' of the system to evolve or adapt to changing requirements. In partnership with NASA Goddard Space Flight Center and the Australian Cooperative Research Centre for Satellite Systems (CRC-SS), APL has built the flight version to be flown on the Australian university-class satellite FEDSAT. The AIM provides satellites the flexibility to adapt to changing mission requirements by reconfiguring standardized processing hardware rather than incurring the large costs associated with new builds. This ability to reconfigure the processing in response to changing mission needs leads to true evolveable computing, wherein the instrument 'brain' can learn from new science data in order to perform state-of-the-art data processing. The development of the AIM is significant in its enormous potential to reduce total life-cycle costs for future space exploration missions. The advent of RAM-based FPGAs whose configuration can be changed at any time has enabled the development of the AIM for processing tasks that could not be performed in software. The use of the AIM enables reconfiguration of the FPGA circuitry while the spacecraft is in flight, with many accompanying advantages. The AIM demonstrates the practicalities of using reconfigurable computing hardware devices by conducting a series of designed experiments. These include the demonstration of implementing data compression, data filtering, and communication message processing and inter-experiment data computation. The second generation is the Adaptive Processing Template (ADAPT) which is further described in this paper. The next step forward is to make the hardware itself adaptable and the ADAPT pursues this challenge by developing a reconfigurable module that will be capable of functioning efficiently in various applications. ADAPT will take advantage of radiation tolerant RAM-based field programmable gate array (FPGA) technology to develop a reconfigurable processor that combines the flexibility of a general purpose processor running software with the performance of application specific processing hardware for a variety of high performance computing applications.

Darrin, Ann Garrison↗

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗

Forward and Inverse Models for Satellite Remote Sensors using Principal Component Analysis

Satellite remote sensors such as AIRS on Aqua, CrIS on S-NPP, NOAA20 and JPSS-2, IASI on Metop A, B, and C make millions of observations each day with thousands of spectral channels for each observation; this poses challenges for efficiently inversion of the inherently large dataset as needed to retrieve atmospheric and surface properties. This presentation will illustrate the use of Principal Component Analysis (PCA) to speed up radiative transfer forward model calculations and to stabilize the inversion algorithms. A Principal Component-based radiative transfer model (PCRTM) developed at NASA Langley Research Center can simulate top of atmosphere (TOA) radiance or reflectance spectra from 50 cm-1 to 50000 cm-1 (200 m to 0.20 m quickly and accurately. PCRTM demonstrated very high accuracy relative to reference line-by-line radiative transfer models and it saves orders of magnitude computational time. Examples of the PCRTM model developed for hyperspectral sensors such as AIRS, CrIS, IASI, NAST-I, SHIS, CPF, TEMPO, SBG, OMI, and SCIAMACHY will be presented. In addition to using the PCRTM as forward model, the NASA Langley developed inversion algorithm also uses PCA to compress the state vector into a compressed dimension to speed up and stabilize the inversion process. Examples of retrieved atmospheric temperature, water vapor, CO2, CO, CH4, N2O, and O3 profiles, cloud properties (optical depth, size, phase, and height), and surface properties (surface emissivity spectra and skin temperatures) will be presented. This algorithm is being transitioned to the NASA Sounder SIPS and NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC).

forward model↗

Multi-level Layout Optimization for Efficient Spatio-temporal Queries on ISABELA-compressed Data

The size and scope of cutting-edge scientific simulations are growing much faster than the I/O subsystems of their runtime environments, not only making I/O the primary bottleneck, but also consuming space that pushes the storage capacities of many computing facilities. These problems are exacerbated by the need to perform data-intensive analytics applications, such as querying the dataset by variable and spatio-temporal constraints, for what current database technologies commonly build query indices of size greater than that of the raw data. To help solve these problems, we present a parallel query-processing engine that can handle both range queries and queries with spatio-temporal constraints, on B-spline compressed data with user-controlled accuracy. Our method adapts to widening gaps between computation and I/O performance by querying on compressed metadata separated into bins by variable values, utilizing Hilbert space-filling curves to optimize for spatial constraints and aggregating data access to improve locality of per-bin stored data, reducing the false positive rate and latency bound I/O operations (such as seek) substantially. We show our method to be efficient with respect to storage, computation, and I/O compared to existing database technologies optimized for query processing on scientific data.

Computer Science↗

NEON AOP Survey of Upper East River CO Watersheds: Waveform LiDAR Binary Data

The waveform Light Detection and Ranging (LiDAR) data in this package were generated through a National Ecological Observatory Network Airborne Observation Platform (NEON AOP) acquisition over watersheds of interest surrounding Crested Butte, Colorado. The remote sensing imagery acquired by the NEON AOP supports an interdisciplinary project on hydrology, biogeochemistry, and ecosystem functioning in a snow-dominated headwater environment. These waveform LiDAR data enable spatially continuous estimation of vegetation structure parameters to facilitate analyses of the major environmental drivers of structural and compositional variability. The package contains 97 compressed file archives in 7-zip (.7z) format, each corresponding to one acquisition flightpath. Within each .7z archive is a set of constituent files describing properties of the LiDAR waveforms, such as return intensity, geolocation, outgoing pulse and other behavior of the sensor and signals. Once downloaded, the files must first be unzipped using the widely distributed command-line software utility 7z, using the command '7z x \[filename\].7z \[target_directory\]'. All files within the .7z archives can be opened in IDL, MatLab, or the open-source R statistical computing environment. Further details about the data package are in the attached user guide (neon_aop_crbu_waveformlidar_userguide.pdf). This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

2018 NEON and 2025 CHESS Campaigns↗

Scientific Discovery with Physics-Informed System Identification (Abbreviated Report)

My fellowship research focused on making physics-based simulations faster and more useful through machine learning. Many problems in science and engineering are governed by partial differential equations, but high-fidelity simulations are often too expensive to run repeatedly. I worked on improving Latent Space Dynamics Identification (LaSDI), a reduced-order modeling framework that compresses large simulation data sets into a smaller representation and then learns how that representation evolves over time. The motivation was to develop reduced models that remain accurate for more challenging systems, especially when predictions must remain reliable over long time intervals or when the underlying dynamics are more complicated than standard methods can easily handle. I also contributed to related work on Quandary, a high-performance software effort for simulation and control of open quantum systems, before focusing primarily on Latent Space Dynamics Identification methods. The main outcomes of the fellowship were two new algorithms (both of which were published), Rollout-LaSDI and Higher-Order LaSDI, together with supporting work on multi-stage Latent Space Dynamics Identification. Rollout-LaSDI improved long-term prediction by training the model to stay accurate over extended time horizons, and Higher-Order LaSDI broadened the method so it could model systems with higher-order time dynamics. My contributions to multistage Latent Space Dynamics Identification also helped show that its later training stages could be simplified without losing effectiveness, and that this behavior held across different model architectures and training strategies. Taken together, these advances improved the accuracy, flexibility, and practical value of reduced-order modeling tools for computational science.

97 MATHEMATICS AND COMPUTING↗

Extending the Density Functional Tight Binding Method to Carbon Under Extreme Conditions

We report herein on simulations of carbon under pressures up to 2000 GPa and 30 000 K using the density functional tight binding method (DFTB) with a parameter set we have specifically designed for these conditions. The DFTB method can provide a high throughput simulation capability compared to Kohn–Sham density functional theory while retaining most of its accuracy. We fit the DFTB repulsive energy to measured and computed diamond isothermal compression data and show that this yields accurate compression curves for diamond, graphite, and the BC8 phase, as well as material properties for all three phases. We then show that our new repulsive energy yields predictions of the Hugoniot of diamond shock compressed to the conducting liquid that are within the range of different experimental measurements. Furthermore, our results provide a straightforward method by which DFTB can be extended to studies of covalently bonded materials under extremely high pressures and temperatures such as the interiors of planets and other large celestial bodies.

36 MATERIALS SCIENCE↗

Ultrasound and X-ray Cross-Characterization of a Graded Impedance Impactor used for Shock-Ramp Compression Experiments

Abstract In this work we perform ultrasound measurements on an impedance graded impactor made by tape casting magnesium, copper, and tungsten. We also destructively extract small representative samples from the part for complementary characterization with x-ray computed tomography. Combining the two data sets enables direct assignment of some of the measured ultrasound features to specific material characteristics identified by x-ray tomography. Our results demonstrate how ultrasound inspection, informed by x-ray computed tomography, can be used to identify sub-millimeter material amalgamations and spatial heterogeneities in this graded material.

36 MATERIALS SCIENCE↗

Real-time Anomaly Detection for Liquid Argon Time Projection Chambers

We present a real-time anomaly detection framework for liquid argon time projection chambers (LArTPCs), targeting applications in particle physics experiments such as the Short Baseline Near Detector (SBND) or the future Deep Underground Neutrino Experiment (DUNE). These experiments employ detectors that generate and stream high-resolution but sparse images of neutrino and other particle interactions. Our approach utilizes anomaly detection with autoencoders, compressed through knowledge distillation (KD), to enable the detection of anomalous signals in the data through efficient inference on resource-constrained hardware. The framework is targeted for deployment on computing platforms equipped with field-programmable gate arrays (FPGAs), GPUs, or CPUs, allowing low-latency selection of relevant activity directly from the raw detector data stream. We demonstrate that our approach is suitable for the detection and localization of anomalously "high-multiplicity" activity, and outline promising applications for LArTPC online data filtering and triggering.

FOS: Physical sciences↗

Additive Manufactured Materials Studies

The production of additively manufactured (AM) materials, is a growing technology that potentially offers more cost effective ways to produce metallic components for various uses. Mechanical properties of some AM metals have been studied, but more work is needed to better understand their dynamic properties as a function of orientation. Due to the melting and reheating of molten metal during fabrication, these materials tend to have internal porosity and slightly different microstructures along the weldments and normal to the weldments that may cause the materials to behave differently than their forged counterparts. The AM process builds up a part layer by layer, which provides opportunities to tailor geometry and optimize structural performance in addition to minimizing the amount of material waste for oddly shaped parts, thereby reducing cost. These advanced materials hold great promise for revolutionizing design, fabrication, and assembly of engineered components and could be used for many NNSA-specific applications including armoring complex elements of nuclear weapons, and customized diagnostic components in subcritical experiments. We have conducted materials studies in collaboration with the University of Nevada, Las Vegas (UNLV) as well as Los Alamos National Laboratory (LANL) under our Site-Directed Research and Development program and the Nevada Science Initiative. We are currently placing emphasis on quantifying how dynamic properties are affected by directionality and porosity of the material. A study of dynamic failure events in forged, layered, and additive manufactured (AM) titanium alloy subject to hypervelocity impact was conducted in collaboration with UNLV. Experiments were conducted using a two stage light-gas gun, 0.22-caliber Lexan projectiles, and different types of titanium target plates. A four-channel Photonic Doppler Velocimetry (PDV) system was used to study the free surface velocities to provide an understanding of the different failure mechanisms in these materials. The experimental measurements were used to validate computational simulations, which describe the behavior of these materials under shock. It was determined that AM, forged titanium, and multi-layered stacks produce similar velocity profiles during the early stage of impact, with the AM targets exhibiting spall at lower velocities and the multi-layered stacks exhibiting vibrations between plates. Simulations of solid and layered forged materials provide a good match to experimental data.

36 MATERIALS SCIENCE↗

Remote visual analysis of large turbulence databases at multiple scales

The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visualization of such datasets was restricted to scientists with access to large supercomputers. The public Johns Hopkins Turbulence database simplifies access to multi-terabyte turbulence datasets and facilitates the computation of statistics and extraction of features through the use of commodity hardware. In this paper, we present a framework designed around wavelet-based compression for high-speed visualization of large datasets and methods supporting multi-resolution analysis of turbulence. By integrating common technologies, this framework enables remote access to tools available on supercomputers and over 230 terabytes of DNS data over the Web. Finally, the database toolset is expanded by providing access to exploratory data analysis tools, such as wavelet decomposition capabilities and coherent feature extraction.

97 MATHEMATICS AND COMPUTING↗

Impact of compressed breast thickness and dose on lesion detectability in digital mammography: FROC study with simulated lesions in real mammograms

Purpose: The aim of this work was twofold: (1) to examine whether, with standard automatic exposure control (AEC) settings that maintain pixel values in the detector constant, lesion detectability in clinical images decreases as a function of breast thickness and (2) to verify whether a new AEC setup can increase lesion detectability at larger breast thicknesses. Methods: Screening patient images, acquired on two identical digital mammography systems, were collected over a period of 2 yr. Mammograms were acquired under standard AEC conditions (part 1) and subsequently with a new AEC setup (part 2), programmed to use the standard AEC settings for compressed breast thicknesses ≤49 mm, while a relative dose increase was applied above this thickness. The images were divided into four thickness groups: T1 ≤ 29 mm, T2 = 30–49 mm, T3 = 50–69 mm, and T4 ≥ 70 mm, with each thickness group containing 130 randomly selected craniocaudal lesion-free images. Two measures of density were obtained for every image: a BI-RADS score and a map of volumetric breast density created with a software application (VolparaDensity, Matakina, NZ). This information was used to select subsets of four images, containing one image from each thickness group, matched to a (global) BI-RADS score and containing a region with the same (local) VOLPARA volumetric density value. One selected lesion (a microcalcification cluster or a mass) was simulated into each of the four images. This process was repeated so that, for a given thickness group, half the images contained a single lesion and half were lesion-free. The lesion templates created and inserted in groups T3 and T4 for the first part of the study were then inserted into the images of thickness groups T3 and T4 acquired with higher dose settings. Finally, all images were visualized using the ViewDEX software and scored by four radiologists performing a free search study. A statistical jackknife-alternative free-response receiver operating characteristic analysis was applied. Results: For part 1, the alternative free-response receiver operating characteristic curves for the four readers were 0.80, 0.65, 0.55 and 0.56 in going from T1 to T4, indicating a decrease in detectability with increasing breast thickness. P-values and the 95% confidence interval showed no significant difference for the T3-T4 comparison (p = 0.78) while all the other differences were significant (p < 0.05). Separate analysis of microcalcification clusters presented the same results while for mass detection, the only significant difference came when comparing T1 to the other thickness groups. Comparing the scores of part 1 and part 2, results for the T3 group acquired with the new AEC setup and T3 group at standard AEC doses were significantly different (p = 0.0004), indicating improved detection. For this group a subanalysis for microcalcification detection gave the same results while no significant difference was found for mass detection. Conclusions: These data using clinical images confirm results found in simple QA tests for many mammography systems that detectability falls as breast thickness increases. Results obtained with the AEC setup for constant detectability above 49 mm showed an increase in lesion detection with compressed breast thickness, bringing detectability of lesions to the same level.

60 APPLIED LIFE SCIENCES↗

SpaceCubeX: A Framework for Evaluating Hybrid Multi-Core CPU FPGA DSP Architectures

The SpaceCubeX project is motivated by the need for high performance, modular, and scalable on-board processing to help scientists answer critical 21st century questions about global climate change, air quality, ocean health, and ecosystem dynamics, while adding new capabilities such as low-latency data products for extreme event warnings. These goals translate into on-board processing throughput requirements that are on the order of 100-1,000 more than those of previous Earth Science missions for standard processing, compression, storage, and downlink operations. To study possible future architectures to achieve these performance requirements, the SpaceCubeX project provides an evolvable testbed and framework that enables a focused design space exploration of candidate hybrid CPU/FPGA/DSP processing architectures. The framework includes ArchGen, an architecture generator tool populated with candidate architecture components, performance models, and IP cores, that allows an end user to specify the type, number, and connectivity of a hybrid architecture. The framework requires minimal extensions to integrate new processors, such as the anticipated High Performance Spaceflight Computer (HPSC), reducing time to initiate benchmarking by months. To evaluate the framework, we leverage a wide suite of high performance embedded computing benchmarks and Earth science scenarios to ensure robust architecture characterization. We report on our projects Year 1 efforts and demonstrate the capabilities across four simulation testbed models, a baseline SpaceCube 2.0 system, a dual ARM A9 processor system, a hybrid quad ARM A53 and FPGA system, and a hybrid quad ARM A53 and DSP system.

Hybrid Flight Architectures↗