Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Scientific Discovery with Physics-Informed System Identification (Abbreviated Report)

My fellowship research focused on making physics-based simulations faster and more useful through machine learning. Many problems in science and engineering are governed by partial differential equations, but high-fidelity simulations are often too expensive to run repeatedly. I worked on improving Latent Space Dynamics Identification (LaSDI), a reduced-order modeling framework that compresses large simulation data sets into a smaller representation and then learns how that representation evolves over time. The motivation was to develop reduced models that remain accurate for more challenging systems, especially when predictions must remain reliable over long time intervals or when the underlying dynamics are more complicated than standard methods can easily handle. I also contributed to related work on Quandary, a high-performance software effort for simulation and control of open quantum systems, before focusing primarily on Latent Space Dynamics Identification methods. The main outcomes of the fellowship were two new algorithms (both of which were published), Rollout-LaSDI and Higher-Order LaSDI, together with supporting work on multi-stage Latent Space Dynamics Identification. Rollout-LaSDI improved long-term prediction by training the model to stay accurate over extended time horizons, and Higher-Order LaSDI broadened the method so it could model systems with higher-order time dynamics. My contributions to multistage Latent Space Dynamics Identification also helped show that its later training stages could be simplified without losing effectiveness, and that this behavior held across different model architectures and training strategies. Taken together, these advances improved the accuracy, flexibility, and practical value of reduced-order modeling tools for computational science.

97 MATHEMATICS AND COMPUTING↗

Baropycnal Work: A Mechanism for Energy Transfer across Scales

The role of baroclinicity, which arises from the misalignment of pressure and density gradients, is well-known in the vorticity equation, yet its role in the kinetic energy budget has never been obvious. Here, we show that baroclinicity appears naturally in the kinetic energy budget after carrying out the appropriate scale decomposition. Strain generation by pressure and density gradients, both barotropic and baroclinic, also results from our analysis. These two processes underlie the recently identified mechanism of “baropycnal work”, which can transfer energy across scales in variable density flows. As such, baropycnal work is markedly distinct from pressure-dilatation into which the former is implicitly lumped in Large Eddy Simulations. We provide numerical evidence from 1024 3 direct numerical simulations of compressible turbulence. The data shows excellent pointwise agreement between baropycnal work and the nonlinear model we derive, supporting our interpretation of how it operates.

42 ENGINEERING↗

Full Spectrum, 350 - 2500 nm, Leaf and Canopy Spectral Reflectance, Seward Peninsula, Alaska, 2017

Full-range (350 - 2500 nm) leaf and canopy reflectance spectra of various Arctic tundra ecosystem endmembers, including species-level leaf reflectance, canopy-scale species endmember spectra, plot-scale spectra, and transect spectra as well as non-vegetated surface (NVS) spectra. The datasets were collected at the three core NGEE-Arctic watersheds, Kougarok, Teller, and Council within the larger Seward Peninsula, Alaska region. The data were collected in the months of July and August of 2017 using a full-range Spectra Vista Corporation (SVC) HR-1024i spectroradiometer. Leaf-level spectra were collected with the original SVC leaf clip/plant probe connected to the spectrometer through a 1.15 meter long fiber optic cable, while canopy-scale reflectance was collected with an 8-degree field-of-view (FOV) foreoptic lens. All spectral measurements were collected as calibrated surface radiance and converted to surface reflectance using a 99.99% reflective Spectralon white reference standard. For those canopy spectra collected with associated functional trait data, the FOV of the instrument was positioned to include the same leaves harvested for functional trait measurements, including leaf mass per area (LMA) and foliar carbon and nitrogen content (see associated dataset). This data package includes 26 files in a variety of formats including processed canopy and leaf spectra (*.csv), processed dGPS locations (*.csv and *.kmz), digital photographs of spectral targets (*.jpg) and raw data from spectroradiometer and dGPS instruments (compressed as tar.gz). Metadata files include data descriptions (_dd.csv) for tabular data and a key to species symbols used in data files. All included files are listed and described in NGA110_flmd.csv.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Full spectrum (350-2500 nm) canopy spectral reflectance, Seward Peninsula, Alaska, 2018

Full-range (350-2500 nm) canopy reflectance spectra of various Arctic tundra ecosystem endmembers, including canopy-scale species endmember spectra, plot-scale spectra, and transect spectra as well as non-vegetated surface (NVS) spectra. The datasets were collected at the three core NGEE-Arctic watersheds, Kougarok, Teller, and Council, as well as the supplemental Kougarok Mile 80 site within the larger Seward Peninsula, Alaska region. The data were collected in July 2018 using a full-range Spectra Vista Corporation (SVC) HR-1024i spectroradiometer. Canopy-scale reflectance was collected with an 8-degree field-of-view (FOV) foreoptic lens. All spectral measurements were collected as calibrated surface radiance and converted to surface reflectance using a 99.99% reflective Spectralon white reference standard. For those canopy spectra collected with associated functional trait data, the FOV of the instrument was positioned to include the same leaves harvested for functional trait measurements, including leaf mass per area (LMA) and foliar carbon and nitrogen content (see associated dataset). Species included in the spectra and functional trait dataset include Alnus viridis, Betula nana, Betula glandulosa, Arctous alpina, and Salix pulchra. This data package includes 35 files in a variety of formats including processed canopy spectra (*.csv), processed dGPS locations (*.csv and *.kmz), digital photographs of spectral targets (*.jpg) and raw data from spectroradiometer and dGPS instruments (compressed as tar.gz). Metadata files include data descriptions (_dd.csv) for tabular data and a key to species symbols used in data files. All included files are listed and described in NGA208_flmd.csv.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Full spectrum (350-2500 nm) leaf and canopy spectral reflectance, Seward Peninsula, Alaska, 2019

Full-range (350 - 2500 nm) leaf and canopy reflectance spectra of various Arctic tundra ecosystem endmembers, including species-level leaf reflectance, canopy-scale species endmember spectra, plot-scale spectra, and transect spectra. These data were collected in support of ongoing NASA ABoVE AVIRIS data synthesis work. The datasets were collected at the NGEE-Arctic watershed sites, Kougarok MM64 and MM80, and Teller MM27 on the Seward Peninsula, Alaska. The data were collected in July 2019 using a full-range Spectra Vista Corporation (SVC) HR-1024i spectroradiometer. Leaf-level spectra were collected with the original SVC leaf clip/plant probe connected to the spectrometer through a 1.15 meter long fiber optic cable, while canopy-scale reflectance was collected with an 8-degree field-of-view (FOV) foreoptic lens. All spectral measurements were collected as calibrated surface radiance and converted to surface reflectance using a 99.99% reflective Spectralon white reference standard. For those canopy spectra collected with associated functional trait data, the FOV of the instrument was positioned to include the same leaves harvested for functional trait measurements, including leaf mass per area (LMA) and foliar carbon and nitrogen content (see leaf trait data package for these data). Canopy spectra were obtained for 23 different species, and leaf spectra for 8 species. This data package includes processed canopy and leaf spectra (*.csv), processed dGPS locations (*.csv and *.kmz), digital photographs of spectral targets (*.jpg) and raw data from spectroradiometer and dGPS instruments (compressed as tar.gz). Metadata files include data descriptions (_dd.csv) for tabular data and a key to species symbols used in data files. All included files are listed and described in NGA212_flmd.csv.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

NLR HPC Eagle Node Power Data

Power time series captured from all Eagle nodes using iLO (Integrated Lights Out) The Eagle HPC operated at NLR from 2019 through 2024. Eagle was a 2,000-node, 8-petaflop system. This dataset is a comprehensive time series of instantaneous snapshots of power usage at 1 minute intervals from all nodes at the node level. Data provided in compressed Hive dataset/Parquet format. iLO Power Time Series Fields ts: Timestamp dv: Device / Node - Rack and Unit - r103u17 == r(ack)103u(nit)17 vl: Value - Value in watts (instantaneous value at sampling time) day month year

97 MATHEMATICS AND COMPUTING↗

NLR HPC Eagle GPU Node Metrics

Ganglia node metrics and iLO (Integrated Lights Out) power data captured from six representative Eagle GPU nodes The Eagle HPC operated at NLR from 2019 through 2024. Eagle was a 2,000-node, 8-petaflop system. This dataset is a representative sample of metrics for 6 of the GPU nodes. Each GPU node contained 2 CPUs and 2 GPUs. Data provided in compressed CSV format. Ganglia and iLO Power Time Series Fields ts: Timestamp dv: Device / Node - Rack and Unit - r103u17 == r(ack)103u(nit)17 mt: Metric (only present for Ganglia) vl: Value - Value in watts for iLO power (instantaneous value at sampling time) or specified Ganglia metric below Ganglia Metrics Metric name -- Metric description -- Unit cpu_aidle -- Percent of time since boot idle CPU -- Percent cpu_idle -- Percent CPU idle -- Percent cpu_nice -- Percent CPU nice -- Percent cpu_speed -- Speed in MHz of CPU -- MHz cpu_user -- Percent CPU user -- Percent cpu_wio -- The percentage of CPU Wait I/O -- Percent gpu0_bar1_memory -- Used GPU bar1 memory -- MB gpu0_decoder_util -- GPU decoder utilization -- Percent gpu0_ecc_db_error -- Total ECC error counts for the GPU -- Number gpu0_encoder_util -- GPU encoder utilization -- Percent gpu0_fan -- Fan speed -- RPM gpu0_fb_memory -- Used GPU framebuffer memory -- MB gpu0_graphics_clock_report -- Current clock speeds for the device -- MHz gpu0_mem_total -- Memory total -- MB gpu0_mem_util -- Memory utilization -- Percent gpu0_power_usage_report -- Power usage report -- Watts gpu0_temp -- GPU 1 temperature -- Celsius gpu1_bar1_memory -- Used GPU bar1 memory -- MB gpu1_decoder_util -- GPU decoder utilization -- Percent gpu1_ecc_db_error -- Total ECC error counts for the GPU -- Number gpu1_encoder_util -- GPU encoder utilization -- Percent gpu1_fan -- Fan speed -- RPM gpu1_fb_memory -- Used GPU framebuffer memory -- MB gpu1_graphics_clock_report -- Current clock speeds for the GPU -- MHz gpu1_mem_total -- Memory total -- MB gpu1_mem_util -- Memory utilization -- MB gpu1_power_usage_report -- Power usage report -- Watts gpu1_temp -- GPU 1 temperature -- Celsius ipmi_cpu1_temp -- CPU 1 temperature -- Celsius ipmi_cpu2_temp -- CPU 2 temperature -- Celsius ipmi_inlet_ambient_temp -- Temperature measured at intake -- Celsius ipmi_vr_p1_temp -- CPU 1 voltage regulator temperature -- Celsius ipmi_vr_p2_temp -- CPU 2 voltage regulator temperature -- Celsius mem_buffers -- Amount of buffered memory -- Bytes mem_cached -- Amount of cached memory -- Bytes mem_free -- Amount of available memory -- Bytes mem_shared -- Amount of shared memory -- Bytes mem_total -- Amount of available memory -- Bytes

97 MATHEMATICS AND COMPUTING↗

Detection of Modified Matrix Encoding Using Machine Learning and Compressed Sensing

In recent years, driven by the development of steganalysis methods, steganographic algorithms have been evolved rapidly with the ultimate goal of an unbreakable embedding procedure, resulting in recent steganographic algorithms with minimum distortions, exemplified by the recent family of Modified Matrix Encoding (MME) algorithms, which has shown to be most difficult to be detected. In this paper we propose a compressed sensing based on approach for intrinsic steganalysis to detect MME stego messages. Compressed sensing is a recently proposed mathematical framework to represent an image (in general, a signal) using a sparse representation relative to an overcomplete dictionary by minimizing the l1-norm of resulting coefficients. Here we first learn a dictionary from a training set so that the performance will be optimized using the KSVD algorithm; since JPEG images are processed by 8x8 blocks, the training examples are 8x8 patches, rather than the entire images and this increases the generalization of compressed sensing. For each 8x8 block, we compute its sparse representation using OMP (orthogonal matching pursuit) algorithm. Using computed sparse representations, we train a support vector machine (SVM) to classify 8x8 blocks into stego and non-stego classes. Then given an input image, we first divide it into 8x8 blocks. For each 8x8 block, we compute its sparse representation and classify it using the trained SVM. After all the 8x8 blocks are classified, the entire image is classified based on the majority rule of 8x8 block classification results. This allows us to achieve a robust decision even when 8x8 blocks can be classified only with relatively low accuracy. We have tested the proposed algorithm on two datasets (Corel-1000 dataset and a remote sensing image dataset) and have achieved 100% accuracy on classifying images, even though the accuracy of classifying 8x8 blocks is only 80.89%. Key Words Compressed Sensing, Sparcity, Data Dictionary, Steganography, steganalysis

Allen, Josef D↗

Full information acquisition in scanning probe microscopy and spectroscopy

Apparatus and methods are described for scanning probe microscopy and spectroscopy based on acquisition of full probe response. The full probe response contains valuable information about the probe-sample interaction that is lost in traditional scanning probe microscopy and spectroscopy methods. The full probe response is analyzed post data acquisition using fast Fourier transform and adaptive filtering, as well as multivariate analysis. The full response data is further compressed to retain only statistically significant components before being permanently stored.

36 MATERIALS SCIENCE↗

A blessing in disguise : Characterisations of ProtoDUNE photon showers for neutrino measurements in DUNE

There are few areas of particle physics as open to new discoveries as the neutrino sector. Although these particles are among the most abundant in the universe, their weak coupling to ordinary matter makes them a challenge to study. The Deep Underground Neutrino Experiment (DUNE) is an upcoming precision experiment in neutrino physics planned to be built between Illinois and South Dakota in the US. The most powerful neutrino beam in the world will be aimed at the largest liquid argon time projection chamber in the world through 1300 km of solid rock. In order to prototype this detector, a scale model was constructed and operated at CERN in 2018 called ProtoDUNE. This thesis presents three studies performed on this detector. The first concerns the readout of detector data by the FELIX data acquisition system, as well as compression studies performed on the same data. The second study concerns the identification of photons created in neutral pion decay, which form a major background to neutrino-produced electrons. The distinction between electrons and photons is based on the differing energy deposition in the initial part of the particle showers they produce. The third study concerns the energy reconstruction of particle showers in the detector. Although photon showers created from neutral pion decay are a major background to the eventual neutrino signal, they have a known energy profile and can thus be used to calibrate the energy reconstruction of particle showers in general, a vital requirement for DUNE’s physics goals.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Snowmass Topical Group Summary Report: IF04 -- Trigger and Data Acquisition Systems

A trend for future high energy physics experiments is an increase in the data bandwidth produced from the detectors. Datasets of the Petabyte scale have already become the norm, and the requirements of future experiments -- greater in size, exposure, and complexity -- will further push the limits of data acquisition technologies to data rates of exabytes per seconds. The challenge for these future data-intensive physics facilities lies in the reduction of the flow of data through a combination of sophisticated event selection in the form of high-performance triggers and improved data representation through compression and calculation of high-level quantities. These tasks must be performed with low-latency (i.e. in real-time) and often in extreme environments including high radiation, high magnetic fields, and cryogenic temperatures.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Snowmass Topical Group Summary Report: IF04 -- Trigger and Data Acquisition Systems

A trend for future high energy physics experiments is an increase in the data bandwidth produced from the detectors. Datasets of the Petabyte scale have already become the norm, and the requirements of future experiments -- greater in size, exposure, and complexity -- will further push the limits of data acquisition technologies to data rates of exabytes per seconds. The challenge for these future data-intensive physics facilities lies in the reduction of the flow of data through a combination of sophisticated event selection in the form of high-performance triggers and improved data representation through compression and calculation of high-level quantities. These tasks must be performed with low-latency (i.e. in real-time) and often in extreme environments including high radiation, high magnetic fields, and cryogenic temperatures.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Carbon Mitigation Potential of Heat Pump Integrated with Thermal Storage for Grid-Interactive Residential Buildings

Residential buildings’ heating and cooling loads are associated with significant carbon emissions and peak electricity demand. Phase change material (PCM) based thermal energy storage (TES) can be used for space heating and cooling by embedding into the heat pump equipment. Past work on TES integrated with heat pumps (HP) has demonstrated significant load shifting and economic benefits. However, the potential for TES to reduce carbon emissions has not been widely explored. This study evaluates carbon mitigation potential of an ice-based TES coupled to HP (HP-TES) based on a simple rule-based control strategy accounting for electric grid emissions data. A vapor compression HP model using Engineering Equation Solver (EES). The modeled HP-TES system for single-family residential building demonstrates decreased carbon emissions with reduced peak utility cost.

Sultan, Sara↗

INL Senior Project

What did my team set out to accomplish:? Can I put a custom Machine Learning Model on FPGA?? Can I analyze network traffic in real time?? Can a QSFP port be used with an FPGA?? Does a visual representation of the latent space enhance our understanding of network traffic?? What is QSFP QSFP (Quad Small Form-Factor Pluggable)? QSFP supports transfer speeds generally up to 100Gb/s? Runs 4 parallel lines running up to 28 Gb/s? Why the latent space is important to our project ?Latent space is the compressed mapping of data points in a non-linear fashion? Create an understanding of the relationship of data collected? Can represent that relationship of a single network packet in 3 points (X, Y, Z) Project Outline FPGA? Custom Xilinx Petalinux Image for the operating system? Python program to collect packets and run them through the DPU (Data Processing Unit)? The program then sends the information over a socket to a computer? Display Program? Python Program that collects the information sent from the FPGA and display it in a graph

99 GENERAL AND MISCELLANEOUS↗

A Blessing in Disguise -- Characterisations of ProtoDUNE photon showers for neutrino measurements in DUNE

This thesis presents three studies performed on ProtoDUNE, a prototype liquid argon time projection chamber for the Deep Underground Neutrino Experiment (DUNE). The first study concerns the readout of detector data by the FELIX data acquisition system, as well as compression studies performed on the same data. The second study concerns the identification of photons created in neutral pion decay, which form a major background to neutrino-produced electrons. The distinction between electrons and photons is based on the differing energy deposition in the initial part of the particle showers they produce. The third study concerns the energy reconstruction of these particle showers in the detector. They have a known energy profile and can thus be used to calibrate the energy reconstruction of particle showers in general, a vital requirement for DUNE’s physics goals.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Utah FORGE: Discrete Fracture Network (DFN) Data

The FORGE team is making these fracture models available to researchers wanting a set of natural fractures in the FORGE reservoir for use in their own modeling work. They have been used to predict stimulation distances during hydraulic stimulation at the open toe section of well 16A(78)-32. These fracture sets are fully stochastic and do not contain the deterministic set that matches the pilot well 58-32 FMI data. Well 58-32 has been completed and 16A(78)-32 is to be drilled as part of Phase 3. The original .fab files are not included due to redundancy. The *.fabgz data for the 800m and 1200m depth areas are in the native FracMan format and have been compressed using Gzip. Filtered data for the 800m depth area includes .csv spreadsheets, native FracMan (.fab), and GOCAD (.ts) files that are in a compressed zip format. The file titled "SGW 2020 Finnila and Podgorney DFN fracture files on GDR.pdf" is a description of the data and should be reviewed prior to data use.

15 GEOTHERMAL ENERGY↗

Next Steps for Compressively Sensed Video in Transmission Electron Microscopy

Here, compressive sensing (CS) is a fast-emerging approach for improving data throughput by taking advantage of the inherent compressibility of real world data—the same kind of compressibility that allows millions of users to stream full-screen video without breaking the internet. Transmission electron microscope (TEM) users regularly collect multi-GB data sets and then use aggressive data reduction strategies to draw out the tiny residual of Shannon entropy that answers the question they wanted to ask. Much of the raw-byte-count information in an acquisition is random, redundant, or irrelevant, yet its collection still represents substantial cost in terms of instrumentation bandwidth and sample damage.

47 OTHER INSTRUMENTATION↗