Search NASA⌕ Search

SEARCH · Search NASA

Results for “value of data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

AnalyzeThat: A Programmable Shared-Memory System for an Array of Processing-In-Memory Devices

Processing In Memory (PIM), the concept of integrating processing directly with memory, has been attracting a lot of attention since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-Aware Data Structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. Our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

97 MATHEMATICS AND COMPUTING↗

A programmable shared-memory system for an array of processing-in-memory devices

Processing in memory (PIM), the concept of integrating processing directly with memory has been attracting a lot of attention, since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-aware data structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a sophisticated key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. In conclusion, our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

97 MATHEMATICS AND COMPUTING↗

Complex network approach to fractional time series

In order to extract correlation information inherited in stochastic time series, the visibility graph algorithm has been recently proposed, by which a time series can be mapped onto a complex network. We demonstrate that the visibility algorithm is not an appropriate one to study the correlation aspects of a time series. We then employ the horizontal visibility algorithm, as a much simpler one, to map fractional processes onto complex networks. The degree distributions are shown to have parabolic exponential forms with Hurst dependent fitting parameter. Further, we take into account other topological properties such as maximum eigenvalue of the adjacency matrix and the degree assortativity, and show that such topological quantities can also be used to predict the Hurst exponent, with an exception for anti-persistent fractional Gaussian noises. To solve this problem, we take into account the Spearman correlation coefficient between nodes' degrees and their corresponding data values in the original time series.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Tethered Balloon System Merged Data (TBSMERGED) Value-Added Product Report

The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s Tethered Balloon System Merged Data (TBSMERGED) Value-Added Product (VAP) integrates data from instruments flown on ARM’s tethered balloon system missions that collect in situ measurements of temperature, humidity, wind speed, wind direction, and aerosol properties with estimates of cloud base and boundary-layer height from a surface-based ceilometer to improve the ease of use of tethered balloon system (TBS) data sets. The TBSMERGEDINCLOUD VAP, a variation of TBSMERGED, includes supercooled liquid water content (tbsslwc) measurements collected within the cloud.

54 ENVIRONMENTAL SCIENCES↗

KAZRARSCL-CLOUDSAT Value Added Product Data Stream

The KAZRARSCL-CLOUDSAT Value-Added Product (VAP) is based on the KAZR-ARSCL VAP, which provides cloud boundaries and best-estimate time-height fields of radar moments. The KAZRARSCL-CLOUDSAT VAP applies a statistically-derived calibration offset to reflectivity fields in order to align them with observations from the spaceborne CloudSat Cloud Profiling Radar. For details on the offset values applied, please refer to "Kollias, P., Puigdomènech Treserras, B., and Protat, A.: Calibration of the 2007–2017 record of ARM Cloud Radar Observations using CloudSat, Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2019-34, 2019."

54 ENVIRONMENTAL SCIENCES↗

Waste Glass Property Database and Data Qualification Plan

The U.S. Department of Energy vitrification facilities currently employ glass property-composition models to ensure processability of waste streams and acceptability of the final glass products. The efficiency (e.g., process flexibility, reduced down time, and reduced cost) of waste vitrification is directly tied to the size of the waste processing envelope. To increase the size of the processing envelope, a larger database was developed. The database will improve the prediction accuracy of glass properties, leading to higher waste loadings, higher waste throughput, broader process flexibility, and reduced mission life. Some of the data in this expanded database has insufficient quality assurance (QA) rigor for use in nuclear facility operation and glass qualification. A plan was developed to qualify high-value data under the appropriate QA rigor for use at the Hanford Waste Treatment and Immobilization Plant’s High-Level Waste (HLW) Facility. This qualification plan first compares the data coverage over the composition region of interest to Hanford HLW by property. Those studies that significantly improve the data coverage in the Hanford HLW glass composition region were prioritized for qualification activities. Qualification methods including QA equivalence, peer review, data corroboration, and confirmatory testing have been assigned to each high-priority study.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The CRUTEM4 land-surface air temperature data set: construction, previous versions and dissemination via Google Earth

The CRUTEM4 (Climatic Research Unit Temperature, version 4) land-surface air temperature data set is one of the most widely used records of the climate system. Here we provide an important additional dissemination route for this data set: online access to monthly, seasonal and annual data values and time series graphs via Google Earth. This is achieved via an interface written in Keyhole Markup Language (KML) and also provides access to the underlying weather station data used to construct the CRUTEM4 data set. A mathematical description of the construction of the CRUTEM4 data set (and its predecessor versions) is also provided, together with an archive of some previous versions and a recommendation for identifying the precise version of the data set used in a particular study. The CRUTEM4 data set used here is available from doi:10.5285/EECBA94F-62F9-4B7C-88D3-482F2C93C468.

54 ENVIRONMENTAL SCIENCES↗

Statistical projections for multi-dimensional visual data exploration

When working with large, multidimensional and multivariate data, science users are frequently interested in understanding variation in data, as opposed to the actual data values. Our work focuses on exploring how a simple statistical metric, the Coefficient of Variation (or Cv), can be used in several different ways to facilitate understanding variation in large data. As a statistical measure, it offers a key advantage over more widely accepted measures like standard deviation, namely to its ability to capture local variation properties. As a multidimensional projection operator, Cv is an effective way of reducing data size while preserving the key variational signal. Visualizations produced from Cv that target conveying variation in data are highly informative, especially compared to those produced with more widely known methods. We demonstrate these ideas within the context of a two-part application case study focusing on understanding long-Term trends in the the changes in precipitation and winds in large-scale climate model ensemble output.

Nguyen, H↗

Optimization of Error-Bounded Lossy Compression for Hard-to-Compress HPC Data

Since today’s scientific applications are producing vast amounts of data, compressing them before storage/transmission is critical. Results of existing compressors show two types of HPC data sets: highly compressible and hard to compress. In this work, we carefully design and optimize the error-bounded lossy compression for hard-tocompress scientific data. We propose an optimized algorithm that can adaptively partition the HPC data into best-fit consecutive segments each having mutually close data values, such that the compression condition can be optimized. Another significant contribution is the optimization of shifting offset such that the XOR-leading-zero length between two consecutive unpredictable data points can be maximized. We finally devise an adaptive method to select the best-fit compressor at runtime for maximizing the compression factor. We evaluate our solution using 13 benchmarks based on real-world scientific problems, and we compare it with 9 other state-of-the-art compressors. Experiments show that our compressor can always guarantee the compression errors within the user-specified error bounds. Most importantly, our optimization can improve the compression factor effectively, by up to 49% for hard-tocompress data sets with similar compression/decompression time cost.

97 MATHEMATICS AND COMPUTING↗

daft [SWR-17-53]

Daft is a Haskell library that implements the key constructs of relational algebra via static structural typing. Its DataCube typeclass supports selection, projection, aggregation, disaggregation, natural joins, semijoins, and antijoins. Instances of DataCube include both map- and function-based representations. An additional abstract data type provides Vinyl-style, keyed relational tables for which an embedded domain-specific language compactly represents relational operations. Utility types and functions support serialization to tab-separated-value data files, JSON arrays, MongoDB documents, and PostgreSQL tables, including automatic detection and verification of the schema for the input data.

Bush, BrianW.↗

Overview of the FAF3 Freight Flow Matrix Construction Process

PROJECT DESCRIPTION The FAF3 is a FHWA freight data product which provides a national O-D matrix of commodity flows to, from, and within the United States. FAF3 freight flows are reported in terms of both annual tons and annual dollars of freight moved by mode of transportation. Based largely on the 2007 CFS, FAF3 utilizes domestic freight flow characteristics, geographic regions, and the SCTG commodity coding system from CFS. However, many freight flows were not captured by the 2007 CFS due to scope and sample size limitations. Approximately 100,000 establishments were sampled out of some 754,000 freight moving establishments in 2007 and imports are out of scope entirely. To estimate missing data values, the approach taken in FAF3 was to use a combination of a novel Log-linear modeling approach (LLM) with an iterative proportional fitting (IPF) routine that also uses additional data inputs to fill in the missing pieces. The complete FAF3 O-D Commodity Mode database is made up of 131 Origins x 131 Destinations x 43 Commodity Classes x 8 Modal categories, for annual tons and dollars. This poster illustrates how the 2007 CFS data were integrated with several additional data sources using LLM and IPF to create a comprehensive FAF3 national freight flow matrix. More detailed documentation on the sources and methods utilized in the development of FAF3 are available from the FHWA website at the following website: http://www.ops.fhwa.dot. gov/freight/freight_analysis/faf/index.htm.

99 GENERAL AND MISCELLANEOUS↗

Nuclear Data Sheets for A=269,273,277,281,285,289, and 293

Herein, information on the production, identification, half-lives and decay modes for experimentally known nuclides with mass numbers of 269, 273, 277, 281, 285, 289, and 293 are presented along with the recommended values. Data here supersede data from the previous evaluation in this region, 2005Gu33. Since the previous evaluation many new and conclusive experiments have explored the region of superheavy nuclei, thus significantly expanding our knowledge of the nuclear chart in this area.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DEEPEN Leapfrog Geodata Model Cleaned and Reformatted Exploration Datasets from Newberry Volcano

DEEPEN stands for DE-risking Exploration of geothermal Plays in magmatic ENvironments. As part of the DEEPEN 3D play fairway analysis (PFA) conducted at Newberry Volcano for multiple play types (conventional hydrothermal, superhot EGS, and supercritical), existing geoscientific exploration datasets needed to be acquired, cleaned, reformatted, and assembled in Leapfrog Geothermal. This GDR submission includes all of the cleaned and reformatted (X (m), Y (m), elevation (m), processed data values) datasets used to build the Leapfrog Geodata model. Existing datasets were acquired from the GDR, from AltaRock, and from other sources. This yielded the following datasets: - Digital elevation model produced from LiDAR data by Ramsey and Bard, 2016 - MT surveys from 2006, 2011, 2014, and 2017 (including single inversions) - Gravity surveys from 2006, 2007, and 2011 (including single) - Earthquake catalogs from PNSN, LLNL, and the Newberry EGS Demonstration project - Seismic velocity model from Templeton et al., 2014 - The Frone, 2015 temperature model and a new one produced through extrapolating downhole temperature measurements and the SMU temperature at depth maps. Two versions of the new model are provided: 250 m spacing and 500 m spacing - EarthVision geologic model with alteration from Moser et al., 2016 - Well data from EGS well 55-29, deep geothermal wells, coreholes (GEO N-2 through 5) and several thermal gradient holes - "Newberry Well Data:" Location, simple lithology, directional survey data, and temperature data for the 34 wells and coreholes used in the Newberry PFA Although there are additional 2D datasets available in the area, such as aeromagnetic surveys, these were not included in the analysis. While it may be possible to project these datasets into three dimensions by assuming the surface measurements do not vary with depth, this method is associated with high uncertainty. Preexisting inversions of these data were unavailable, and inverting additional geophysical datasets is outside the scope of this project.

15 GEOTHERMAL ENERGY↗

DS-GL: Advancing Graph Learning via Harnessing the Power of Nature within Dynamic Systems

With the rapid digitization of the world, an increasing number of real-world applications are turning to nonEuclidean data, modeled as graphs. Due to their intrinsic high complexity and irregularity, learning from graph data demands tremendous computational power. Recently, CMOS-compatible Ising machines, i.e., dynamic systems composed of CMOS components, have emerged as a new approach that harnesses the inherent power of natural annealing within dynamic systems to efficiently resolve binary optimization problems and have been adopted for traditional graph computation, such as max-cut. However, when performing complex Graph Learning (GL) tasks, Ising machines face significant hurdles: (i) they are inherently binary and thus ill-suited for real-valued problems; (ii) their expensive all-to-all coupling network that guarantees effective natural annealing poses daunting scalability concerns. To address these challenges, this paper proposes a nature-powered graph learning framework dubbed DS-GL, which is the first effort to transform the process of solving graph learning problems into the natural annealing process within a parameterized dynamic system embodied as a CMOS chip. To tackle the two major hurdles, DS-GL first augments the Ising machine architecture to modify the self-reaction term of its Hamiltonian function from linear to quadratic, effectively serving as an energy regulator. This adjustment maintains the system’s original physical interpretation while enabling it to process continuous, real-valued data. Second, to address the scaling issue, DS-GL further upgrades the real-valued dense Ising machine by decomposing it into a mesh-based multi-PE dynamic system that supports efficient distributed spatial-temporal co-annealing across different PEs through sparse interconnects. By exploiting the inherent sparsity and component structures in real-world graphs, DS-GL is able to map complex graph learning tasks onto the scalable dynamic system while maintaining high accuracy. Evaluations with three diverse GL applications across six real-world datasets, including traffic flow and COVID-19 prediction, show that DS-GL can deliver from 102× to 106× speedups and 500× energy reduction over Graph Neural Networks on GPUs, with 5% - 20% accuracy enhancement.

Song, Ruibing↗

Water quality data collected from the Muskegon River (MI, USA) using AquaBOT June-July 2024.

This dataset contains processed output from AquaBOT, which combines 30-second interval measurements of GPS location and YSI water quality parameters (temperature (degrees Celsius), dissolved oxygen (mg/l), specific conductance(microSiemens/cm at 25 degrees Celsius), and turbidity (NTU)), for 3 sections of the Muskegon River in Michigan, USA. The file aquabot2024_MuskegonRiver.csv has information on locations, dates and times, and observations. The data were processed by removing observations recorded before and after AquaBOT was in the water and observations where no data values were recorded. No other QA/QC was done. This research was performed as part of the DOE Research Development and Partnership Pilot award “Expanding Collaborative Capacity to Address Climate Resiliency in the Great Lakes Region”, which aims to expand collaborations between researchers at Central Michigan University and U.S. Department of Energy labs and projects focused on enhancing climate resilience in Great Lakes communities and ecosystems.

54 ENVIRONMENTAL SCIENCES↗

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗

AESD (Advanced Energy System Design) Records API [SWRs-17-50, 17-51, 17-52]

The Records API (application program interface) for Advanced Energy System Design (AESD) enables software that serves multidimensional record-oriented data to interoperate with software than uses such data. In the context of the Records API, multidimensional data records are simply tuples of real numbers, integers, and character strings, where each data value is tagged by a variable name, according to a pre-defined schema, and each record is assigned a unique integer identifier. Conceptually, these records are isomorphic to rows in a relational database, JSON objects, or key-value maps. Records servers might supply static datasets, sensor measurements that periodically update as new telemetry become available, or the results of simulations as the simulations generate new output. Records client software might display or analyze the data, but in the case of simulations the client request the creation of new ensembles for specified input parameters. It is also possible to chain records clients and servers together so that a client consuming data from a server might transform that data and serve it to additional clients. This minimalist API avoids imposing burdensome metadata, structural, or implementation requirements on developers by relying on open-source technologies that are readily available for common programming languages. In particular, the API has been designed to place the least possible burden on services that provide data. This document defines the message format for the Records API, a transport mechanism for communicating the data, and the semantics for interpreting it. The message format is specified as Google Protocol Buffers and the transport mechanism uses WebSockets. We discuss three major use cases for serving and consuming records data: (i) static data, (ii) dynamically augmented data, (iii) on-demand simulations, (iv) with filters, and (v) with bookmarks. Separate implementations of the API exist in C++, Haskell, JavaScript, Python, and R.

Bush, Brian↗

Advanced Energy System Design (AESD): Technical Manual for the Records API

The Records API (application program interface) for Advanced Energy System Design (AESD) enables software that serves multidimensional record-oriented data to interoperate with software than uses such data. In the context of the Records API, multidimensional data records are simply tuples of real numbers, integers, and character strings, where each data value is tagged by a variable name, according to a pre-defined schema, and each record is assigned a unique integer identifier. Conceptually, these records are isomorphic to rows in a relational database, JSON objects, or key-value maps. Records servers might supply static datasets, sensor measurements that periodically update as new telemetry become available, or the results of simulations as the simulations generate new output. Records client software might display or analyze the data, but in the case of simulations the client request the creation of new ensembles for specified input parameters. It is also possible to chain records clients and servers together so that a client consuming data from a server might transform that data and serve it to further clients. This minimalist API avoids imposing burdensome metadata, structural, or implementation requirements on developers by relying on open-source technologies that is readily available for common programming languages. In particular, the API has been designed to place the smallest possible burden on services that provide data. This document defines the message format for the Records API, a transport mechanism for communicating the data, and semantics for interpreting it. The message format is specified as Google Protocol Buffers and the transport mechanism uses WebSockets. We discuss three major use cases for serving and consuming records data: (i) static data, (ii) dynamically augmented data, (iii) on-demand simulations, (iv) with filters, and (v) with bookmarks. Separate implementations of the API exist in C++, Haskell, JavaScript, Python, and R.

97 MATHEMATICS AND COMPUTING↗