Search NASA⌕ Search

SEARCH · Search NASA

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

SharP Data Constructs: Data Constructs to Enable Data-Centric Computing

Extreme-scale applications (i.e., Big-Compute) are becoming increasingly data-intensive, i.e., producing and consuming increasingly large amounts of data. The HPC systems traditionally used for these applications are now used for Big-Data applications such as data analytics, social network analysis, machine learning, and genomics. As a consequence of these trends, the system architecture should be flexible and data-centric. This can already be witnessed in the pre-exascale systems with TBs of on-node hierarchical and heterogeneous memories, PBs of system memory, low-latency, high-throughput networks, and many threaded cores. As such, the pre-exascale systems suit the needs of both Big-Compute and Big-Data applications. Though the system architecture is flexible enough to support both Big-Compute and Big-Data, we argue there is a software gap. Particularly, we need data-centric abstractions to leverage the full potential of the system, i.e., there is a need for native support for data resilience, the ability to express data locality and affinity, mechanisms to reduce data movement, the ability to share data, and abstractions to express User's data usage and data access patterns. In this paper, we (i) show the need for taking a holistic approach towards data-centric abstractions, (ii) show how these approaches were realized in the SHARed data-structure centric Programming abstraction (SharP) library, a data-structure centric programming abstraction, and (iii) apply these approaches to a variety of applications that demonstrate its usefulness. Particularly, we apply these approaches to QMCPack and the Graph500 benchmark and demonstrate the advantages of this approach on extreme-scale systems.

Aderholdt, William Ferrol↗
DOE OSTI2021

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository: Preprint

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗
DOE OSTI2021

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗
DOE OSTI2017

Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures

Data quality control is one of the most time consuming activities within Research Infrastructures (RIs), especially when involving observational data and multiple data providers. In this work we report on our ongoing development of data rogues, a scalable approach to manage data quality issues for observational data within RIs. The motivation for this work started with the creation of the FLUXNET2015 dataset, which includes carbon, water, and energy fluxes plus micrometeorological and ancillary data measured in over 200 sites around the world. To create an uniform dataset, including derived data products, extensive work on data quality control was needed. The unpredictable nature of observational data quality issues makes the automation of data quality control inherently difficult. Developed based on this experience, the data rogues methodology allows for increased automation of quality control activities by systematically identifying, cataloging, and documenting implementations of solutions to data issues. We believe this methodology can be extended and applied to others domains and types of data, making the automation of data quality control a more tractable problem.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Open-Source Framework for Data Storage and Visualization of Real-Time Experiments

Digital real time simulators (DRTS) are increasingly being used for the evaluation of power hardware and controller hardware in the laboratory prior to field deployment. Although DRTS are capable of simulating large models in real-time, it is challenging to visualize the results of large models without overdrawing or confounding the viewer. This paper provides an open-source framework for users to visualize their DRTS-based hardware-in-the-loop (HIL) experimental results in real-time. This proposed framework can be used by experimental test beds that can push data through an internet protocol based network. The proposed framework includes three main components. First, it includes the DRTS that generates and pushes the data to a relay. Second, it includes an application that serves multiple purposes, from data storage, testing, and translation of the data to a publisher/subscriber protocol. Finally, it includes libraries and applications that can be used to visualize the data by subscribing to the relay. This framework is available in open source, and it is tested using the HIL platform developed for testing advanced distribution management systems.

advanced distribution management systems↗
DOE OSTI2022

Machine Learning Based Network Parameter Estimation Using AMI Data

The expansion of distribution power system and the growing penetration of distributed energy resources present new challenges for situational awareness. Calibrating the extended system model with sensor measurements and maintaining the usability is critical for utilities. This paper presents a distribution network parameter estimation (DNPE) approach using machine learning (ML) and metering data that improve the quality of extended distribution power system modeling. The reliability model can improve the ability of endpoint data to be translated into network-level situational awareness in real time and help distribution system operators (DSOs) solve branch flow and voltage problems. In addition, a data analytic and automate processing scheme is proposed to improve the sensor data quality and prevent misleading information. The effectiveness of the proposed method is verified with actual advanced metering infrastructure (AMI) data on a real utility feeder model, while considering the higher penetration of photovoltaic power generation. The test of DNPE and study results are demonstrated in this paper.

Parameter estimation, machine learning, power dist↗
DOE OSTI2019

Assessing overall building energy performance of a large population of residential single-family homes using limited field data

The U.S. Department of Energy (DOE), through the Building Energy Codes Program (BECP), supports building energy code development and implementation processes to achieve the maximum practicable, cost-effective improvements in energy efficiency while providing safe, healthy buildings for occupants. Currently, there is a lack of statistically representative data that shows how energy codes are implemented in the field and the energy savings opportunities that exist. To address this gap, DOE initiated a research study targeted at new residential single-family homes in 8 states starting in the fall of 2014. The study comprises of a baseline phase to determine current levels of compliance with the key energy provisions of the residential building energy code, identify issues, and calculate potential energy savings from improved compliance; a second phase implementing education, training and outreach activities designed to increase targeted code compliance; and a post-study phase to measure the impact of the training and outreach activities compared to the baseline phase. Initial analyses determined the code requirements with the largest energy impact and the minimum number of observations required to detect statistically significant differences in energy use. Data was collected in the field during a single site visit to avoid bias from repeat visits. However, the single site visit principle prevented energy simulation of individual homes due to the lack of all required model inputs. To address this, energy simulation was conducted using Pacific Northwest National Laboratory’s prototype building models and employing bootstrap random sampling on the data collected from all homes visited. The modeled energy consumption using the observed field data was evaluated relative to that associated with the minimum code requirements and potential savings associated with improved compliance of select items were assessed. This paper discusses the methodology developed for this field study, and presents the results and findings from the baseline phase.

Xie, YuLong↗
DOE OSTI2018

The Dark Energy Survey Data Release I

We describe the first public data release of the Dark Energy Survey, DES DR1, consisting of reduced single-epoch images, co-added images, co-added source catalogs, and associated products and services assembled over the first 3 yr of DES science operations. DES DR1 is based on optical/near-infrared imaging from 345 distinct nights (2013 August to 2016 February) by the Dark Energy Camera mounted on the 4 m Blanco telescope at the Cerro Tololo InterAmerican Observatory in Chile. We release data from the DES wide-area survey covering similar to 5000 deg(2) of the southern Galactic cap in five broad photometric bands, grizY. DES DR1 has a median delivered point-spread function of g = 1.12, r = 0.96, i = 0.88, z = 0.84, and Y = 0.'' 90 FWHM, a photometric precision of <1% in all bands, and an astrometric precision of 151 mas. The median co-added catalog depth for a 1.'' 95 diameter aperture at signal-to-noise ratio (S/N) = 10 is g = 24.33, r = 24.08, i = 23.44, z = 22.69, and Y = 21.44 mag. DES DR1 includes nearly 400 million distinct astronomical objects detected in similar to 10,000 co-add tiles of size 0.534 deg(2) produced from similar to 39,000 individual exposures. Benchmark galaxy and stellar samples contain similar to 310 million and similar to 80 million objects, respectively, following a basic object quality selection. These data are accessible through a range of interfaces, including query web clients, image cutout servers, jupyter notebooks, and an interactive co-add image visualization tool. DES DR1 constitutes the largest photometric data set to date at the achieved depth and photometric precision.

Abbott, T. M. C.↗
DOE OSTI2023

Connecting People to Data: Enabling Data Connected Communities through Enhancements to the Geothermal Data Repository: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented a series of new features designed to connect people to data. These features, which are based on feedback from the GDR user community and surveys of the greater geothermal research community, are designed to improve data quality and empower members of all communities to better engage with geothermal data resources by providing universal access to data and by improving the connections between data providers, subject matter experts, and the communities of people using GDR data. This paper will explore some of the recent enhancements made to the GDR to improve data discoverability, reduce submission time, and result in better quality data submissions. These improvements include the ability for users to save a list of their favorite datasets, search for insight into geothermal datasets or data availability, or sign up to receive notifications of future updates to specific datasets. These improvements aim to enhance the overall user experience of the GDR while further connecting communities to the data they need to inform decisions, advance geothermal research, and develop innovative solutions to local energy problems.

DOE↗
DOE OSTI2023

Connecting People to Data: Enabling Data Connected Communities through Enhancements to the Geothermal Data Repository

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented a series of new features designed to connect people to data. These features, which are based on feedback from the GDR user community and surveys of the greater geothermal research community, are designed to improve data quality and empower members of all communities to better engage with geothermal data resources by providing universal access to data and by improving the connections between data providers, subject matter experts, and the communities of people using GDR data. This paper will explore some of the recent enhancements made to the GDR to improve data discoverability, reduce submission time, and result in better quality data submissions. These improvements include the ability for users to save a list of their favorite datasets, search for insight into geothermal datasets or data availability, or sign up to receive notifications of future updates to specific datasets. These improvements aim to enhance the overall user experience of the GDR while further connecting communities to the data they need to inform decisions, advance geothermal research, and develop innovative solutions to local energy problems.

access↗
DOE OSTI2022

Data from: “Enabling FAIR data in Earth and environmental science with community-centric (meta)data reporting formats”

This dataset contains supplementary information for a manuscript describing the ESS-DIVE (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) data repository's community data and metadata reporting formats. The purpose of creating the ESS-DIVE reporting formats was to provide guidelines for formatting some of the diverse data types that can be found in the ESS-DIVE repository. The 6 teams of community partners who developed the reporting formats included scientists and engineers from across the Department of Energy National Lab network. Additionally, during the development process, 247 individuals representing 128 institutions provided input on the formats. The primary files in this dataset are 10 data and metadata crosswalk for ESS-DIVE’s reporting formats (all files ending in _crosswalk.csv). The crosswalks compare elements used in each of the reporting formats to other related standards and data resources (e.g., repositories, datasets, data systems). This dataset also contains additional files recommended by ESS-DIVE’s file-level metadata reporting format. Each data file has an associated dictionary (files ending in _dd.csv) which provide a brief description of each standard or data resource consulted in the data reporting format development process. The flmd.csv file describes each file contained within the dataset.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Bayesian Inversion of Seismic and Electromagnetic Data for Marine Gas Reservoir Characterization Using Multi-chain Markov Chain Monte Carlo Sampling

In this study we developed an efficient Bayesian inversion framework for interpreting marine seismic amplitude versus angle (AVA) and controlled source electromagnetic (CSEM) data for marine reservoir characterization. The framework uses a multi-chain Markov-chain Monte Carlo (MCMC) sampler, which is a hybrid of DiffeRential Evolution Adaptive Metropolis (DREAM) and Adaptive Metropolis (AM) samplers. The inversion framework is tested by estimating reservoir-fluid saturations and porosity based on marine seismic and CSEM data. The multi-chain MCMC is scalable in terms of the number of chains, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. As a demonstration, the approach is used to efficiently and accurately estimate the porosity and saturations in a representative layered synthetic reservoir. The results indicate that the seismic AVA and CSEM joint inversion provides better estimation of reservoir saturations than the seismic AVA-only inversion, especially for the parameters in deep layers. The performance of the inversion approach for various levels of noise in observational data was evaluated – reasonable estimates can be obtained with noise levels up to 25%. Sampling efficiency due to the use of multiple chains was also checked and was found to have almost linear scalability.

Ren, Huiying↗
DOE OSTI2019

Big Federal Data Centers Implementing FAIR Data Principles: ARM Data Center Example

Atmospheric Radiation Measurement (ARM) is a multi-laboratory/multi-institutional, US Department of Energy Office of Science National User Facility. ARM’s data is currently hosted at the ARM Data Center (ADC) in Oak Ridge, Tennessee. The ADC holds more than 12,000 data products, with a total holding of more than 1.8 PB of data that dates back to 1992. This includes data from instruments, value-added products, model outputs, field campaigns, and principle investigator contributed data. In this paper, we discuss how big federal scientific data centers, such as ARM, that use modern and scalable architecture apply findable, accessible, interoperable, and reusable (FAIR) data principles to improve overall efficiency. These principles mainly emphasize machine-to-machine interactions that are directly applicable to ARM because of its data volume.

Devarakonda, Ranjeet↗
DOE OSTI2022

Data for: Patel et al. Carbon flux estimates are sensitive to data source: A comparison of field and lab temperature sensitivity data

This dataset contains data and code used for the paper "Carbon flux estimates are sensitive to data source: A comparison of field and lab temperature sensitivity data" [DOI COMING SOON]A large literature exists on mechanisms driving soil production of the greenhouse gases CO2 and CH4. Measurements of these gases’ fluxes are often performed using closed-chamber incubations in the laboratory or in situ, i.e., in the field. Although it is common knowledge that measurements obtained through field studies vs. laboratory incubations can diverge because of the vastly different conditions of these environments, few studies have systematically examined these patterns. It is crucial to understand the magnitude and reasons for any differences, as these data are used to parametrize and benchmark ecosystem- to global-scale models, which are then susceptible to the biases of the source data. Here, we specifically examine how greenhouse gas measurements may be influenced by whether the measurement/incubation was conducted in the field vs. laboratory, focusing on CO2 and CH4 measurements. We use Q10 of greenhouse gas flux (temperature sensitivity) for our analyses, because of the ubiquity of this metric in biological and Earth system sciences and its importance to many modeling frameworks. We predicted that laboratory measurements would be less variable, but also less representative of true field conditions. However, there was greater variability in the Q10 values calculated from lab-based measurements of CO2 fluxes, because lab experiments explore extremes rarely seen in situ, and reflect the physical and chemical disturbances occurring during sampling, transport, and incubation. Overall, respiration Q10 values were significantly greater in laboratory incubations (mean = 4.19) than field measurements (mean = 3.05), with strong influences of incubation temperature and climate region/biome. However, this was in part because field measurements typically represent total respiration (Rs), whereas lab incubations typically represent heterotrophic respiration (Rh), making direct comparisons difficult to interpret. Focusing only on Rh-derived Q10, these values showed almost identical distributions across laboratory (n = 1110) and field (n = 581) experiments, providing strong support for using the former as an experimental proxy for the latter, although we caution that geographic biases in the extant data make this conclusion tentative. Due to a smaller sample size of CH4 Q10 data, we were unable to perform a comparable robust analysis, but we expect similar interactions with soil temperature, moisture, and environmental/climatic variables. Our results here suggest the need for more concerted efforts to document and standardize these data, including sample and site metadata. This dataset contains a compressed (.zip) archive of the data and R scripts used for this manuscript. The dataset includes files in .csv format, which can be accessed and processed using MS Excel or R. This archive can also be accessed on GitHub at https://github.com/kaizadp/field_lab_q10 (DOI: 10.5281/zenodo.7106554).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Data-Conforming Data-Driven Control: Avoiding Premature Generalizations Beyond Data

Data-driven and adaptive control approaches face the problem of introducing sudden distributional shifts beyond the distribution of data encountered during learning. Therefore, they are prone to invalidating the very assumptions used in their own construction. This is due to the linearity of the underlying system, inherently assumed and formulated in most data-driven control approaches, which may falsely generalize the behavior of the system beyond the behavior experienced in the data. This article seeks to mitigate these problems by enforcing consistency of the newly designed closed-loop systems with data and slowing down any distributional shifts in the joint state-input space. This is achieved through incorporating affine regularization terms and linear matrix inequality constraints to data-driven approaches, resulting in convex semi-definite programs that can be efficiently solved by standard software packages. We discuss the optimality conditions of these programs and then conclude this article with a numerical example that further highlights the problem of premature generalization beyond data and shows the effectiveness of our proposed approaches in enhancing the safety of data-driven control methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Respiration data, microbial community assembly data, and FTICR-MS data associated with: “Disturbance Triggers Non-Linear Microbe-Environment Feedbacks. Sengupta et al., 2021, Biogeosciences”

This data package is associated with the manuscript “Disturbance Triggers Non-Linear Microbe-Environment Feedbacks, in revision in Biogeosciences (Sengupta et al. and 2021;https://bg.copernicus.org/preprints/bg-2021-51/). The study used hyporheic zone sediments as a model system to provide an integrated view of how disturbance modulates linkages among microbial ecology, biogeochemistry, and organic matter thermodynamics. Laboratory experiments exposed hyporheic sediment to varying wetting/drying dynamics. Data types include dissolved oxygen rates used to derive respiration rates, Fourier-transform ion cyclotron resonance mass spectrometry (FTICR-MS) data used to derive thermodynamic properties of organic matter, and microbial community assembly metrics derived from amplicon-sequence data of putatively active (cDNA) and whole community (gDNA). The outcomes of the study are condensed into a broadly applicable conceptual model linking external forcing, internal dynamics, and history. This data package is comprised of a file-level metadata (FLMD) csv, metadata csv, and seven folders that contain csv files, R scripts, xml files, and associated documentation: (1) Rates, (2) bNTI, and (3) FTICR, (4) Statistics_Analyses, (5) Raw OTU Beta dispersion Analysis, (6) bMNTD Randomizations, and (7) Data Dictionaries. The FLMD file has a description of each file included in the data package.

54 ENVIRONMENTAL SCIENCES↗