Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE's) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and helping users access data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data. This paper provides an update on recent improvements made to the GDR's data lakes and automated data pipelines, including: (1) streamlining the data lake intake process, (2) better educating users on the process and requirements through a new data lakes page, (3) adding data lake direct access links to GDR data lake submission pages, (4) implementing a DAS data pipeline to convert DAS data uploaded in SEG-Y format to a standardized hierarchical data format v5 (HDF5), (5) extending this pipeline to encompass data in the GDR data lake, (6) adding metadata requirements for geospatial data, (7) making user interface/user experience (UX) enhancements to the data pipelines' documentation pages, and (8) improving the GDR's data standards and pipelines pages to better guide users in ensuring that their data is standardized by the GDR's automated data pipelines. 2024 Geothermal Resources Council. All rights reserved.

accessibility↗
DOE OSTI2024

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

accessibility↗
DOE OSTI2026

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

The Atacama Cosmology Telescope: Summary of DR4 and DR5 Data Products and Data Access

Two recent large data releases for the Atacama Cosmology Telescope (ACT), called DR4 and DR5, are available for public access. These data include temperature and polarization maps that cover nearly half the sky at arcminute resolution in three frequency bands; lensing maps and component-separated maps covering ~ 2,100 deg 2 of sky; derived power spectra and cosmological likelihoods; a catalog of over 4,000 galaxy clusters; and supporting ancillary products including beam functions and masks. The data and products are described in a suite of ACT papers; here we provide a summary. In order to facilitate ease of access to these data we present a set of Jupyter IPython notebooks developed to introduce users to DR4, DR5, and the tools needed to analyze these data. The data products (excluding simulations) and the set of notebooks are publicly available on the NASA Legacy Archive for Microwave Background Data Analysis (LAMBDA); simulation products are available on the National Energy Research Scientific Computing Center (NERSC).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Open Energy Data Initiative (OpenEDI) - Open Data Access Tools (OEDI-ODAT) [SWR 20-57]

Open Energy Data Initiative (OpenEDI): Advancing Analytics and Research Innovation through Improved Data Access. The Open Energy Data Initiative (OEDI) aims to improve and automate access of high-value energy data sets across the U.S. Department of Energy’s (DOE’s) programs, offices, and national laboratories. Sponsored by the DOE, this platform is being implemented by the National Renewable Energy Laboratory (NREL) to make data actionable and discoverable by researchers and industry to accelerate analysis and advance innovation. Partners at other national laboratories will help utilize the platform for analysis, can be directly involved in determining requirements, and will be resources for providing new datasets that can be shared with the public to expand innovation.

Rager, David↗
DOE OSTI2022

A Review of the Use of Wearables in Indoor Environmental Quality Studies and an Evaluation of Data Accessibility from a Wearable Device

An understanding of indoor environmental quality (IEQ) and its effects on occupant well-being can inform building system design and operation. The use of wearables in field studies to collect subjective and objective health performance indicators (HPIs) from a large number of occupants could deliver important improvements in IEQ. To facilitate the use of wearables in IEQ studies, there is a need to identify which HPIs should be collected and to evaluate data accessibility from these devices. To address this issue, a literature review of previous IEQ studies was conducted to identify relationships between different IEQ factors and HPIs, with a focus on HPIs that were collected using wearables. A preliminary assessment of data accessibility from a selected wearable device (Fitbit Versa 2) was performed and documented. The review suggested the need to further investigate and collect sleep quality parameters, heart rate, stress response, as well as subjective ratings of comfort using wearables. The data accessibility assessment revealed issues related to missing data points and data resolution from the examined device. A set of recommendations is outlined to inform future studies.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Researcher Views on Water Data Accessibility, Usability, and Dissemination

Ultimately, the challenges that researchers face with finding, accessing, using, and disseminating water data affect their ability to do mission critical research for DOE and other sponsors. We interviewed PNNL researchers who support DOE’s Water Power Technologies Office and summarize their views on issues pertaining to data access, use, and dissemination. We also take a closer look at approaches to disseminating data to help inform WPTO and lab researcher decisions about incorporating data dissemination into future projects.

13 HYDRO ENERGY↗
DOE OSTI2020

Opportunities for an Integrated Web-Based Workbench for Data Access and Analysis - 20244

Management of environmental issues can require integration of multiple types of data and information, conducting data analysis and interpretation, and providing data visualization for effective communications. These data elements are important for site management to support regulator interactions and provide defensibility for remedial decisions. Databases and information repositories are core elements of managing data; however, efficient data access and analysis also enable effective site management. The U.S. Department of Energy (DOE) Hanford Site is an example of a complex site with a voluminous quantity of environmental data and a need for efficient site management. Different tiers of data and information tools have been developed and deployed to address site needs. These tools are configured for ready access via the web site interfaces and meet the rigorous quality requirements for environmental site management. Evolving efforts are focused on an integrated platform to meet site environmental management needs. In this platform, users can access site information at multiple levels of detail based on their need and permissions, so that data and associated analyses are presented within the context of the site mission and the user's management or technical needs. This concept is not only applicable at individual sites like Hanford but also applicable at other sites within the DOE complex. An integrated web-based architecture that links data visualization, data analytics, and management tools can provide holistic access to large data sets, minimize complexity, and maximize interactivity and technical communication. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Analysis of GPU Data Access Patterns on Complex Geometries for the D3Q19 Lattice Boltzmann Algorithm

GPU performance of the lattice Boltzmann method (LBM) depends heavily on memory access patterns. When implemented with GPUs on complex domains, typically, geometric data is accessed indirectly and lattice data is accessed lexicographically. Although there are a variety of other options, no study has examined the relative efficacy between them. Here, we examine a suite of memory access schemes via empirical testing and performance modeling. We find strong evidence that semi-direct is often better suited than the more common indirect addressing, providing increased computational speed and reducing memory consumption. For the layout, we find that the Collected Structure of Arrays (CSoA) and bundling layouts outperform the common Structure of Array layout; on V100 and P100 devices, CSoA consistently outperforms bundling, however the relationship is more complicated on K40 devices. When compared to state-of-the-art practices, our recommendations lead to speedups of 10–40 percent and reduce memory consumption up to 17 percent. Using performance modeling and computational experimentation, we determine the mechanisms behind the accelerations. We demonstrate that our results hold across multiple GPUs on two leadership class systems, and present the first near-optimal strong results for LBM with arterial geometries run on GPUs.

42 ENGINEERING↗
DOE OSTI2023

Azure Data Tools (FKA: OEDI (Open Energy Data Initiative) Data Access Tools) [SWR-23-92]

The Open Energy Data Initiative (OEDI) provides a number of tools to enable the use of the open data published through this initiative. The source is largely written in Python, including Jupyter notebooks. The Open Energy Data Initiative (OEDI) is a partnership between the National Renewable Energy Laboratory (NREL), the U.S. Department of Energy (DOE), Amazon, Microsoft, and Google to provide universal access to big data in the cloud. At the heart of OEDI is a centralized repository of high-value energy research datasets aggregated from DOE Program Offices, National Laboratories and other collaborators.

Jonathan, Weers↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2022

Next generation experimental data access at NSLS-II

The NSLS-II network and computing infrastructure has been significantly updated recently. The re-IP process in 2020-2021 enabled the NSLS-II network to be routable to the rest of the BNL campus. Then, standardization of the operating systems and deployment procedures helped to deliver a consistent environment to workstations and servers used by all NSLS-II beamlines. In particular, the RedHat Enterprise Linux 8 was deployed to 700+ machines using the RedHat Satellite infrastructure management product, and all critical services (IOCs, databases, etc.) were migrated to the new OS. NFS users’ home directories are consistent across all of the machines, which eliminates the need for the individual configuration of the user environment on each host. The standard suite of software packages is available to the beamline staff and users, which includes the system packages (deployed via RPM) as well as the conda environments for data acquisition and analysis. Security measures were implemented to comply with the industry standards, which include multi-factor authentication (using Duo), secure screen lock for the beamline machines, and advanced access control to the experimental data that is stored in shared central storage available on all hosts. These major enhancements facilitated sharing the experimental data (currently for a number of selected beamlines, with a plan to extend it to the whole facility in the nearest future) with the users via an externally facing JupyterHub instance. The beamlines keep using the Bluesky data acquisition framework to orchestrate their experiments, and the new infrastructure enabled them to use a next-generation data access library called tiled.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Efforts and Innovations to Promote Data Sharing and Data Accessibility in the EGS Collab Experiments

Large-scale scientific research programs such as the EGS Collab experiments and the FORGE program play extremely important roles in advancing geothermal technologies. Such efforts involve a large number of researchers from multiple institutions, last multiple years, and generate large, complex datasets. The value of the research efforts is only realized when the datasets are used by a large community of researchers in the decades to come. A challenge is that due to the complexity of the data, it could require a user to devote a serious effort into understanding the data before the data can be effectively utilized. In the EGS Collab experiments, the team has devoted remarkable efforts and developed many innovative solutions to make the data more accessible to broader team members and future users. This paper documents the experience gained and lessons learned by the EGS Collab team in disseminating the data in the most informative and inspiring forms to maximize the value of this precious dataset. "Accessibility"in the title does not only mean making the raw data available for download. Particularly, we want to emphasize the importance of organizing, annotating, and presenting the data in ways to make it easy to digest by prospective consumers of the data.

circulation↗
DOE OSTI2023

Improved quality metrics for association and reproducibility in chromatin accessibility data using mutual information

Correlation metrics are widely utilized in genomics analysis and often implemented with little regard to assumptions of normality, homoscedasticity, and independence of values. This is especially true when comparing values between replicated sequencing experiments that probe chromatin accessibility, such as assays for transposase-accessible chromatin via sequencing (ATAC-seq). Such data can possess several regions across the human genome with little to no sequencing depth and are thus non-normal with a large portion of zero values. Despite distributed use in the epigenomics field, few studies have evaluated and benchmarked how correlation and association statistics behave across ATAC-seq experiments with known differences or the effects of removing specific outliers from the data. Here, we developed a computational simulation of ATAC-seq data to elucidate the behavior of correlation statistics and to compare their accuracy under set conditions of reproducibility. Using these simulations, we monitored the behavior of several correlation statistics, including the Pearson’s R and Spearman’s ρ coefficients as well as Kendall’s τ and Top–Down correlation. We also test the behavior of association measures, including the coefficient of determination R 2 , Kendall’s W, and normalized mutual information. Our experiments reveal an insensitivity of most statistics, including Spear man’s ρ, Kendall’s τ, and Kendall’s W, to increasing differences between simulated ATAC-seq replicates. The removal of co-zeros (regions lacking mapped sequenced reads) between simulated experiments greatly improves the estimates of correlation and association. After removing co-zeros, the R 2 coefficient and normalized mutual information display the best performance, having a closer one-to-one relationship with the known portion of shared, enhanced loci between simulated replicates. When comparing values between experimental ATAC-seq data using a random forest model, mutual information best predicts ATAC-seq replicate relationships. Collectively, this study demonstrates how measures of correlation and association can behave in epigenomics experiments. We provide improved strategies for quantifying relationships in these increasingly prevalent and important chromatin accessibility assays.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Technical Track on Biomass Carbon Removal and Storage (BiCRS): Mapping bioresources, phase 1 - Consistency check comparing Mission Innovation’s Data Visualization Tool for Bioresources and the Clean Energy Ministerial Biofuture Initiative Global Biomass data accessible via the US Department of Energy’s Bioenergy Knowledge Discovery Framework (KDF)

The Mission Innovation (MI) Carbon Dioxide Removal (CDR) Mission, Technical Track on Biomass Carbon Dioxide Removal and Storage (BiCRS), has produced a biomass resource database for its members. In parallel, Oak Ridge National Laboratory (ORNL) developed the International Feedstock Reporting data portal—herein referred to as the CEM Biofuture-KDF data—on behalf of the Clean Energy Ministerial Biofuture Initiative (CEM Biofuture), as a specific task under Biofuture’s 2024–25 Action Plan. This work was conducted at the request of CEM Biofuture and funded by the U.S. Department of Energy in support of that initiative, and it is hosted within DOE’s Knowledge Discovery Framework (KDF).

09 BIOMASS FUELS↗
DOE OSTI2024

HPC and Cloud Convergence Beyond Technical Boundaries: Strategies for Economic Sustainability, Standardization, and Data Accessibility

At the IEEE/ACM International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC23), held in Denver, experts discussed the convergence of high-performance computing and cloud computing. Experts explored how this integration could address current scientific computing limitations, enhance computational capabilities, and foster global collaboration while focusing on economic, security, technical, and community challenges and opportunities.

97 MATHEMATICS AND COMPUTING↗