Search NASA⌕ Search

SEARCH · Search NASA

Results for “resource data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Proxy-based Bayesian inversion of strain tensor data measured during well tests

Recent instrument developments have made it possible to measure the strain tensor caused by injecting or pumping fluid from aquifers or reservoirs, but the full value of these data is limited because the long runtimes of poroelastic forward models makes it impractical to use many inversion schemes. This limits the interpretation of strain data for managing the recovery of resources or storage of wastes in the subsurface. This paper describes a method of inverting deformation data using a poroelastic numerical simulator so the results can be used to manage reservoirs or aquifers. We developed a workflow designed to reduce the number of simulations sufficiently to make it feasible to use DREAMzs, an advanced Bayesian inversion method that translates the uncertainties from different sources into unbiased posterior parameter distributions and uncertainty envelopes around the field data. Using a KNN proxy model for the poroelastic simulator is key to reducing the overall computations, and the workflow includes a strategy for ensuring the proxy model results converge on the results from the simulator. The workflow is tested using an idealized example that verifies the ability to correctly identify parameters and characterize noise used to perturb the data. Field data from an injection test at an oil reservoir near Tulsa, Oklahoma, are also used to evaluate the efficacy of the workflow with a real dataset. The workflow identified 265 history matching solutions out of 1240 total simulation runs (21% acceptance ratio), where the results were used to characterize posterior parameter distribution and evaluate the prediction uncertainty. Furthermore, this workflow is significant because it enables strain tensor, or other geomechanical measurements to be interpreted to guide decision-making during energy and environmental processes in the subsurface.

42 ENGINEERING↗
DOE OSTI2025

A Performance Model of In-Situ Techniques

The computational capacity of High-Performance Computing (HPC) systems increases continuously with the rapid development of central processing units (CPUs) and graphic processing units (GPUs), while the in-/output (IO) subsystem develops relatively slowly and storage capacity is also limited. Data-intensive applications, which are designed to leverage the high computational capacity of HPC resources, typically generate a considerable amount of data for post-processing visualizations and data analytics. The limited IO speed and storage space could lead to constraints in the actual performance of these applications and, therefore, scientific discovery. In-situ techniques, where data is visualized/analysed while still in memory rather than through disk, can contribute to alleviating these problems as they can reduce or even fully avoid data writing/reading through the IO subsystem to/from storage. However, the overall efficiency of insitu techniques crucially depends on the characteristics of both the in-situ tasks and the applications, and the resource distribution among them. Therefore, choosing the right in-situ approach (synchronous, asynchronous, or hybrid) and resource allocation is essential to minimize overhead and maximize the benefits of concurrent execution. In this paper, we present a performance model of in-situ techniques to find the most beneficial in-situ approach and the preferred resource configuration. We verify the high accuracy of our approach with over 6800 measurements and provide use cases with different applications.

Ju, Yi [Max Planck Computing and Data Facility, Ga↗
DOE OSTI2024

Diaspora: Resilience-enabling services for science from HPC to edge

Scientific applications of interest to DOE must increasingly engage distributed resources (e.g., instruments, remote computers, data stores, edge devices) and deliver more stringent levels of service (e.g., uninterrupted processing of experiment data streams). In such systems, state is distributed and components can fail in many ways, often silently, making application resilience a major concern. Addressing the resilience needs of such applications requires methods for gaining knowledge of resources and applications and for translating that knowledge into action. We are working on addressing these needs in the context of multi-messenger astronomy, where detecting and responding to unusual transient events in multiple cosmic messengers (gravitational wave, electromagnetic, high- energy particles) from different instruments leads to a federated learning problem.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Moving small files in a networked environment

Globally distributed computing infrastructures, such as clouds and supercomputers, are currently used to manage data that is generated with an unprecedented speed from a variety of resources. Coping with this trend, the volume of data exchanged across distant sites increases substantially. To accelerate data transfer, high-speed networks are provided to connect remote sites. Most existing data movement solutions are optimized for moving large files. However, it is still challenging to transfer a large number of small files across networks. This disadvantage not only lowers data transfer performance, but also decreases overall system utilization. Here, we identify that moving small files is mainly constrained by degraded file system throughput, not just network performance as might be suspected. We have built a data transfer pipeline model to analyze the impact of small network I/O and storage I/O on data movement. Extending one of the widely used open source data movement solutions, GridFTP, we demonstrate several appropriate engineering approaches that mitigate the bottleneck and increase data transfer efficiency. We show optimizations that improve data transfer performance more than 5 times. In comparison to existing solutions, our approaches can save a significant amount of system resources for moving lots of small files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Dynamical simulation via quantum machine learning with provable generalization

Much attention has been paid to dynamical simulation and quantum machine learning (QML) independently as applications for quantum advantage, while the possibility of using QML to enhance dynamical simulations has not been thoroughly investigated. Here we develop a framework for using QML methods to simulate quantum dynamics on near-term quantum hardware. We use generalization bounds, which bound the error a machine learning model makes on unseen data, to rigorously analyze the training data requirements of an algorithm within this framework. Our algorithm is thus resource efficient in terms of qubit and data requirements. Furthermore, our preliminary numerics for the XY model exhibit efficient scaling with problem size, and we simulate 20 times longer than Trotterization on IBMQ-Bogota. Published by the American Physical Society 2024

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speedup jobs completion, better utilize the available hardware and visibly reduce costs, especially when renting computers on the cloud. This was demonstrated in earlier studies on HEPCloud. But the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. With GlideinBenchmark and the help of the GldieinWMS Factory it controls the benchmark execution. Finally, a scheduler like HEPCloud’s Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco↗
DOE OSTI2021

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. iDDS has been designed to intelligently orchestrate workflow and data management systems, decoupling data pre-processing, delivery, and main processing in various workflows. It is an experiment-agnostic service around a workflow-oriented structure to work with existing and emerging use cases in ATLAS and other experiments. Here we will present the motivation for iDDS, its design schema and architecture, use cases and current status, and plans for the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflows and data management systems, decoupling data pre-processing, delivery, and primary processing in large scale workflows. It is an experiment-agnostic service that has been deployed to serve data carousel (orchestrating efficient processing of tape-resident data), machine learning hyperparameter optimization, active learning, and other complex multi-stage workflows defined via DAG (Directed Acyclic Graph), CWL (Common Workflow Language) and other descriptions, including a growing number of analysis workflows. We will at first introduce some deployed use cases in a summary. Then we will focus on new improvements and use cases under developments in ATLAS, Rubin Observatory and sPHENIX, together with future efforts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

WHOLESCALE: Seismic Survey Data from San Emidio Nevada 2021

This dataset includes raw and processed seismic data from the 2021 seismic survey at the San Emidio geothermal field in Nevada. In April and May 2021, 37 tri-axial short period seismographs were deployed in a 1.8km diameter cluster centered on 40.367278 N, 119.409019 W. The first data record started at 2021-04-06T07:09:10Z UTC and the last record ended at 2021-05-11T02:58:52Z UTC. The 37 stations included 29 SmartSolo IGU-16HR 3C all-in-one 5 Hz seismographs and 8 DataCube seismographs with 4.5 Hz HGS HG-6(B coil) tri-axial geophones. The raw format (level 0) data includes 353 GB of 500 sps SmartSolo data in native DLD format, 113 GB of 400 sps DataCube data in native DataCube format, and 3.4 GB of GPS data collected during the RTK GPS survey. The SAC data (level 1) files include 564 GB of hourly SAC files. The experiment was designed to monitor seismic activity before, during, and after the planned three-day plant maintenance shutdown April 19-21, 2021. The pumping stopped at 2021/04/19 12:51:45 UTC and resumed about 2021/04/21 21:00:00 UTC. The dataset is hosted in an AWS data lake, with an associated GDR data set providing the metadata. Links to both of these resources are included below. Additionally, this collection features data and metadata from a 2016 seismic survey at the same site for comparative analysis.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Predicting Kyasanur forest disease in resource-limited settings using event-based surveillance and transfer learning

In recent years, the reports of Kyasanur forest disease (KFD) breaking endemic barriers by spreading to new regions and crossing state boundaries is alarming. Effective disease surveillance and reporting systems are lacking for this emerging zoonosis, hence hindering control and prevention efforts. We compared time-series models using weather data with and without Event-Based Surveillance (EBS) information, i.e., news media reports and internet search trends, to predict monthly KFD cases in humans. We fitted Extreme Gradient Boosting (XGB) and Long Short-Term Memory models at the national and regional levels. We utilized the rich epidemiological data from endemic regions by applying Transfer Learning (TL) techniques to predict KFD cases in new outbreak regions where disease surveillance information was scarce. Overall, the inclusion of EBS data, in addition to the weather data, substantially increased the prediction performance across all models. The XGB method produced the best predictions at the national and regional levels. The TL techniques outperformed baseline models in predicting KFD in new outbreak regions. Novel sources of data and advanced machine-learning approaches, e.g., EBS and TL, show great potential towards increasing disease prediction capabilities in data-scarce scenarios and/or resource-limited settings, for better-informed decisions in the face of emerging zoonotic threats.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Carbon Transport and Storage Planning and Viability Support Tools

The EDX disCO2ver Carbon Transport and Storage Planning and Viability Support Tools are made up of the Carbon Storage Planning Inquiry Tool (CS PlanIT, Justman et al. 2024) and the Carbon Storage Technical Viability Approach Support Tool (CS TVA). Together, these tools support data access to support understanding data availability to support planning efforts for carbon transport and storage. The Carbon Storage Planning Inquiry Tool (CS PlanIT) is an online web mapping application designed to help users explore, query, and evaluate multiple data layers to support and accelerate carbon storage resource and feasibility assessments and planning efforts. CS PlanIT currently contains a range of datasets associated with geologic, technical, and infrastructure factors. The data sets can be filtered geographically for an area of interest to update statistics and charts within the dashboard. The dashboard is divided into different sections called widgets, relating to different steps in the carbon storage planning process. The resources in this submission include a link to PlanIT, as well as a data catalog and link to user documentation. The original citation for the CS PlanIT tool, which has now been integrated into the toolset here, was: - Devin Justman, Scott Pantaleone, Maneesh Sharma, Lucy Romeo, Paige Morkner, CS PlanIT (Carbon Storage Planning Inquiry Tool) , 6/28/2024, https://edx.netl.doe.gov/dataset/cs-planit-carbon-storage-planning-inquiry-tool, DOI: 10.18141/2377953 The Carbon Storage Technical Viability Approach Support (CS TVA) Tool displays spatial data availability for the many components of Geologic Carbon Storage (GCS) technical viability assessment (Creason al 2025). Identifying sites suitable for GCS requires evaluating the intersection of myriad factors, including reservoir conditions, subsurface and surface hazards, infrastructure requirements, and energy community metrics. The technical viability of a site can only be confirmed for instances where all these factors have data available, and where those data support viability. Additional Resources related to the Technical Viability Assessment Tool: - Julia Mulhern, Casey White, Araceli Lara, Neyda Cordero Rodriguez, Zachary Jackson, Jacob Shay, Gabriel Creason, MacKenzie Mark-Moser, Paige Morkner, Kelly Rose, Carbon Storage Technical Viability Approach (CS TVA) Database, 3/26/2025, https://edx.netl.doe.gov/dataset/edx4ccs-carbon-storage-technical-viability-approach-database , DOI:10.18141/1984655 - Julia Mulhern, MacKenzie Mark-Moser, Gabriel Creason, Casey White, Araceli Lara, Neyda Cordero Rodriguez, Zach Jackson, Paige Morkner, Kelly Rose, Carbon Storage Technical Viability Approach (CS TVA) Matrix, 3/27/2025, https://edx.netl.doe.gov/dataset/carbon-storage-technical-viability-approach-cs-tva-matrix , DOI: 10.18141/2539979 - Gabriel Creason, Zach Jackson, Neyda Cordero Rodriguez, Julia Mulhern, Casey White, Araceli Lara, MacKenzie Mark-Moser, Paige Morkner, Kelly Rose, Carbon Storage Technical Viability Approach (CS TVA) Data Availability Results Database, 3/27/2025, https://edx.netl.doe.gov/dataset/carbon-storage-technical-viability-approach-cs-tva-data-availability-results-database, DOI:10.18141/2538557

Carbon storage↗
DOE OSTI2021

An Open-Source Virtual Testbed for a Real Net-Zero Energy Community

Net zero energy communities (NZECs) are critical to ensure sustainability and resilience of modernized power systems. System modeling helps overcome technical challenges in designing and operating NZECs. In this paper, we present the modeling work based on a real NZEC. Two sets of models are developed: higher-fidelity physics-based models considering the interaction between subsystems of the studied NZEC and capturing fast-dynamics; and lower-fidelity data-driven models requiring less resource to establish and/or run. All models are validated against measurements from this real NZEC. In addition, we create a simulation framework which streamlines the processes for simulation and thus allows using developed models to form a virtual testbed. To demonstrate the usage of the virtual testbed, a case study is conducted where a building-to-grid integration control is evaluated via simulation. The evaluation results suggest that the tested control significantly smooths the power draw of the studied community and doesn’t sacrifice the thermal comfort to a great extent.

Huang, Sen↗
DOE OSTI2022

Offshore reanalysis wind speed assessment across the wind turbine rotor layer off the United States Pacific coast

Abstract. The California Pacific coast is characterized by considerable wind resource and areas of dense population, propelling interest in offshore wind energy as the United States moves toward a sustainable and decarbonized energy future. Reanalysis models continue to serve the wind energy community in a multitude of ways, and the need for validation in locations where observations have been historically limited, such as offshore environments, is strong. The U.S. Department of Energy (DOE) owns two lidar buoys that collect wind speed observations across the wind turbine rotor layer along with meteorological and oceanographic data near the surface to characterize the wind resource. Lidar buoy data collected from recent deployments off the northern California coast near Humboldt County and the central California coast near Morro Bay allow for validation of commonly used reanalysis products. In this article, wind speeds from the Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2), the Climate Forecast System version 2 (CFSv2), the North American Regional Reanalysis (NARR), the European Centre for Medium-Range Weather Forecasts Reanalysis version 5 (ERA5), and the analysis system of the Rapid Refresh (RAP) are validated at heights within the wind turbine rotor layer ranging from 50 to 100 m. The validation results offer guidance on the performance and uncertainty associated with utilizing reanalyses for offshore wind resource characterization, providing the offshore wind energy community with information on the conditions that lead to reanalysis error. At both California coast locations, the reanalyses tend to underestimate the observed rotor-level wind resource. Occasions of large reanalysis error occur in conjunction with stable atmospheric conditions, wind speeds associated with peak turbine power production (> 10 m s−1), and mischaracterization of the diurnal wind speed cycle in summer months.

17 WIND ENERGY↗
DOE OSTI2024

Hyper‐Local Temperature Prediction Using Detailed Urban Climate Informatics

The accurate modeling of urban microclimate is a challenging task given the high surface heterogeneity of urban land cover and the vertical structure of street morphology. Recent years have witnessed significant efforts in numerical modeling and data collection of the urban environment. Nonetheless, it is difficult for the physical‐based models to fully utilize the high‐resolution data under the constraints of computing resources. The advancement in machine learning (ML) techniques offers the computational strength to handle the massive volume of data. In this study, we proposed a modeling framework that uses ML approach to estimate point‐scale street‐level air temperature from the urban‐resolving meso‐scale climate model and a suite of hyper‐resolution urban geospatial data sets, including three‐dimensional urban morphology, parcel‐level land use inventory, and weather observations from a sensor network. We implemented this approach in the City of Chicago as a case study to demonstrate the capability of the framework. The proposed approach vastly improves the resolution of temperature predictions in cities, which will help the city with walkability, drivability, and heat‐related behavioral studies. Moreover, we tested the model's reliability on out‐of‐sample locations to investigate the modeling uncertainties and the application potentials to the other areas. This study aims to gain insights into next‐gen urban climate modeling and guide the observation efforts in cities to build the strength for the holistic understanding of urban microclimate dynamics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

EMT data generation

The integration of inverter-based resources (IBRs) in power systems is accelerating, bringing with it significant benefits such as reduced greenhouse gas emissions, improved grid resilience, and increased energy independence. Despite these advantages, the widespread adoption of IBRs introduces several challenges, including issues related to grid stability, increased operational complexity, and the need for updated regulatory frameworks. To address these challenges, IEEE released Standard 2800 in 2022, which sets forth the necessary interconnection capabilities and performance criteria for IBRs connected to transmission and sub-transmission systems. This standard outlines the performance requirements to ensure the reliable integration of IBRs into the bulk power system. Furthermore, in 2023, the North American Electric Reliability Corporation (NERC) published a reliability guideline for electromagnetic transient (EMT) modeling of BPS-connected IBRs. This guideline provides recommendations for developing EMT model requirements, performing model quality checks, and implementing verification practices specifically for EMT models representing BPS-connected inverter-based resources in reliability studies conducted by transmission planners and planning coordinators. These standards and guidelines have a profound impact on EMT studies for transmission networks, influencing system stability analyses, grid recovery and resynchronization processes, fault ride-through evaluations, protection and coordination strategies, advanced control methodologies, and the inclusion of IBRs in transient models of transmission networks. As a result, the generation of EMT data is crucial for conducting various transient-based studies to understand the impact of IBRs. EMT data generation use cases serve as the basis for scenarios in event detection and identification use cases, providing comprehensive details about EMT data generation for transmission grids with inverter-based resources. These use cases supply sufficient training and validation datasets for subsequent EMT analysis algorithms.

Xia, Qianxue↗
DOE OSTI2020

CO₂ Storage prospeCtive Resource Estimation Excel aNalysis (CO₂-SCREEN) User’s Manual

This user’s manual guides the use of the National Energy Technology Laboratory’s (NETL) CO₂ Storage prospeCtive Resource Estimation Excel aNalysis (CO₂-SCREEN) tool, which was developed to aid users screening geologic formations for prospective CO₂ storage resources. This manual is specific to the CO₂-SCREEN 4.0 version which is based in Python. CO₂-SCREEN applies U.S. Department of Energy (DOE) methods and equations for estimating prospective CO₂ storage resources for saline formations, shale formations, and residual oil zones (ROZ). CO₂-SCREEN was developed to be substantive and user-friendly and provide a consistent method for calculating prospective CO₂ storage resources. CO₂-SCREEN uses a Java based graphical user interface for data inputs and uses Python to calculate prospective CO₂ storage resources.

54 ENVIRONMENTAL SCIENCES↗