Search NASA⌕ Search

SEARCH · Search NASA

Results for “resource data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Hybrid Data-Driven and Model-Based Anomaly Detection Scheme for DER Operation: Preprint

This paper proposes a hybrid data and model-based anomaly detection for securing the operation of distributed energy resources (DERs) in distribution grids. Data-driven autoencoders (AE) are set up at the edge level by taking local DER data and detect anomalous operations by leveraging the reconstruction ability. In parallel, model-based state estimation (SE) is running at the system level by taking system models and measurements, the anomalies are identified by analyzing the measurements residual. The hybrid scheme preserves the benefits of both data-driven and model-based analysis and thus improves the robustness and accuracy of anomaly detection. It can be established by getting full use of the existing infrastructures in distribution grids. Numerical tests on a realistic distribution feeder in Southern California highlight the effectiveness as well as benefits of the proposed scheme.

anomaly detection↗
DOE OSTI2023

Full-Scale Soft-Sensor Implementations Enable WRRF Hybrid Digital-Twins

Digital twins (DTs) are on the rise in the wastewater treatment industry and, with ever-increasinguse of online data in more complex applications, it is bringing us towards a digital revolution withinwater resource recovery facilities (WRRFs). An integral data requirement is dynamic real-timeinfluent concentrations. However, few WRRFs have the ability to deploy sensors to generate therequired dynamic influent concentrations. This paper presents an innovative approach to generatedynamic influent profiles with a soft sensor primarily using bioreactor airflow rates and DT models.It has been implemented on three different full-scale facilities and demonstrate it is feasible toretrieve 15-minute interval diurnal influent concentrations. This study overcomes a key bottleneckin DT applications - the lack of realistic high-resolution influent profiles - fostering the developmentof DTs. The fact that the data used in this study are commonly available ensures wide applicabilityfor most WRRFs and prepares them for the advanced controls that digital twins can unlock.

Johnson, Bruce↗
DOE OSTI2025

Open data sets for assessing photovoltaic system reliability

Photovoltaic (PV) systems have become a cornerstone of renewable energy strategies, particularly due to the significant reduction in solar power costs over the past decade. However, the long-term reliability of PV installations presents a persistent challenge, requiring the development of advanced monitoring and predictive maintenance strategies. A wide range of data types is used to evaluate the health of PV systems, including environmental conditions, electrical performance, and inspection imagery. These data enable methodologies such as machine learning (ML) models for lifetime prediction and computer vision techniques for defect detection. However, the acquisition of high-quality and comprehensive data is difficult, particularly in terms of long-term consistency and data variety. Publicly available data sets serve as valuable resources for addressing these challenges, but they often suffer from fragmentation and are difficult to access. This paper presents a comprehensive review of existing open-source data sets related to PV degradation, analyzing their features, functionalities, and potential applications. We categorize these data sets based on the specific aspects of PV system information they cover, such as environmental conditions, operational monitoring, image inspection and module materials, and propose relevant tools and ML models for processing them. In addition, we propose practices for future data collection and usage, while also discussing potential directions in data-driven research. Our aim is to enhance data utilization and publication among researchers and industry professionals, promoting a deeper understanding of the role of data in enhancing the performance and durability of PV systems.

14 SOLAR ENERGY↗
DOE OSTI2023

IRIS-DMEM: Efficient Memory Management for Heterogeneous Computing

This paper proposes an efficient data memory management approach for the Intelligent RuntIme System (IRIS) heterogeneous computing framework along with new data transfer policies. IRIS provides a task-based programming model for extreme heterogeneous computing (e.g., CPU, GPU, DSP, FPGA) with support for today's most important programming languages (e.g., OpenMP, OpenCL, CUDA, HIP, OpenACC). However, the IRIS framework either forces the programmer to introduce data transfer commands for each task or relies on suboptimal memory management for automatic and transparent data transfers. The work described here extends IRIS with novel heterogeneous memory handling and introduces novel data transfer policies by employing the Distributed data MEMory handler (DMEM) for efficient and optimal movement of data among the various computing resources. The proposed approach achieves performance gains of up to 7× for tiled LU factorization and tiled DGEMM (i.e., matrix multiplication) benchmarks. Moreover, this approach also reduces data transfers by up to 71% when compared to previous IRIS heterogeneous memory management handlers. This work compares the performance results of the IRIS framework's novel DMEM with the StarPU runtime and MAGMA math library for GPUs. Experiments show a performance gain of up to 1.95× over StarPU and 2.1× over MAGMA.

Miniskar, Narasinga Rao↗
DOE OSTI2022

Developing an Interactive Landscape for Mobility Resources: Preprint

As the world continues to be increasingly driven by data, the ways researchers and professionals sort and collect this data is critical. In the world of mobility data, new levels of data from public transportation systems, location services, and other means are being lost due to how little organization exists. Much of the data is proprietary, and there are few if any de jure or even de facto standards connecting data. There is also little knowledge about the gaps that exist in the data. In this project, we created an interactive landscape where mobility resources are categorized and organized in an easy to use, living document. We made this landscape with open-source code from the CNCF Cloud Native Landscape and repurposed it to the mobility data's needs. Additionally, unlike previous sources that organize mobility data, this document can be updated through GitHub by those in the field to keep its sources relevant. Following the creation of a beta version of the landscape, we conducted several interviews with industry researchers and professionals to ensure the landscape would be useful. The result is an online hub where mobility researchers and resource creators can easily access research and collaborate.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

BioSiting Tool (BioSiting) v2

The BioSiting Tool provides a geospatial interface for analyzing bioeconomy resources and infrastructure across the continental U.S. The tool integrates empirical and modeled data from a broad range of sources. Bioeconomy resources mapped in the tool include agricultural residues, forest residues, municipal solid waste streams, food waste, manure, fats, oils and greases and potential yields of energy crops. Infrastructure mapped in the tool includes biorefineries, material recovery facilities, anaerobic digesters, wastewater treatment plants, combustion plants, district energy systems, crude oil pipelines, petroleum pipelines, natural gas pipelines, railways and freight terminals. Additional data layers include environmental justice indicators at the census tract level and carbon dioxide geologic storage potential. Users can select a location on the map, define a buffer radius in kilometers and generate an inventory of all bioecomony resources within the buffer zone. Data from the tool can be downloaded from individual buffer zones, or at the state or national level.

Huntington, Tyler↗
DOE OSTI2023

Vertical Aerosol Profiling during SAIL (VAPS) Field Campaign Report

To more fully address the science goals of the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility’s Surface Atmosphere Integrated Field Laboratory (SAIL) campaign, particularly the key science question of “How strongly do aerosols affect the surface energy and water balance by altering clouds, precipitation, and surface albedo, and how do these impacts vary seasonally?”, our campaign added instrumentation to planned tethered balloon system (TBS) flights in May and July of 2022. A key aspect of aerosol-cloud interactions is understanding cloud condensation nuclei (CCN) and ice nucleating particle (INP) populations, and how they vary seasonally and spatially. Because the interactions between aerosols and clouds is the target of this work, vertical distributions are particularly important to evaluate. Measurements of aerosol at ground level, below-cloud, and in-cloud will allow a better understanding of the airmasses that are feeding cloud-active particles into low- and mid-level clouds. Measurements of INP and aerosol populations often differ between the ground and aloft, with relationships that likely depend on regional and seasonal. Because of this, collocated measurements of aerosol, CCN, and INP, across different seasons, are crucial to reducing uncertainties in relationships between ground-level and cloud-base aerosol populations. Our campaign added online measurements of CCN using a compact CCN counter and offline filter collections for measurement of INP populations to the SAIL campaign TBS flights in May and July of 2022. The CCN instrument, the CloudPuck, flew and successfully collected data for a total of eight flights across both campaign periods. The IcePuck flew for most flights during the same period. Both instruments have continued to be used for other campaign requests in the SAIL domain. Analysis of this data was proposed through a concurrent DOE Atmospheric System Research (ASR) project submission, which was not funded in 2022. A similar proposal, which would involve analysis of the samples and data collected from this campaign, has been resubmitted in 2023 but a funding decision has not yet been made. If the proposal is again declined, we will seek other resources to ensure that this data can be analyzed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Enhancing Streamflow Reanalysis Across the Conterminous US Leveraging Multiple Gridded Precipitation Data Sets

Streamflow observations, essential for various water resource applications, are often unavailable at critical locations in need. Although different models have been proposed to enhance streamflow predictability at ungauged locations, the challenge extends beyond model fidelity. Differences in meteorologic forcing data sets, precipitation in particular, can significantly affect the accuracy of hydrologic predictions. This challenge intensifies across regions characterized by diverse hydro-climatological and geographical conditions, such as in the conterminous US (CONUS) where a single precipitation product struggles to consistently replicate observed hydrographs, particularly peak flow dynamics. To enhance streamflow predictions, we utilize a VIC-RAPID hydrologic modeling framework driven by multiple commonly used meteorological forcing data sets, such as Daymet, PRISM, ST4, AORC, and their hybrids and create multiple sets of 40-year (1980–2019) hourly, daily, and monthly streamflow reanalysis, Dayflow Version 2, for 2.7 million river reaches across the CONUS. Most forcings lead to skillful streamflow performance, except for ST4 in the mountainous west, where severe radar blockage adversely affects the accuracy. The evaluation using over 6,000 hourly stream gauges shows that hourly AORC and ST4 lead to improved annual peak flow performance over Daymet—driven streamflow (Dayflow V1), particularly in smaller basins, highlighting the value of high temporal resolution forcings in hydrologic predictions. Compared with other benchmark data sets like National Water Model V3.0, AORC-driven VIC-RAPID exhibits improved regional streamflow performance, with comparable peak flow representation. We envision that multi-forcing streamflow reanalysis data can inform regions in need of forcing data enhancement, diagnose hydrologic model performance, and benefit diverse water resource applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Enriching OpenStreetMap network data for transportation applications: Insights into the impact of urban congestion on accessibility

OpenStreetMap (OSM) data is a valuable open-source resource for various transportation, traffic, and planning applications. However, OSM network data lack operating traffic speed information, which is critical for transport planning and operations. Addressing this shortcoming, this study leverages commercial vendor data (to serve as ground truth) with exogenous, open-source variables characterizing local transport infrastructure, land use, and demographic information to predict average congested traffic speeds on OSM networks. Three machine-learning models were tested and estimated for OSM links with and without speed limit information in the Denver metropolitan region. Among these, XGBoost performed best, with mean absolute errors of 3.27 and 3.62 mph for links with and without speed limits, respectively. The developed models accurately predicted traffic speeds for different hours and days of the week compared to ground truth data. Using these predicted speeds, drive accessibility scores were computed for the Denver region for different time periods using the Mobility Energy Productivity (MEP) metric to understand the impact of congestion on energy-efficient accessibility. Results show that congestion-adjusted drive accessibility can be significantly lower compared to accessibility calculated using free flow speeds. Specifically, weekday evening hours saw a 42 % drop in accessibility due to reduced speeds, particularly around downtown Denver. Across the Denver metro region, approximately half as many opportunities and jobs are accessible in under 20 min by car during the evening peak period relative to free flow conditions. These findings underscore the importance of using congestion-adjusted operating speeds rather than speed limits in accessibility calculations, as reliance on speed limits can substantially overestimate energy-efficient drive accessibility in large, car-centric cities susceptible to significant congestion. In conclusion, the methodology presented here could further enrich OSM network data, making them useful for an even broader range of transportation applications.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Developing Synthetic Distribution Models Using Open-Source Data Sets

The deployment of distributed generation resources at scale with the distribution side of the power system network over the last two decades has spurred a lot of research interest in distribution networks. Given, the sensitive nature of power system data sets, utilities are still reluctant to open-source distribution networks. Although some open-source data sets are available, they may not cover the region of interest. In this paper, we introduce the Synthetic dIstribution Network Generator (SING), new software that allows users to develop synthetic distribution models, using open-source data sets, in any region of interest within the continental US.

data set↗
DOE OSTI2022

A Hybrid Data-Driven and Model-Based Anomaly Detection Scheme for DER Operation

This paper proposes a hybrid data and model-based anomaly detection scheme to secure the operation of distributed energy resources (DERs) in distribution grids. Data-driven autoencoders are set up at the edge device level and they use local DER operational data as inputs. The abnormal statuses are detected by analyzing reconstruction errors. In parallel, modelbased state estimation (SE) is set up at the central level and it uses system-wide models and measurements as data inputs. The anomalies are identified by analyzing measurement residuals. The hybrid scheme preserves the benefits of both data-driven and model-based analyses and thus improves the robustness and the accuracy of anomaly detection. Numerical tests based on the model of a real distribution feeder in Southern California highlight the proposed scheme's effectiveness and benefits.

anomaly detection↗
DOE OSTI2023

Scaling Automatic Vector Data Alignment to Satellite Imagery

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Potnis, Abhishek↗
DOE OSTI2023

High-Resolution Synthetic Solar Irradiance Sequence Generation: An LSTM-Based Generative Adversarial Network

The rapid growth of renewable energy resources penetration is bringing more challenges to power system planning and operation. Relevant renewable energy integration studies, such as the capability and dynamic performance of inverter-based resources' primary frequency response and fast frequency response, require high-resolution renewable generation output data that are representative of renewable energy resources. This paper focuses on creating synthetic but realistic solar irradiance data and proposes a long short-term memory-based generative adversarial network to generate high-resolution (second-level) solar irradiance sequences from low-resolution (minute-level) measurements. Combined with a classifier to recognize the solar irradiance patterns, the proposed model is trained using multi-loss functions to accurately capture the temporal correlations among both high-resolution and low-resolution sequences. Verification of the proposed approach is performed on the data set of the Oahu Solar Measurement Grid collected through the National Renewable Energy Laboratory. The results of the case studies demonstrate the proposed approach's capability to capture the statistical characteristics of different solar irradiance patterns and to generate high-quality synthetic solar irradiance sequences in high resolution.

dynamic scheduling↗
DOE OSTI2022

Limitations of reanalysis data for wind power applications

Wind energy resource estimates commonly depend on simulated wind speed profiles generated by reanalysis or weather models due to the lack of long time series measurements with sufficient coverage at relevant heights (roughly 90 m above ground). However, modeled data, including reanalyses, can be noisy and display a wide range of biases and errors, variously attributed to terrain effects, poor coverage of assimilated inputs, and model resolution. Wind generation records, if available at high temporal and geographical resolution, can provide a proxy for wind measurements and allow for evaluation of reanalyses and weather model wind time series. We use a 7-year-long data set of hourly, plant-level generation records from over 100 wind plants across Texas to evaluate two commonly used reanalysis data sets (MERRA2 and ERA5). Additionally, we use 1-year of records (2019) to evaluate an operational, high-resolution regional weather modeling product (HRRR v3). We find that across the region, and across all modeling products, the modeled representation of wind generation (i.e., wind speeds at hub heights passed through a power curve) has relatively small mean errors when aggregated daily, but that accuracy and hourly correlation have a strong diurnal sensitivity. Accuracy and correlation systematically decline through the evening and markedly improve after sunrise. These diurnal patterns persist even in the highest resolution model tested (HRRR v3). We hypothesize the nighttime decline in accuracy is mostly due to poorly represented boundary layer conditions, perhaps related to model representation of stability, while other uncertainties (such as wake effects) play a secondary role.

17 WIND ENERGY↗
DOE OSTI2025

Scalable Hybrid Learning Techniques for Scientific Data Compression

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data (PD), scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Here, this article presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.

ITER↗
DOE OSTI2021

Scripts and raw/processed precipitation NRCS network data, Upper Colorado, 2008-2017

This data package consists of scripts and data that were used to generate all the results and figures for Mital et al., 2020 (https://doi.org/10.3389/frwa.2020.00020). The purpose of the study was to develop a new algorithm to impute (or gap-fill) missing daily precipitation data. Study area was the Upper Colorado Water Resources Region (UCWRR), over a time period of 2008-2017. In terms of data, the package consists of raw precipitation measurements from Natural Resources Conservation Service (NRCS) network. These raw measurements are compiled into a single csv file (NRCS_dates_2008_2017.csv). The corresponding metadata are compiled in Metadata_processed.csv. The package also consists of outputs generated during Baseline and Sequential imputation runs as documented in Mital et al., 2020. Scripts used to generate all the results and figures for Mital et al., 2020 are also uploaded. A readme file documenting the layout of the data archive is also uploaded, and a KML file shows the geographic range for the data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

SKA shakes hands with Summit

Recently, a full-scale data processing workflow (Fig. 1) of the Square Kilometre Array (SKA) Phase 1 was successfully executed on the world’s fastest supercomputer Summit, proving that scientists have the expertise, software tools and computing resources to process the SKA data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Bioinformatic teaching resources - for educators, by educators - using KBase, a free, user-friendly, open source platform

Over the past year, biology educators and staff at the Department of Energy Systems Biology Knowledgebase (KBase) initiated a collaborative effort to develop a curriculum for bioinformatics education. KBase is a free and easily accessible data science platform that integrates many bioinformatics resources into a graphical user interface built upon reproducible analysis notebooks. KBase held conversations with college and high school instructors to understand how KBase could potentially support their educational goals. These conversations morphed into a working group of biological and data science instructors that adapted the KBase platform to their curriculum needs, specifically around concepts in Genomics, Metagenomics, Pangenomics, and Phylogenetics. The KBase Educators Working Group developed modular, adaptable, and customizable instructional units. Each instructional module contains teaching resources, publicly available data, analysis tools, and markdown capability to tailor instructions and learning goals for each class. The online user interface enables students to conduct hands-on data science research and analyses without requiring programming skills or their own computational resources (these are provided by KBase). Alongside these resources, KBase continues to work with instructors, supporting the development of additional curriculum modules. For anyone new to the platform, KBase, and the growing KBase Educators Organization, provides a community network, accompanied by community-sourced guidelines, instructional templates, and peer support to use KBase within a classroom whether virtual or in-person.

59 BASIC BIOLOGICAL SCIENCES↗