Search NASA⌕ Search

SEARCH · Search NASA

Results for “data infrastructures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Generalization Equations for Machine Learners Based on Physical and Abstract Laws

The physical and abstract laws derived from the first principles have been recently exploited to customize and sharpen machine learning (ML) methods and also derive their generalization equations. These laws often encapsulate knowledge that complements datasets and ML models. We present a generic framework that uses these laws to provide ML codes that are transferable across multiple areas, including data transport infrastructures and thermal hydraulics analytics of nuclear reactors. By anchoring on datasets from these areas and the statistical generalization theory, we present a rigorous approach to co-develop ML solutions and the generalization equations that characterize them, by exploiting the structure and constraints from the laws. We present illustrative examples using practical problems from existing literature on the performance characterization of data transport infrastructures, and the sensor error and power level estimation in nuclear reactor systems using sensor measurements of primary and secondary coolant systems, respectively.

Rao, Nageswara↗
DOE OSTI2025

Utilizing AI and Spatial Data to Identify & Rapidly Disseminate Energy Infrastructure Insights

GeoGov Summit Final Presentation entitled "Utilizing AI and Spatial Data to Identify & Rapidly Disseminate Energy Infrastructure Insights". Maintaining the integrity of energy infrastructure plays a critical role in ensuring energy security. Robust foundational AI models using data from federal, state, industry, and other sources can help address integrity risk management & mitigation issues as well as evaluate extended use strategies. Trusted foundational models can help with industry adoption and accelerate innovation by enhancing integrity predictions, reduce costs, and informing infrastructure build-out. Coordination, collaboration & data sharing to develop robust models to aid in: Optimizing operations; Minimizing costs; Ensuring energy security.

Advanced Infrastructure Integrity Model (AIIM)↗
DOE OSTI2021

Atmospheric Radiation Measurement (ARM) User Facility 2020 Decadal Vision

The Atmospheric Radiation Measurement (ARM) user facility was established in 1989 by the U.S Department of Energy (DOE) Office of Biological and Environmental Research (BER) to provide an observational basis for studying the Earth’s climate. ARM began collecting observations in 1992 and was designated a user facility in 2003. The facility includes a network of extensively instrumented long-term fixed-location observatories and mobile facilities. The ARM facility also includes an aerial component to augment these ground-based measurements. Because of the diversity of in situ and remotely obtained observations, ARM’s data management infrastructure is designed to collect, process, and deliver data to the research community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Urban Energy Systems: Research at Oak Ridge National Laboratory

In the coming decades, our planet will witness unprecedented urban population growth in both established and emerging communities. The development and maintenance of urban infrastructures are highly energy-intensive. Urban areas are dictated by complex intersections among physical, engineered, and human dimensions that have significant implications for traffic congestion, emissions, and energy usage. In this chapter, we highlight recent research and development efforts at Oak Ridge National Laboratory (ORNL), the largest multipurpose science laboratory within the U.S. Department of Energy’s (DOE) national laboratory system, that characterizes the interactions between the human dynamics and critical infrastructures in conjunction with the integration of four distinct components: data, critical infrastructure models, and scalable computation and visualization, all within the context of physical and social systems. Discussions focus on four key topical themes: population and land use, sustainable mobility, the energy-water nexus, and urban resiliency, that are mutually aligned with DOE’s mission and ORNL’s signature science and technology capabilities. Using scalable computing, data visualization, and unique datasets from a variety of sources, the institute fosters innovative interdisciplinary research that integrates ORNL expertise in critical infrastructures including energy, water, transportation, and cyber, and their interactions with the human population.

Bhaduri, Budhu↗
DOE OSTI2025

dCache: The Storage System of Choice for Data-Intensive Applications

The ever-increasing volumes of data produced by modern scientific facilities like EuXFEL and LHC put significant stress on data management infrastructure operated by laboratories and research centers. The challenges to be addressed span the entire data life cycle, from ingest and efficient data analysis to long-term preservation, typically involving large tape libraries. dCache, a storage system developed in collaboration between the Deutsches Elektronen-Synchrotron (DESY), Fermi National Accelerator Laboratory, and Nordic e-Infrastructure Collaboration (NeIC), is designed to manage a large number of disk servers and to facilitate transparent data migration to and from archival storage. Its multifaceted approach offers a unified method to support a variety of scientific use cases with the same storage infrastructure, including high-throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and long-term data preservation on tertiary storage. Initially developed for high energy physics (HEP) experiments, dCache is now used by various scientific communities, including astrophysics, biomedical research, and life sciences, each having specific requirements. This paper presents architecture, deployment strategies, performance and scalability enhancements, and recent advancements in dCache addressing the needs of scientific communities. Finally, we touch on the development and release process, ensuring the software’s high quality.

DCache↗
DOE OSTI2020

An Overview of Gemma FY20 Verification Activities

Gemma verification activities for FY20 can be divided into three categories: the development of specialized quadrature rules, initial progress towards the development of manufactured solutions for code verification, and automated code-verification testing. In the method-of-moments implementation of the electric-field integral equation, the presence of a Green’s function in the four-dimensional integrals yields singularities in the integrand when two elements are nearby. To address these challenges, we have developed quadrature rules to integrate the functions through which the singularities can be characterized. Code verification is necessary to develop confidence in the implementation of the numerical methods in Gemma. Therefore, we have begun investigating the use of manufactured solutions to more thoroughly verify Gemma. Manufactured solutions provide greater flexibility for testing aspects of the code; however, the aforementioned singularities provide challenges, and existing work is limited in rigor and quantity. Finally, we have implemented automated code-verification testing using the VVTest framework to automate the mesh refinement and execution of a Gemma simulation to generate mesh convergence data. This infrastructure computes the observed order of accuracy from these data and compares it with the theoretical order of accuracy to either develop confidence in the implementation of the numerical methods or detect coding errors.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Modeling of Microgrid for Critical Data Center Applications

As part of continuing efforts to develop support, understanding, and infrastructure of data center integration onto the electric grid, our work aims to model and analyze the behavior of data center loads in a microgrid power system. Our model consists of renewable energy sources, batteries, and a nuclear reactor-steam Rankine cycle to power data center loads. The simulation studies investigate the electrical behavior of the microgrid system to assess its ability in supporting large data center electrical demands.

14 - SOLAR ENERGY↗
DOE OSTI2025

Adapting Grid Criticality for Data Centers

This presentation explores the evolving definition of “critical load” in the electric grid, emphasizing the growing importance of digital infrastructure—particularly data centers—in grid resilience, restoration, and modernization. As utilities increasingly rely on AI-driven analytics and software-defined control systems, data centers have shifted from passive electricity consumers to essential computational hubs that enable National Critical Functions (NCFs) and support real-time grid operations. The deck examines the scale and impact of digital loads, the need for grid modernization to manage rapid load growth, and the diverse computing paradigms required for AI deployment. It introduces a tiered taxonomy for classifying critical loads, highlights operational dependencies between the grid and digital infrastructure, and discusses policy implications for integrating data centers into emergency planning and restoration protocols. Through case studies and practical frameworks, the presentation provides actionable insights for utilities, regulators, and planners navigating the digital transformation of the power sector.

29 - ENERGY PLANNING, POLICY AND ECONOMY↗
DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗
DOE OSTI2024

Solar PV Railroad Setbacks: Ordinances (2022) and Extrapolated Trends

This dataset represents solar energy setback requirements from railroads. A setback requirement is a minimum distance from a railroad that an energy project may be developed. As of April 2022, no ordinances were discovered for any counties. Such ordinances are likely to arise as regulations continue to expand. Therefore, this dataset applies a 30-meter setback, sourced from trends in other infrastructure. A TIF data file and a PNG map of the data are provided, showing areas where solar energy is prohibited or permitted across the contiguous United States. For further details and citation, please refer to the publication linked below: Lopez, Anthony, Pavlo Pinchuk, Michael Gleason, Wesley Cole, Trieu Mai, Travis Williams, Owen Roberts, Marie Rivers, Mike Bannister, Sophie-Min Thomson, Gabe Zuckerman, and Brian Sergi. 2024. Solar Photovoltaics and Land-Based Wind Technical Potential and Supply Curves for the Contiguous United States: 2023 Edition. Golden, CO: National Renewable Energy Laboratory. NREL/TP-6A20-87843.

Array↗
DOE OSTI2024

Wind Turbine Oil and Gas Pipeline Setbacks: Ordinances (2022) and Extrapolated Trends, 115 Hub Height 170 Rotor Diameter

This dataset represents wind energy setback requirements from oil and gas pipelines. A setback requirement is a minimum distance from a pipeline that an energy project may be developed. As of April 2022, no ordinances were discovered for any counties. Such ordinances are likely to arise as regulations continue to expand. Therefore, this dataset applies a median setback equivalent to 1.1 times the turbine tip-height, sourced from trends in other infrastructure. A TIF data file and a PNG map of the data are provided, showing areas where wind energy is prohibited or permitted across the contiguous United States. The turbine parameters used were a hub-height of 115 meters and a rotor diameter of 170 meters, as obtained from the Annual Technology Baseline (ATB) 2022. For further details and citation, please refer to the publication linked below: Lopez, Anthony, Pavlo Pinchuk, Michael Gleason, Wesley Cole, Trieu Mai, Travis Williams, Owen Roberts, Marie Rivers, Mike Bannister, Sophie-Min Thomson, Gabe Zuckerman, and Brian Sergi. 2024. Solar Photovoltaics and Land-Based Wind Technical Potential and Supply Curves for the Contiguous United States: 2023 Edition. Golden, CO: National Renewable Energy Laboratory. NREL/TP-6A20-87843.

Array↗
DOE OSTI2023

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗
DOE OSTI2024

Statistical Estimation of EV Driver Charging Behavior and Influential Factors

INL received data collected via telematics from battery electric vehicles (BEVs), and these vehicles were owned by retail customers who had entered into a telematics user agreement. The goal of analyzing these data was to develop mathematical models to characterize how different sets of BEV drivers use charging infrastructure at home and away from home (i.e., public charging) and quantify how various factors influence BEV drivers’ decision to charge and use available infrastructure. The data used in this analysis are unique because they provide real world BEV driving and charging behavior at the individual driving and parking event level. In this study we seek to leverage this data to quantify BEV charging and driving metrics to help inform models that predict quantities like the specific times when loads are imposed on the electrical grid due to BEV charging. Most models that have been developed to predict electrical grid load due to BEV charging, use simulations of BEV driving events and rely on assumptions such as every vehicle charges every night. Using a statistical modelling framework, we seek to investigate BEV charging behavior and quantitatively assess these common assumptions of BEV charging behavior.

33 - ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Estimating Flexibility Envelopes for Residential Customers From Utility Smart Meter Data: Preprint

Demand response from residential customers has significant potential to support power system operations, but accurate flexibility estimation is challenging due to the limited resolution of advanced metering infrastructure (AMI) data. Most utility AMI measurements are recorded at hourly intervals, with only a small portion at higher resolutions, and even fewer households have appliance-level energy usage data. To address this issue, this paper proposes a two-stage long short-term memory (LSTM) framework for estimating household flexibility envelopes from low-resolution AMI data. In the first stage, the heating, ventilating, and air-conditioning (HVAC) load and non-HVAC loads are estimated by using a model trained on a small set of households with appliance-level profiles. These estimated data are then used to compute the upper- and lower-flexibility bounds, which are subsequently down-sampled to lower-resolution data. In the second stage, these flexibility bounds serve as training inputs for another LSTM model, enabling direct prediction of flexibility envelopes for households with only hourly AMI data. This method is validated using Pecan Street data from two different areas, and the results demonstrate its applicability and effectiveness.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Integrated Framework of Multisource Data Fusion for Outage Location in Looped Distribution Systems

Accurate outage location is essential for expediting post-outage power restoration, minimizing outage duration, and enhancing the resilience of distribution networks. With the advent of advanced metering infrastructure, data-driven outage location methods have significantly advanced beyond traditional approaches that rely on manual inspections. However, existing methods still face critical challenges, like reliance on single-source data, limited ability to handle partially observable systems or difficulties with loop networks. To the best of our knowledge, no single approach has comprehensively addressed all of these challenges at once. To this end, this paper proposes a comprehensive multisource data fusion framework for outage locations via probabilistic graph networks. The framework consists of three key phases. First, a novel method for reconstituting distribution networks with loops is developed, transforming looped networks into multiple radial subnetworks that retain all outage causalities of the original network. Second, Bayesian network (BN) models are established for each subnetwork, integrating multiple data sources and network structures. Finally, a joint Gibbs sampling mechanism, featuring forward and backward information flow, is designed to merge data from separate BN models and maximize the utilization of limited evidence, ensuring accurate outage location identification. In conclusion, the framework was validated on two modified public test systems, and comparative studies confirmed its effectiveness.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Reducing Uncertainty of Fielded Photovoltaic Performance (Final Technical Report)

Improved analysis and reporting of photovoltaic (PV) field performance increases the certainty of owners and financiers that systems will perform as expected. Advanced module technologies (e.g., PERC, HJT, and bifacial) introduce new degradation mechanisms and performance characteristics. The FY19-21 Reducing Uncertainty project leveraged data from the ever-increasing PV fleet to develop models and understanding of the field performance of existing and new technologies. Specifically, we accomplished: report on field performance and degradation rates for high-efficiency silicon (HJT, PERC, IBC) and more conventional technologies; developed automated analysis techniques to quantify system performance (performance ratio, energy yield) and production shortfalls (soiling, degradation, availability); refined the RdTools software toolkit to bring standard, validated analysis techniques to bear on third-party data; analyzed and reported on large datasets including Treasury data and Lawrence Berkeley National Laboratory's Utility-Scale dataset to expand the high-quality degradation-rate histogram published previously; worked with industry partners and the DuraMAT data hub to enable private parties to share and aggregate PV production data anonymously, leveraging cloud-based data analysis infrastructure and publishing on US fleet-scale performance comprising over 7GW of operating systems. (https://www.nrel.gov/pv/fleet-performance-data-initiative.html). Through our industry collaborations we have engaged in NDA-covered data transfer with twelve PV fleet owners as of January 2022, with more agreements in negotiation. Our scalable cloud-based time series database contains over 30 billion rows (20TB) of PV time series data, representing over 1700 commercial and utility-scale systems, and over 7.2 GW of DC capacity (Fig 1). Initial field performance results have been distributed in several public reports. Because our fleet composition and data quality methods are continually improving, annual updates to these results are published to our PV Fleet webpage [ https://www.nrel.gov/pv/fleet-performance-data-initiative.html ] and DuraMAT data hub [DOI: 10.21948/1842958]. Another existing dissemination channel used for observed soiling losses is a map we maintain for soiling losses. Additional products developed include a report detailing fleet-wide performance index, availability, startup loss and snow loss factors, a detailed report on the 1603 grant dataset comprising over 100,000 PV systems with failure and performance details and a utility-scale report coauthored with LBNL on 31 GW of system performance.

14 SOLAR ENERGY↗