Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Multi-Source Data Aggregation and Real-Time Anomaly Classification and Localization in Power Distribution Systems

This paper proposes a real-time anomaly location and classification framework for power distribution systems to simultaneously determine the type of anomaly (i.e., short-circuit fault, cyber attack, DER switching) and its location. The proposed framework employs the data aggregation module to collect the measurement data from multiple field devices operating at different sampling rates, such as protection relays and D-PMUs. The output of the data aggregation is then fed into a multi-task learning-based long-based short-term memory (MTL-LSTM) to classify the type of anomaly and the location in two separate tasks. The proposed MTL-LSTM approach can be utilized in real-time operation in order to distinguish between normal and several anomalous operations and locate the anomaly. The proposed framework is tested on a modified IEEE 33-bus test feeder benchmark that integrates solar generation and energy storage. Furthermore, the results show that the proposed framework can locate and classify anomalies for several operation conditions with more than 96% accuracy. Further experiments highlight the impact of aggregating multiple sources of data on the performance of the proposed model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Divide and conquer: using RhizoVision Explorer to aggregate data from multiple root scans using image concatenation and statistical methods

Roots are important in agricultural and natural systems for determining plant productivity and soil carbon inputs. Sometimes, the amount of roots in a sample is too much to fit into a single scanned image, so the sample is divided among several scans, and there is no standard method to aggregate the data. Here, we describe and validate two methods for standardizing measurements across multiple scans: image concatenation and statistical aggregation. We developed a Python script that identifies which images belong to the same sample and returns a single, larger concatenated image. These concatenated images and the original images were processed with RhizoVision Explorer, a free and open-source software. An R script was developed, which identifies rows of data belonging to the same sample and applies correct statistical methods to return a single data row for each sample. These two methods were compared using example images from switchgrass, poplar, and various tree and ericaceous shrub species from a northern peatland and the Arctic. Most root measurements were nearly identical between the two methods except median diameter, which cannot be accurately computed by statistical aggregation. We believe the availability of these methods will be useful to the root biology community.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Putting Our Industry's Data to Work: A Case Study of Large-Scale Data Aggregation: Preprint

With increasing deployment of Advanced Metering Infrastructure (AMI), Building Automation System (BAS) controls, Internet of Things (IoT) network devices, and data-driven evaluation, measurement, and verification studies, the building sector is currently generating a staggering amount of energy-related data. In the right hands, these data sets can contribute to increased comfort and energy savings for building occupants and a more reliable electrical grid; however, due to a combination of factors, including significant privacy concerns, much of the data that are presently generated and stored are not used outside of basic operational applications. In the past year, our team has dedicated over 2,000 person-hours to accessing building energy data for a project funded by the U.S. Department of Energy (DOE) Building Technologies Office. We sought whole-building or end-use (e.g., lighting) timeseries data at the individual-building or equipment level where possible or aggregated information, such as timeseries averages and quartiles by building type (e.g., office, retail, hospital), where sharing individual building information was not an option. We are additionally working with IoT and BAS data sets to derive information important to the project. We have assembled an extensive data set that will enable the development of publicly available end-use load profiles to benefit the U.S. building and electricity industries. Here we present an overview of the data set that we have assembled to date, the motivators and approaches that got us here, and the lessons we learned through our efforts. We also discuss work underway that presents additional options for future data access.

building energy data↗
DOE OSTI2024

Aggregate data‐driven dynamic modeling of active distribution networks with DERs for voltage stability studies

Abstract Electric distribution networks increasingly host distributed energy resources based on power electronic converter (PEC) toward active distribution networks (ADN). Despite advances in computational capabilities, electromagnetic transient models are limited in scalability because of their reliance on exact data about the distribution system and each of its components. Similarly, the use of the DER_A model, which is intended to examine the combined dynamic behavior of many DERs, is limited by the difficulty in parameterization. There is a need for improved dynamic models of DERs for use in large power system simulations for stability analysis. This paper proposes an aggregate model‐free, data‐driven approach for deriving a dynamic partitioned model (DPM) of ADNs. Detailed residential distribution feeders were first developed, including PEC‐based DERs and composite load models (CMLDs), from which the aggregated DPM was derived. The performance was evaluated through various case studies and validated against the detailed ADN model and state‐of‐the‐art DER_A model with CMLD. The data‐driven DPM achieved a of over 90%, accurately representing the aggregated dynamic behavior of ADNs. Furthermore, the DPM significantly accelerated the simulation process with a computational speedup of 68 times compared to the detailed ADN and a 3.5 times speedup compared to the DER_A CMLD model.

42 ENGINEERING↗
DOE OSTI2025

HexWeather: Hexagonal Spatial Data Aggregation for Weather-Driven Grid Resilience Analysis

Extreme weather accounts for over 8 0 % of major U.S. power outages since 2000, highlighting the need for spatial tools that align weather data with the irregular boundaries of electric infrastructure. This paper introduces HexWeather, a modular, resolution-aware framework for aggregating historical and forecasted weather data using Uber's H3 hexagonal spatial indexing system. Unlike traditional methods that rely on state or county-level grids, HexWeather enables weather analysis across custom geographies such as utility service areas where public datasets are often unavailable or misaligned. Using Open-Meteo data, we evaluate how H3 resolution affects anomaly detection, spatial variability, and forecast uncertainty across three scales: state, county, and utility. Results show that while coarse resolutions suffice for broad trend tracking, finer resolutions are essential for identifying localized variability and operational risks. By applying metrics like Z-score standard deviation and interquartile range, HexWeather quantifies the spatial spread of both historical anomalies and forecasted conditions, allowing users to assess resolution adequacy for each analysis. This framework supports rapid weather data reuse, reproducible anomaly detection, and predictive modeling for infrastructure resilience. By bridging spatial misalignment in traditional datasets and enabling retrospective and forward-looking analysis within the same pipeline, HexWeather lays the groundwork for better post event analysis, outage prediction, and resilience planning.

Morris, Jacob [ORNL]↗
DOE OSTI2025

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Data Integration Aggregated Model For Nuclear Deployment

The data of a nuclear utility or plant are stored in various isolated forms in different systems. These data have different structures and tools and are, therefore, used independently. The integration of data sources is often needed and is performed manually and on an as-needed basis. The integration of the data would enable creating a data warehouse for a single nuclear power plant (NPP) or multiple plants. A data model contains objects that make up a given domain along with their attributes and relationships and could be used as a standard for exchange between applications for simple integration and data aggregation. After a survey of existing models, it was determined that a new model was required to meet the scope of this problem. DIAMOND has been developed in the form of an ontology to meet these needs. An ontology is simply a data model that also acts as a taxonomy for the domain, organizing the objects in a hierarchy with the most general objects at the top.

Browning, JerenM↗
DOE OSTI2021

Deep Learning Based Approach to Integrate MyShake's Trigger Data with ShakeAlert for Faster and Robust EEW Alerts

Earthquake Early Warning (EEW) systems depend on the dense seismic network to make fast and accurate earthquake detections to issue useful early alerts. The recent development and implementation of the ShakeAlert system is relying on the west coast seismic network to detect and estimate the earthquake parameters in real-time. While working on making improvements on the current system, one potential approach is to include more seismic recordings from various sources, such as the low-cost smartphone seismic network, such as the MyShake network, or the Google Android smartphones, since these smartphone seismic networks have much more portable stations (phones) to potentially provide useful data to the system. This report summarizes the initial exploration of using a deep learning approach to combine the data from both traditional seismic stations and the smartphone data in California. Due to the heterogeneity nature of the data, we aggregate data from the phones (using simulation data), as well as that from traditional seismic stations to grid cells. By generating real-time triggering ratio grid cell maps, the designed deep learning algorithm can process the data from multiple sources and detect the earthquake faster than only using that from a traditional seismic network.

58 GEOSCIENCES↗
DOE OSTI2020

Using Meteorological Data to Evaluate Worldwide PV Degradation Rates

Here we automate the process to enable the creation of world maps for specific PV durability metrics. The end goal is to create a tool for the creation of world maps predicting the sensitivity of modules to degradation to inform the development and utilization of module testing standards and to provide information to module designers. TMY (Typical Meteorological Year) Data was aggregated from sites around the world to render insight into factors contributing to the degradation of solar modules. The focus of aggregating data was to create a single dataset to work with utilizing Solar Irradiance values combined with Ambient Temperature, Relative Humidity, Wind Speed and Dew-Point Temperature.

14 SOLAR ENERGY↗
DOE OSTI2020

Aggregation and data driven identification of building thermal dynamic model and unmeasured disturbance

An aggregate model is a single-zone equivalent of a multi-zone building, and is useful for many purposes, including model based control of large heating, ventilation and air conditioning (HVAC) equipment. This paper deals with the problem of simultaneously identifying an aggregate thermal dynamic model and unknown disturbances from input–output data of multi-zone buildings. The unknown disturbance is a key challenge since it is not measurable but non-negligible. In this paper, we first present a principled method to aggregate a multi-zone building model into a single zone model, and show the aggregation is not as trivial as it has been assumed in the prior art. We then provide a method to identify the parameters of the model and the unknown disturbance for this aggregate (single-zone) model. Finally, we test our proposed identification algorithm to data collected from a multi-zone building testbed in Oak Ridge National Laboratory. A key insight provided by the aggregation method allows us to recognize under what conditions the estimation of the disturbance signal will be necessarily poor and uncertain, even in the case of a specially designed test in which the disturbances affecting each zone are known (as the case of our experimental testbed). This insight is used to provide a heuristic that can be used to assess when the identification results are likely to have high or low accuracy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Community Choice Aggregation(CCA) Data Collection Webinar for Status and Trends in the Voluntary Market Report (2024 Data) [Slides]

We have subcontracted LEAN Energy US, to help us improve our CCA data collection effort for the Annual Voluntary Energy Markets Data Report. LEAN Energy US (Local Energy Aggregation Network) is a national 501(c)3 non-profit organization dedicated to accelerating the country's transition to clean and renewable power, supporting competition and customer choice in the energy sector, and maintaining affordable electricity rates. We work in partnership with a range of organizations to actively support the formation and operational success of Community Choice Aggregation (CCA) programs around the country. This webinar, hosted in partnership with LEAN Energy US, is intended to introduce their members to our data collection effort and encourage CCAs in their network to participate.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Environmental Effects Assessment for Proposed Offshore Wind Farm off the Coast of Grays Harbor, Washington

Grays Harbor Wind LLC (GHW) is proposing to develop a floating offshore wind farm offshore of west Grays Harbor County, Washington (Grays Harbor). The proposed GHW Offshore Wind Project (Project) would entail construction, installation and operation of a 1,000-megawatt (MW) offshore wind farm consisting of approximately 75 floating units, each containing a floating foundation and wind turbine generator (WTG). The Project location is approximately 25 miles (21.7 nautical miles [nmi]) offshore west of Grays Harbor, at waters depths of 360 to 700 feet. The Pacific Northwest National Laboratory (PNNL) was contracted by Herrera Environmental Consultants, Inc. on behalf of GHW to carry out this preliminary scoping study to evaluate baseline conditions and potential effects on fish and marine mammals from development and operation of a floating offshore wind farm installed within a designated area off the coast of Washington1. Floating offshore wind units installed in an ocean environment as part of the Project would interact with marine wildlife. This Study report provides an initial data aggregation and analysis, using publicly available data, of the Project effects, both negative and positive, on the marine environment. The scope of this assessment is limited by the fact that the Project development is presently at the conceptual level. Data on marine organisms were aggregated and evaluated; however seabirds were evaluated by Herrera Environmental Consulting and, along with bats, were not included in the scope of this study. Significant additional work is necessary to characterize ocean, seafloor, and environmental conditions; select appropriate floating offshore wind technologies; identify construction methods and locations; and assess facility locations, including electrical interconnection. Data aggregation and analysis of seabirds and bats is also needed. Evaluation of the full range of potential environmental effects would be conducted following an award of a lease from the Bureau of Ocean Energy Management (BOEM) as part of the leasing, National Environmental Policy Act (NEPA)/State Environmental Policy Act (SEPA) environmental review and permitting processes. While this initial data aggregation and analysis uses best available public scientific information and current assumptions about the Project configuration, the effects discussed herein are based on the status of review to-date and may change as Project-specific details are developed.

17 WIND ENERGY↗
DOE OSTI2021

Replication Data for: Aggregated nanoparticles: Sample preparation and analysis by atom probe tomography

The data underlying this published work have been made publicly available in this repository as part of the IMASC Data Management Plan. This work was supported as part of the Integrated Mesoscale Architectures for Sustainable Catalysis (IMASC), an Energy Frontier Research Center funded by the U.S. Department of Energy, Office of Science, Basic Energy Sciences under Award # DE-SC0012573.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Final Report)

The AGGREGATE project team successfully developed and validated various modules for outage management. Brief summaries of each module are provided to showcase their strength for outage management and restoration for a distribution system with a high penetration of connected distribution energy resources (DERs). In recent years, inverter-based DERs have been widely deployed in distribution system. A most of behind-the-meter (BTM) solar power generation is not visible to the utility. The data-driven DER and load estimation modules are using machine learning (ML) and artificial intelligence (AI) to manage this issue, which provides an opportunity for distribution system operators (DSOs) to operate systems and make decisions in real-time for a distribution system with a high penetration of DERs deployed. Also, the estimated DER and true load can be further leveraged in network aggregation and cold-load pick up estimation for reducing the computing complexity and providing for fast restoration. After load demand and DER power generations have been estimated, the information will support topology and state estimation (SE). The topology estimation module demonstrated the viability of mixed integer linear programming (MILP) formulation to estimate the most likely operational radial topology and outage sections using power flow measurements, historical/estimated load and DERs data and smart meter ping measurements. Formulation includes continuous (power flow, load and DERs data) and binary measurements (smart meter ping measurements) in a single formulation. Errors in continuous data and binary data are modeled as normal distribution and Bernoulli distribution, respectively. In the future distribution grid, the power injection from controllable DERs will be essential for efficient and resilient grid operation. However, determining the optimal DER injections and restoration actions is dependent on knowledge of the system states. State estimation (SE), already the cornerstone of transmission energy management systems, will become commonplace in distribution management systems as more measurements become available from deployment of automated metering infrastructure (AMI). Observability analysis is the first step in SE, as it determines the sufficiency of the available measurements for accurately estimating the current system states. A new type of pseudo-measurement called a Correlational Measurement (CM) is introduced in this module, to enhance the observability of the system to enable more accurate SE. CMs encapsulate knowledge of correlation between demand patterns for similar classes of loads as well as injection patterns for same-technology renewable DERs. During grid contingency scenarios, DERs have been traditionally disconnected, without any fault ride-through capabilities. However, with new regulations and better technology, it is feasible for these resources to contribute to the grid’s restoration after an adverse event and hence enhance resilience. The controllability module proposes a two-step restoration scheme for the power system restoration process by leveraging additional degrees of freedom in power electronics interfaced DERs for mitigating voltage problems. In a resilience mode without the utility system, the distribution grid relies on DERs to serve critical load. In such a severe event with multiple faults on the distribution feeders, actuation of various protective devices (PDs) divides the distribution system into electrical islands. The undetected actuated PDs due to fault current contributions from DERs can delay the restoration process, thereby reducing the system resilience. The Advanced Outage Management (AOM) and the Advanced Feeder Restoration (AFR) modules developed in this project provide improved system resilience with multiple DERs. AOM identifies the faulted sections and actuated PDs in a distribution system with DERs by incorporating smart meter data. The most credible outage scenario including fault locations, PD actuations, and fault indicator (FI) failures is identified by a set of binary integer linear programming incorporating hypotheses. The AFR module serves to restore a distribution system with available energy resources taking into consideration the availability of utility sources and DERs. By partitioning the system into islands, critical load will be served with the available generation resources within islands based on the solution of a MILP. When the utility systems become available, the optimal path will be determined by a spanning tree search algorithm that reconnects these islands back to substations and restores the remaining load. The transmission and distribution (T&D) co-simulation module was used to validate the effect of a control action performed on the distribution side assets as it propagates to the transmission side. This ensures that the control action performed results in a feasible operating point on both the transmission and the distribution system. In addition to validation, the team used the T&D co-simulation module to demonstrate how distribution system assets can be used to mitigate issues on the transmission system. Specifically, the team demonstrated that appropriate switching operations on the distribution side can alleviate the line overload condition on the transmission side without causing new operational constraint violations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Implementing Directive-Based Deferred Execution for Effective Network Aggregation

Remote direct memory access technology provides an efficient mechanism for one-sided communication that can be leveraged to implement a distributed shared memory programming model. However, when applications generate large numbers of small, irregular messages, network congestion often arises. Existing solutions address this small message problem by facilitating message aggregation but typically require disruptive code transformations that detract from the algorithmic intent of applications, or can be limited by dependent operations on aggregated data between synchronisation points. A solution is to use a directive-assisted approach that enables compilers to transform code dependent on aggregated communication for deferred execution. This paper presents an algorithm that a compiler can use to implement and optimise deferred execution for code dependent on aggregated data, based on an "aggregation context" extension for the OpenSHMEM partitioned global address space library. This new capability addresses a key challenge of message aggregation, allowing its full potential to reduce network congestion and enhance programmability to be realised.

Welch, Aaron [ORNL]↗
DOE OSTI2025

Satellite Imagery of PV Site Storm Damage

"This repository contains multiple data sets focused on visible damage to photovoltaic (PV) installations following extreme weather events such as hailstorms and hurricanes. Data sets are split into two categories: the first category, the ‘manually labeled’ data, was compiled by researchers manually, and contains manually identified PV sites exposed to storms. The second data set, the ‘aggregated’ data, is a compilation of the manually labeled PV sites and deep learning-identified PV sites. The hail damage data set focuses on post-storm PV damage following a September 24, 2023 hailstorm in Austin, TX, which caused over $600 million in damages in the Austin metro area. The hurricane damage data set focuses on post-storm PV damage following Hurricanes Irma and Maria in Puerto Rico and the US Virgin Islands. Hurricanes Irma and Maria were back-to-back category 5 hurricanes, which pummeled the Caribbean and southeastern United States in September 2017, causing an estimated $115.2 billion in damages."

14 SOLAR ENERGY↗