Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Enhancing Privacy in Participatory Sensing Applications with Multidimensional Data

Participatory sensing applications rely on individuals to share personal data to produce aggregated models and knowledge. In this setting, privacy concerns can discourage widespread adoption of new applications. We present a privacy-preserving participatory sensing scheme based on negative surveys for both continuous and multivariate categorical data. Without relying on encryption, our algorithms enhance the privacy of sensed data in an energy and computation efficient manner. Simulations and implementation on Android smart phones illustrate how multidimensional data can be aggregated in a useful and privacy-enhancing manner.

Forrest, Stephanie↗

Data-Driven Pricing Strategy for Demand-Side Resource Aggregators

We consider a utility who seeks to coordinate the energy consumption of multiple demand-side flexible resource aggregators. For the purpose of privacy protection, the utility has no access to the detailed information of loads of resource aggregators. Instead, we assume that the utility can directly observe each aggregator's aggregate energy consumption outcomes. Furthermore, the utility can leverage resource aggregator energy consumption via time-varying electricity price profiles. Based on inverse optimization technique, we propose an estimation method for the utility to infer the energy requirement information of aggregators. Subsequently, we design a data-driven pricing scheme to help the utility achieve system-level control objectives (e.g., minimizing peak demand) by combining hybrid particle swarm optimizer with mutation algorithm and an iterative algorithm. Case studies have demonstrated the effectiveness of the proposed approach against two benchmark pricing strategies-a flat-rate scheme and a time-of-use scheme.

data-driven optimization↗

Reducing Uncertainty of Fielded Photovoltaic Performance (Final Technical Report)

Improved analysis and reporting of photovoltaic (PV) field performance increases the certainty of owners and financiers that systems will perform as expected. Advanced module technologies (e.g., PERC, HJT, and bifacial) introduce new degradation mechanisms and performance characteristics. The FY19-21 Reducing Uncertainty project leveraged data from the ever-increasing PV fleet to develop models and understanding of the field performance of existing and new technologies. Specifically, we accomplished: report on field performance and degradation rates for high-efficiency silicon (HJT, PERC, IBC) and more conventional technologies; developed automated analysis techniques to quantify system performance (performance ratio, energy yield) and production shortfalls (soiling, degradation, availability); refined the RdTools software toolkit to bring standard, validated analysis techniques to bear on third-party data; analyzed and reported on large datasets including Treasury data and Lawrence Berkeley National Laboratory's Utility-Scale dataset to expand the high-quality degradation-rate histogram published previously; worked with industry partners and the DuraMAT data hub to enable private parties to share and aggregate PV production data anonymously, leveraging cloud-based data analysis infrastructure and publishing on US fleet-scale performance comprising over 7GW of operating systems. (https://www.nrel.gov/pv/fleet-performance-data-initiative.html). Through our industry collaborations we have engaged in NDA-covered data transfer with twelve PV fleet owners as of January 2022, with more agreements in negotiation. Our scalable cloud-based time series database contains over 30 billion rows (20TB) of PV time series data, representing over 1700 commercial and utility-scale systems, and over 7.2 GW of DC capacity (Fig 1). Initial field performance results have been distributed in several public reports. Because our fleet composition and data quality methods are continually improving, annual updates to these results are published to our PV Fleet webpage [ https://www.nrel.gov/pv/fleet-performance-data-initiative.html ] and DuraMAT data hub [DOI: 10.21948/1842958]. Another existing dissemination channel used for observed soiling losses is a map we maintain for soiling losses. Additional products developed include a report detailing fleet-wide performance index, availability, startup loss and snow loss factors, a detailed report on the 1603 grant dataset comprising over 100,000 PV systems with failure and performance details and a utility-scale report coauthored with LBNL on 31 GW of system performance.

14 SOLAR ENERGY↗

Initial Mobility Analysis for ORNL VA-EDH Synthetic Populations

Travel burdens are a major barrier to healthcare access among US Veteran patient populations, particularly those residing in rural areas. Spatial accessibility to points of care for US Veteran populations is commonly assessed in two ways. The first approach uses open data from the US Census to represent collective travel burdens, for example the distance between population-weighted census tract centroids and VHA points of care. The second approach uses restricted-access VHA patient data to measure travel costs (e.g., distance, time) for accessing points of care with respect to geolocated patient addresses and real or approximated transportation networks. While the advantage of the open data approach lies in its reproducibility, it has notable limitations in its tendency to infer individual travel behavior from aggregate population characteristics, a problem known as ecological fallacy. Conversely, while the patient data approach is able to account for individual travel behavior, its ability to account for localized access disparities (e.g., a neighborhood with exceptionally high transportation costs) and patient demographics is limited as protecting individual patient data requires their storage in closed systems with limited capacity for adequately modeling real-world travel patterns or for supplementing patient attributes. Additionally, the patient data approach cannot account for veterans who are not enrolled in the VHA system but who may be eligible for care. These challenges limit the ability to perform “what if” analyses on the effects of place-specific interventions on veteran populations with high access barriers to healthcare. To address these challenges, we explore the application of realistic synthetic populations to examine travel burdens and spatial accessibility issues among veteran patient populations. Synthetic populations provide a virtual, individually-resolved and cross-sectional representation of the veteran patient population that enables investigation of spatial access to points of care in ways in which aggregate data and patient data do not. First, synthetic populations allow one to directly assess how individuals access points of care, from synthesized residential locations to outpatient facilities on real-world transportation networks. Modeling access to points of care at the individual scale addresses the ecological fallacy problem associated with using aggregated census data to represent veteran populations and patterns of movement. Second, synthetic populations provide a means of completely representing an area’s veteran population using only publicly available, anonymized census microdata from the American Community Survey (ACS) to ensure the privacy of real-world individuals. Generating synthetic populations from the ACS also expands descriptive characteristics beyond what patient data typically offers to include socio-demographic, economic, housing, and mobility attributes. More detailed profiles of both VHA patient populations and veterans not enrolled in the VA system will provide a comprehensive picture of groups that may benefit from interventions or outreach. As an initial exercise for using synthetic populations to measure veteran travel burdens to VA care, we apply Oak Ridge National Laboratory’s (ORNL) UrbanPop capability to generate a series of synthetic VHA patient populations for 9 Veterans Integrated Services Networks (VISN) market areas in 9 Census Divisions across the continental United States, which are listed in Table 1. We use UrbanPop to produce synthetic populations for the VISN markets selected for each US Census Division, then assign VA outpatient clinic destinations to synthetic VHA patients based on travel about each VISN market’s road network. To demonstrate using the synthetic populations to evaluate healthcare travel burdens, we compare the time-based impedance between simulated home locations and VA outpatient clinics in each VISN market. We then perform validation exercises on the synthetic populations with respect to neighborhood (block group) demographic composition as well as patient mobility, comparing aggregate origin-destination statistics for the synthetic population to outpatient visits available in restricted patient data from the VA’s Corporate Data Warehouse (CDW) database.

97 MATHEMATICS AND COMPUTING↗

Curating Carbon Storage Data for Reuse: Enabling Research and Modeling from Earth’s Surface to Subsurface

The volume of public geologic carbon storage (GCS) data resources has continued to increase in recent years as the result of an increase in funding from government, industry, and academia towards national, basin, regional and field scale studies to ensure carbon capture and storage becomes a commercially viable operation. Despite the increasing volume of data, GCS data applied towards analyses such as geologic, cost, and risk modeling continues to be multi-sourced and often disparate in nature, published across government agencies, websites, data repositories and buried in derivative reports and documents. Much of the time preparing for an analysis and derivative product development is spent collecting, aggregating, transforming and preparing input data. There have been significant efforts within the DOE National Energy Technology Laboratory’s Carbon Storage Program to optimize multi-source, multi-scale subsurface geologic data curation and aggregation to support data discovery, interoperability, and reuse. Methods include the use of artificial intelligence, machine learning, and data science techniques. This talk will discuss the workflows, best practices, and processes developed to support the aggregation and curation of data through the whole system – surface to subsurface data - that support multi-scale, multi-purpose analysis for carbon storage research.

Morkner, Paige↗

Soil pore network response to freeze-thaw cycles in permafrost aggregates

This dataset contains data used for the paper "Pore network response to freeze-thaw cycles in permafrost aggregates". The Related References field will be updated with a full citation when available.Climate change in Arctic landscapes may increase freeze-thaw frequency within the active layer as well as newly thawed permafrost. A highly disruptive process, freeze-thaw can deform soil pores and alter the architecture of the soil pore network with varied impacts to water transport and retention, redox conditions, and microbial activity. Our objective was to investigate how freeze-thaw cycles impacted the pore network of newly thawed permafrost aggregates to improve understanding of what type of transformations can be expected from warming Arctic landscapes. We measured the impact of freeze-thaw on pore morphology, pore throat diameter distribution, and pore connectivity with X-ray computed tomography (XCT) using six permafrost aggregates with sizes of 2.5 cm3 from a mineral soil horizon (Bw; 28-50 cm depths) in Toolik, Alaska. Freeze-thaw cycles were performed using a laboratory incubation consisting of five freeze-thaw cycles (-10˚C to 20˚C) over five weeks. Our findings indicated decreasing spatial connectivity of the pore network across all aggregates with higher frequencies of singly connected pores following freeze-thaw. Water-filled pores that were connected to the pore network decreased in volume while the overall connected pore volumetric fraction was not affected. Shifts in the pore throat diameter distribution were mostly observed in pore throats ranges of 100 microns or less with no corresponding changes to the pore shape factor of pore throats. Responses of the pore network to freeze-thaw varied with aggregate, suggesting that initial pore morphology may play a role in driving freeze-thaw response. Our research suggests that freeze-thaw alters the microenvironment of permafrost aggregates during the incipient stage of deformation following permafrost thaw, impacting soil properties and function in Arctic landscapes undergoing transition. This dataset contains a compressed (.zip) archive of the data and R scripts used for this manuscript. The dataset includes files in .csv format, which can be accessed and processed using MS Excel or R. This archive can also be accessed on GitHub at https://github.com/Erin-Rooney/XCT-freezethaw (DOI: 10.5281/zenodo.5816355).

54 ENVIRONMENTAL SCIENCES↗

Incoherent range-based holography of far-distant objects using LIDAR

Los Alamos National Laboratory has developed an incoherent, long-range, sub-centimeter resolution (LIDAR) with which we achieve centimeter-scale reflection holography at extremely long ranges. The system consists of a pulsed laser and photon-counting receiver. This combination yields round-trip time of flight data to illuminate parts of the object of interest. The aggregation of these data for many LIDAR pulses yields a plot with a range on the X axis and reflectance on the Y axis, which we refer to as a range profile. Observing that the range profile is a projection of the reflection map of the object onto the view vector, we collect profiles from a variety of viewing angles and invert these data to form an image. We adapt imaging algorithms from the field of computer aided tomography to suit our application and present results from imaging demonstrations at a 10 km range.

Hoffmann, Mitchell P. (ORCID:0000000170551324)↗

Changes in Exercise Data Management

The suite of exercise hardware aboard the International Space Station (ISS) generates an immense amount of data. The data collected, treadmill, cycle ergometer, and resistance strength training hardware, are basic exercise parameters (time, heart rate, speed, load, etc.). The raw data are processed in the laboratory and more detailed parameters are calculated from each exercise data file. Updates recently have been made to how these valuable data are stored, adding an additional level of security, increasing accessibility, and resulting in overall increased efficiency of medical report delivery. Questions regarding exercise performance or how exercise may influence other variables of crew health frequently arise within the crew health care community. Inquiries regarding the health of the exercise hardware often need quick analysis and response to ensure the exercise system is operable on a continuous basis. Consolidating all of the exercise system data in a single repository enables a quick response to both the medical and engineering communities. A SQL server database is currently in use, and provides a secure location for all of the exercise data starting at ISS Expedition 1 to current date. The database has been structured to update derived metrics automatically, making analysis and reporting available within minutes of dropping the in-flight data into the database. Commercial tools were evaluated to help aggregate and visualize data from the SQL database. The Tableau software provides manageable interface, which has improved the laboratory’s output time of crew reports by 67%. Expansion of the SQL database, to be inclusive of additional medical requirement metrics, addition of ‘app-like’ tools for mobile visualization, and collaborative use (e.g., operational support teams, research groups, and International Partners) of the data system, is currently being explored.

Buxton, R. E.↗

Sustaining the National Geothermal Data System: Considerations for a System Wide Approach and Node Maintenance, Geothermal Resources Council 37th Annual Meeting, Las Vegas, Nevada, September 29-October 2, 2013

Since the 2009 American Recovery and Reinvestment Act the U.S. Department of Energy’s Geothermal Technologies Office has funded $33.7 million for multiple data digitization and aggregation projects focused on making vast amounts of geothermal relevant data available to industry for advancing geothermal exploration. These projects are collectively part of the National Geothermal Data System (NGDS), a distributed, networked system for maintaining, sharing, and accessing data in an effort to lower the levelized cost of electricity (LCOE). Determining “who owns” and “who maintains” the NGDS and its data nodes (repositories in the distributed system) is yet to be determined. However, the invest- ment in building and populating the NGDS has been substantial, both in terms of dollars and time; it is critical that this investment be protected by ensuring sustainability of the data, the software and systems, and the accessibility of the data. Only then, will the benefits be fully realized. To keep this operational system sustainable will require four core elements: continued serving of data and applications; maintenance of system operations; a governance structure; and an effective business model. Each of these presents a number of challenges. Data being added to the NGDS are not strictly geothermal but data considered relevant to geothermal exploration and develop- ment, including vast amounts of oil and gas and groundwater wells, among other data. These are relevant to a broader base of users. By diversifying the client base to other users and other fields, the cost of maintaining core infrastructure can be spread across an array of stakeholders and clients. It is presumed that NGDS will continue to provide free and open access to its data resources. The next-phase NGDS operation should be structured to eventually pursue revenue streams to help off-set sustainability expenses as necessary and appropriate, potentially including income from: grants and contracts (agencies, foundations, pri- vate sector), membership, fees for services (consulting, training, customization, ‘app’ development), repository services (data, services, apps, models, documents, multimedia), advertisements, fees for premier services or applications, subscriptions to value added services, licenses, contributions and donations, endow- ments, and sponsorships.

15 GEOTHERMAL ENERGY↗

Finding Atmospheric Composition (AC) Metadata

The Atmospheric Composition Portal (ACP) is an aggregator and curator of information related to remotely sensed atmospheric composition data and analysis. It uses existing tools and technologies and, where needed, enhances those capabilities to provide interoperable access, tools, and contextual guidance for scientists and value-adding organizations using remotely sensed atmospheric composition data. The initial focus is on Essential Climate Variables identified by the Global Climate Observing System CH4, CO, CO2, NO2, O3, SO2 and aerosols. This poster addresses our efforts in building the ACP Data Table, an interface to help discover and understand remotely sensed data that are related to atmospheric composition science and applications. We harvested GCMD, CWIC, GEOSS metadata catalogs using machine to machine technologies - OpenSearch, Web Services. We also manually investigated the plethora of CEOS data providers portals and other catalogs where that data might be aggregated. This poster is our experience of the excellence, variety, and challenges we encountered.Conclusions:1.The significant benefits that the major catalogs provide are their machine to machine tools like OpenSearch and Web Services rather than any GUI usability improvements due to the large amount of data in their catalog.2.There is a trend at the large catalogs towards simulating small data provider portals through advanced services. 3.Populating metadata catalogs using ISO19115 is too complex for users to do in a consistent way, difficult to parse visually or with XML libraries, and too complex for Java XML binders like CASTOR.4.The ability to search for Ids first and then for data (GCMD and ECHO) is better for machine to machine operations rather than the timeouts experienced when returning the entire metadata entry at once. 5.Metadata harvest and export activities between the major catalogs has led to a significant amount of duplication. (This is currently being addressed) 6.Most (if not all) Earth science atmospheric composition data providers store a reference to their data at GCMD.

metadata search↗

Spatial and Temporal Uncertainty of Crop Yield Aggregations

The aggregation of simulated gridded crop yields to national or regional scale requires information on temporal and spatial patterns of crop-specific harvested areas. This analysis estimates the uncertainty of simulated gridded yield time series related to the aggregation with four different harvested area data sets. We compare aggregated yield time series from the Global Gridded Crop Model Inter-comparison project for four crop types from 14 models at global, national, and regional scale to determine aggregation-driven differences in mean yields and temporal patterns as measures of uncertainty. The quantity and spatial patterns of harvested areas differ for individual crops among the four datasets applied for the aggregation. Also simulated spatial yield patterns differ among the 14 models. These differences in harvested areas and simulated yield patterns lead to differences in aggregated productivity estimates, both in mean yield and in the temporal dynamics. Among the four investigated crops, wheat yield (17% relative difference) is most affected by the uncertainty introduced by the aggregation at the global scale. The correlation of temporal patterns of global aggregated yield time series can be as low as for soybean (r = 0.28).For the majority of countries, mean relative differences of nationally aggregated yields account for10% or less. The spatial and temporal difference can be substantial higher for individual countries. Of the top-10 crop producers, aggregated national multi-annual mean relative difference of yields can be up to 67% (maize, South Africa), 43% (wheat, Pakistan), 51% (rice, Japan), and 427% (soybean, Bolivia).Correlations of differently aggregated yield time series can be as low as r = 0.56 (maize, India), r = 0.05∗Corresponding (wheat, Russia), r = 0.13 (rice, Vietnam), and r = −0.01 (soybean, Uruguay). The aggregation to sub-national scale in comparison to country scale shows that spatial uncertainties can cancel out in countries with large harvested areas per crop type. We conclude that the aggregation uncertainty can be substantial for crop productivity and production estimations in the context of food security, impact assessment, and model evaluation exercises.

Aggregation uncertainty↗

Analysis for Building Envelopes and Mechanical Systems Using 2012 CBECS Data

This report describes the aggregation and mapping of certain building characteristics data available in the most recent Commercial Building Energy Consumption Survey (CBECS) (DOE EIA 2012) to describe most typical building construction practices. This report provides summary data for potential use in the support of modifications to the Pacific Northwest National Laboratory’s commercial building prototypes used for building energy code analysis. Specifically, this report outlines findings and most typical design choices for certain building envelope and heating, ventilating, and air-conditioning (HVAC) system choices.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Artificial Intelligence for Data Center Operations (AIOps): Cooperative Research and Development (Final Report)

High performance computing data centers will increasingly need to rely on automation to keep pace with exascale growth in compute capability and to manage and optimize the data center environment and facility resources. Artificial intelligence and machine learning approaches provide the means to improve HPC data center operational efficiency, by learning historical trends and training models to operate on real-time data collected from both IT and facilities sources. NREL has developed methods of real-time collection, aggregation and streaming of these data in the ESIF HPC Data Center and has collected a significant dataset of relevant metrics across computer systems, racks, environmental, building and utility sources for research into various predictive analytics problems. HPE's Advanced Technology Group (ATG) is doing comprehensive research into exascale monitoring and management for High Performance Computing (HPC) systems (hereinafter HPE's Data Monitoring/ Management Technology). NREL and HPE will collaborate to add Artificial Intelligence (AI) to NREL's real-time data collection/ aggregation/ streaming system and HPE's Data Monitoring/ Management System, with the goal of improving the operational efficiency of NREL's Energy Systems Integration Facility (ESIF) HPC Data Center through data analytics on both historical and real-time data from IT systems and facilities operations. This collaboration will consist of efforts in Data Management, Data Analytics, and AI/ML Optimization for both manual and autonomous intervention in data center operations. This will be a multi-year, multi-staged effort with a goal towards building capabilities for an Advanced Smart Facility, and demonstration of these techniques in the NREL ESIF HPC Data Center.

97 MATHEMATICS AND COMPUTING↗

Changes in Exercise Data Management

The suite of exercise hardware aboard the International Space Station (ISS) generates an immense amount of data. The data collected from the treadmill, cycle ergometer, and resistance strength training hardware are basic exercise parameters (time, heart rate, speed, load, etc.). The raw data are post processed in the laboratory and more detailed parameters are calculated from each exercise data file. Updates have recently been made to how this valuable data are stored, adding an additional level of data security, increasing data accessibility, and resulting in overall increased efficiency of medical report delivery. Questions regarding exercise performance or how exercise may influence other variables of crew health frequently arise within the crew health care community. Inquiries over the health of the exercise hardware often need quick analysis and response to ensure the exercise system is operable on a continuous basis. Consolidating all of the exercise system data in a single repository enables a quick response to both the medical and engineering communities. A SQL server database is currently in use, and provides a secure location for all of the exercise data starting at ISS Expedition 1 - current day. The database has been structured to update derived metrics automatically, making analysis and reporting available within minutes of dropping the inflight data it into the database. Commercial tools were evaluated to help aggregate and visualize data from the SQL database. The Tableau software provides manageable interface, which has improved the laboratory's output time of crew reports by 67%. Expansion of the SQL database to be inclusive of additional medical requirement metrics, addition of 'app-like' tools for mobile visualization, and collaborative use (e.g. operational support teams, research groups, and International Partners) of the data system is currently being explored.

Buxton, R. E.↗

Usage based indicators to assess the impact of scholarly works: architecture and method

Although recording of usage data is common in scholarly information services, its exploitation for the creation of value-added services remains limited due to concerns regarding, among others, user privacy, data validity, and the lack of accepted standards for the representation, sharing and aggregation of usage data. A technical, standards-based architecture for sharing usage information is presented. In this architecture, OpenURL-compliant linking servers aggregate usage information of a specific user community as it navigates the distributed information environment that it has access to. This usage information is made OAI-PMH harvestable so that usage information exposed by many linking servers can be aggregated to facilitate the creation of value-added services with a reach beyond that of a single community or a single information service.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Method of recommending items to a user based on user interest

Although recording of usage data is common in scholarly information services, its exploitation for the creation of value-added services remains limited due to concerns regarding, among others, user privacy, data validity, and the lack of accepted standards for the representation, sharing and aggregation of usage data. A technical, standards-based architecture for sharing usage information is presented. In this architecture, OpenURL-compliant linking servers aggregate usage information of a specific user community as it navigates the distributed information environment that it has access to. This usage information is made OAI-PMH harvestable so that usage information exposed by many linking servers can be aggregated to facilitate the creation of value-added services with a reach beyond that of a single community or a single information service.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Aggregating available soil water holding capacity data for crop yield models

The total amount of water available to plants that is held against gravity in a soil is usually estimated as the amount present at -0.03 MPa average water potential minus the amount present at -1.5 MPa water potential. This value, designated available water-holding capacity (AWHC), is a very important soil characteristic that is strongly and positively correlated to the inherent productivity of soils. In various applications, including assessing soil moisture status over large areas, it is necessary to group soil types or series as to their productivity. Current methods to classify AWHC of soils consider only total capacity of soil profiles and thus may group together soils which differ greatly in AWHC as a function of depth in the profile. A general approach for evaluating quantitatively the multidimensional nature of AWHC in soils is described. Data for 902 soil profiles, representing 184 soil series, in Indiana were obtained from the Soil Characterization Laboratory at Purdue University. The AWHC for each of ten 150-mm layers in each soil was established, based on soil texture and parent material. A multivariate clustering procedure was used to classify each soil profile into one of 4, 8, or 12 classes based upon ten-dimensional AWHC values. The optimum number of classes depends on the range of AWHC in the population of oil profiles analyzed and on the sensitivity of a crop to differences in distribution of water within the soil profile.

Seubert, C. E.↗