Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

HAPI: An API Standard for Accessing Heliophysics Time Series Data

Heliophysics data analysis often involves combining diverse science measurements, many of them captured as time series. Although there are now only a few commonly used data file formats, the diversity in mechanisms for automated access to and aggregation of such data holdings can make analysis that requires intercomparison of data from multiple data providers difficult. The Heliophysics Application Programmer's Interface (HAPI) is a recently developed standard for accessing distributed time series data to increase interoperability. The HAPI specification is based on the common elements of existing data services, and it standardizes the two main parts of a data service: the request interface and the response data structures. The interface is based on the REpresentational State Transfer (REST) or RESTful architecture style, and the HAPI specification defines five required REST endpoints. Data are returned via a streaming format that hides file boundaries; the metadata is detailed enough for the content to be scientifically useful, e.g., plotted with appropriate axes layout, units, and labels. Multiple mature HAPI-related open-source projects offer server-side implementation tools and client-side libraries for reading HAPI data in multiple languages (IDL, Java, MATLAB, and Python). Multiple data providers in the US and Europe have added HAPI access alongside their existing interfaces. Based on this experience, data can be served via HAPI with little or no information loss compared to similar existing web interfaces. Finally, HAPI has been recommended as a COSPAR standard for time series data delivery.

Robert S. Weigel↗

Model choice considerations and information integration using analytical hierarchy process

Using the theory of information-gap for decision-making under severe uncertainty, it has been shown that model output compared to experimental data contains irrevocable trade-offs between fidelity-to-data, robustness-to-uncertainty and confidence-in-prediction. We illustrate a strategy for information integration by gathering and aggregating all available data, knowledge, theory, experience, similar applications. Such integration of information becomes important when the physics is difficult to model, when observational data are sparse or difficult to measure, or both. To aggregate the available information, we take an inference perspective. Models are not rejected, nor wasted, but can be integrated into a final result. We show an example of information integration using Saaty's Analytic Hierarchy Process (AHP), integrating theory, simulation output and experimental data. We used expert elicitation to determine weights for two models and two experimental data sets, by forming pair-wise comparisons between model output and experimental data. In this way we transform epistemic and/or statistical strength from one field of study into another branch of physical application. The price to pay for utilizing all available knowledge is that inferences drawn for the integrated information must be accounted for and the costs can be considerable. Focusing on inferences and inference uncertainty (IU) is one way to understand complex information.

99 GENERAL AND MISCELLANEOUS↗

Variability and Diversity Load Model Tool [SWR-20-03]

The motivation for the development of this tool and the underlying algorithms and methods was to enable the development of high-temporal resolution, realistic time-series data for quasi-static time-series (QSTS) analysis of distribution systems. Often, aggregated load profile data for a distribution circuit is available (e.g. feeder loading data collected via SCADA at the utility substation) and, while this data is typically accurate it masks the considerable variability of the 100’s or 1000’s of individual loads connected on the circuit. This tool was developed to model both the increased variability expected for these individual loads (e.g. the load of a single distribution transformer connected to 8-12 houses) and the expected diversity between loads on the circuit. It is important to note that the difference in variability and diversity, in the context of this tool, is that variability modeling only adds representative variability due to disaggregated load characteristics (e.g. the presence in the load profile of loads turning off and on like an air conditioner/oven) while the average energy profile remains the same as the user supplied power profile. Diversity modeling generates multiple individual load profiles which, in aggregate, sum to the user supplied power profile. Diversity is effectively variability in the energy usage over longer periods of time than seen in the variability model. Put another way, variability modeling supplies the expected variability due to the operation of various end-use loads and diversity modeling supplies the usage differences due to human behavior, schedules, etc. This load modeling tool was developed for use in generating data for distribution systems. Modeling is summarized by two major functions: 1) taking low resolution load profiles and adding intra-seconds variability onto the profiles, and 2) taking a user supplied load profile and distribution factors and adding both diversity and variability to the user supplied profile.

Zhu, Xiangqi↗

The Industry Energy Tool (IET): Documentation

Studies of industrial energy use typically mention the sector's heterogeneity and complexity as modelling barriers. One of the more significant impediments to analyzing and building models of U.S. industrial energy use is the lack of current, disaggregated data. The Industry Energy Tool (IET) and its foundational data addresses some of these barriers. The IET uses county-level energy data with stock turnover, energy efficiency, fuel switching, and emissions functionalities to calculate energy use and emissions from agriculture, construction, mining, and manufacturing industries from 2014 to 2050. The IET is an open-source, transparent, and flexible tool for exploring industrial sector energy and emissions scenarios. The IET takes a simplified approach to developing scenario projections of energy and emissions by allowing users to modify pre-populated inputs for achieved energy efficiency, rate of equipment stock turnover, and future penetration renewable electricity. In addition to its transparency and flexibility, the strengths of the IET lie in its geographically- and operationally-disaggregated data set. The IET's foundational data is built on energy data first calculated at the facility- and technology-level for many of the largest energy-using industries. These data are then aggregated to the county and end-use level with other publicly-available data for use in scenarios estimated by the IET.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Is disaggregation the holy grail of energy efficiency? The case of electricity

This paper aims to address two timely energy problems. First, significant low-cost energy reductions can be made in the residential and commercial sectors, but these savings have not been achievable to date. Second, billions of dollars are being spent to install smart meters, yet the energy saving and financial benefits of this infrastructure - without careful consideration of the human element - will not reach its full potential. We believe that we can address these problems by strategically marrying them, using disaggregation. Disaggregation refers to a set of statistical approaches for extracting end-use and/or appliance level data from an aggregate, or whole-building, energy signal. In this paper, we explain how appliance level data affords numerous benefits, and why using the algorithms in conjunction with smart meters is the most cost-effective and scalable solution for getting this data. We review disaggregation algorithms and their requirements, and evaluate the extent to which smart meters can meet those requirements. Research, technology, and policy recommendations are also outlined. (C) 2012 Elsevier Ltd. All rights reserved.

Armel, KC↗

Oak Ridge National Laboratory EAGLE-I TM : Modeling Electric Utility County Customers for Situational Awareness

During natural hazard events (hurricanes, wildfires, earthquakes, etc.) and recent man-made events (e.g., cyber attacks), the exchange of near real-time, spatially refined data within the response community is critical. The EAGLE-I$^{TM}$ platform is one tool that facilitates this data for decision makers within the energy sector. While much information can be collected and integrated into the system directly, other pertinent data must be augmented by other derived data products to enhance the information and allow for a consistent evaluation of on-the-ground conditions. One such data set that requires the addition of other derived data is the electric utility customer outage data that is aggregated to the county level within the EAGLE-I application. Without a county customer data set, outages can only be compared on total counts, which gives greater importance to higher population outages. Including an electric utility customer data set at the county level allows for these outage counts to be converted to percent outages and brings a consistent classification of outages and equal importance to all outages. To achieve this, several available data sets were combined and spatial disaggregation techniques were employed to model customer estimates at the county scale. This paper presents the approach to produce this data for the United States and lessons learned from working with these disparate data sets. Data validation is provided, where possible, and limitations of the model and possible improvements are discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Common Cause Failure Modeling: Aerospace Versus Nuclear

Aggregate nuclear plant failure data is used to produce generic common-cause factors that are specifically for use in the common-cause failure models of NUREG/CR-5485. Furthermore, the models presented in NUREG/CR-5485 are specifically designed to incorporate two significantly distinct assumptions about the methods of surveillance testing from whence this aggregate failure data came. What are the implications of using these NUREG generic factors to model the common-cause failures of aerospace systems? Herein, the implications of using the NUREG generic factors in the modeling of aerospace systems are investigated in detail and strong recommendations for modeling the common-cause failures of aerospace systems are given.

Stott, James E.↗

Why We Do What We Do: Data Reuse, Open Access, and Privacy in Data Management at the Life Sciences Data Archive

As custodian of the unique and irreplaceable collections of human subject research data generated by the Human Research Program and its predecessors throughout the agency’s history, the Life Sciences Data Archive (LSDA) is charged with protecting participants’ privacy and implementing their consent decisions as it provides retrospective data for use in new studies. This active, stewardship-focused approach to data management and preservation shapes the products that LSDA provides to researchers and the responsibilities of researchers in using the data and publishing their results. This presentation reviews how federal and agency mandates shape LSDA’s data management procedures and expectations for researchers. Topics covered will include LSDA’s movement towards implementation of the FAIR (Findable, Accessible, Interoperable, Reusable) principles and how the archive’s evolving data management practices support FAIR-ness; collaboration between LSDA and the Lifetime Surveillance of Astronaut Health (LSAH) project (the repository of astronaut medical data); LSDA’s response to the challenges of performing its stewardship role and maintaining trust given the public profiles of the subjects whose data it preserves; and the ever-increasing challenges to expectations of subject privacy stemming from the growing power and ubiquity of of data analysis and aggregation tools.

Data↗

Why We Do What We Do: Data Reuse, Open Access and Privacy in Data Management at the Life Sciences Data Archive

As custodians of the unique and irreplaceable collections of human subject research data generated by the Human Research Program and its predecessors throughout the agency’s history, the Life Sciences Data Archive (LSDA) is charged with protecting participants’ privacy and implementing their consent decisions as it provides retrospective data for use in new studies. This active, stewardship-focused approach to data management and preservation shapes the products that LSDA provides to researchers and the responsibilities of researchers in using the data and publishing their results. This presentation reviews how federal and agency mandates shape LSDA’s data management procedures and expectations for researchers. Topics covered will include LSDA’s movement towards implementation of the FAIR (Findable, Accessible, Interoperable, Reusable) principles and the archive’s evolving data management practices; collaboration between LSDA and the Lifetime Surveillance of Astronaut Health (LSAH) project (the repository of astronaut medical data); LSDA’s response to the challenges of performing its stewardship role and maintaining trust given the public profiles of the subjects whose data it preserves; and the ever-increasing challenges to expectations of subject privacy stemming from the growing power and ubiquity of data analysis and aggregation tools.

data management↗

Microbial interactions in the anaerobic oxidation of methane: model simulations constrained by process rates and activity patterns: Process modelling of anaerobic methane oxidation

Proposed syntrophic interactions between the archaeal and bacterial cells mediating anaerobic oxidation of methane coupled with sulfate reduction include electron transfer through (1) the exchange of H2 or small organic molecules between methane-oxidizing archaea and sulfate-reducing bacteria, (2) the delivery of disulfide from methane-oxidizing archaea to bacteria for disproportionation and (3) direct interspecies electron transfer. Each of these mechanisms was implemented in a reactive transport model. The simulated activities across different arrangements of archaeal and bacterial cells and aggregate sizes were compared to empirical data for AOM rates and intra-aggregate spatial patterns of cell-specific anabolic activity determined by FISH-nanoSIMS. Additionally, simulation results showed that rates for chemical diffusion by mechanism (1) were limited by the build-up of metabolites, while mechanisms (2) and (3) yielded cell specific rates and archaeal activity distributions that were consistent with observations from single cell resolved FISH-nanoSIMS analyses. The novel integration of both intra-aggregate and environmental data provided powerful constraints on the model results, but the similarities in model outcomes for mechanisms (2) and (3) highlight the need for additional observational data (e.g. genomic or physiological) on electron transfer and metabolic functioning of these globally important methanotrophic consortia.

36 MATERIALS SCIENCE↗

Electrical Resistivity Correlation to Vadose Zone Sediment and Pore-Water Composition for the BC Cribs and Trenches Area

This technical report documents the results of geochemical and soil resistivity characterization of sediment obtained from four boreholes drilled in the BC Cribs and Trench area. Vadose zone sediment samples were obtained at a frequency of about every 2.5 ft from approximately 5 ft bgs to borehole total depth. In total, 505 grab samples and 39 six-inch long cores were obtained for characterization. The pore-water chemical composition data, laboratory-scale soil resistivity and other ancillary physical and hydrologic measurements and analyses described in this report are designed to provide a crucial link between direct measurements on sediments and the surface-based electrical-resistivity information obtained via field surveys. A second goal of the sediment characterization was to measure the total and water-leachable concentrations of key contaminants of concern as a function of depth and distance from the footprints of inactive disposal facilities. The total and water-leachable concentrations of key contaminants will be used to update contaminant distribution conceptual models and to provide more data for improving base-line risk predictions and remedial alternative selections. The ERC “ground truthing” exercise for the individual boreholes showed mixed results. In general, the high concentrations of dissolved salts in the pore waters of sediments from C5923, C5924 and C4191 produced a low resistivity “target” in the processed resistivity field surveys, and variability could be seen in the resistivity data that could relate to the variability in pore- water concentrations but the correlations (regression R2 were mediocre ranging from 0.2 to 0.7 at best; where perfect correlation is 1.0). The field-based geophysical data also seemed to suffer from a sort of vertigo, where looking down from the ground surface, the target (e.g., maximum pore-water salt concentration) depth was difficult to resolve. The best correlations between the field electrical resistivity surveys and borehole pore water data sets were obtained when focusing on areal extent of the salt plume. Lateral resolution of the geophysical field data is best conducted by comparing an aggregated set of geophysical data on all boreholes together. When assembling the pore-water data for all four boreholes in an aerial view, the field ERC data produce a reasonable aerial picture of where high salt plumes exist below the BC Cribs and Trenches area. Future work that relies on more laboratory soil resistivity and incorporation of other field data (spectral gamma, neutron moisture and soil density logs) and physical and hydraulic measurements on samples obtained from the boreholes will used develop a more detailed petrophysical model of the sediments below BC Cribs and Trenches. This more detailed model can be used as a more realistic “earth model” in the inversion process to better manipulate the raw field survey data. It is also recommended that one more borehole be drilled after a thorough vetting of the current data with geophysics experts and other Hanford stakeholder to optimize where to place the borehole, what electrical and other geophysical surveys should be conducted , where to take sediment samples and what parameters should be measured on the sediments to attempt one more “ground truthing” exercise.

54 ENVIRONMENTAL SCIENCES↗

Detection of Alzheimer's amyloid beta aggregation by capturing molecular trails of individual assemblies

Assembly of Amyloid beta (A{beta}) peptides, in particular A{beta}-42 is central to the formation of the amyloid plaques associated with neuro-pathologies such as Alzheimer's disease (AD). Molecular assembly of individual A{beta}-42 species was observed using a simple fluorescence microscope. From the molecular movements (aka Brownian motion) of the individual peptide assemblies, we calculated a temporal evolution of the hydrodynamic radius (R{sub H}) of the peptide at physiological temperature and pH. The results clearly show a direct relationship between R{sub H} of A{beta}-42 and incubation period, corresponding to the previously reported peptide's aggregation kinetics. The data correlates highly with in solution-based label-free electrochemical detection of the peptide's aggregation, and A{beta}-42 deposited on a solid surface and analysed using atomic force microscopy (AFM). To the best of our knowledge, this is the first analysis and characterisation of A{beta} aggregation based on capturing molecular trails of individual assemblies. The technique enables both real-time observation and a semi-quantitative distribution profile of the various stages of A{beta} assembly, at microM peptide concentration. Our method is a promising candidate for real-time observation and analysis of the effect of other pathologically-relevant molecules such as metal ions on pathways to A{beta} oligomerisation and aggregation. The method is also a promising screening tool for AD therapeutics that target A{beta} assembly.

60 APPLIED LIFE SCIENCES↗

Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments

We present a monitoring system for large-scale parallel and distributed computing environments that allows to trade-off accuracy in a tunable fashion to gain scalability without compromising fidelity. The approach relies on classifying each gathered monitoring metric based on individual needs and on aggregating messages containing classes of individual monitoring metrics using a tree-based overlay network. The MRNet-based prototype is able to significantly reduce the amount of gathered and stored monitoring data, e.g., by a factor of ~56 in comparison to the Ganglia distributed monitoring system. A simple scaling study reveals, however, that further efforts are needed in reducing the amount of data to monitor future-generation extreme-scale systems with up to 1,000,000 nodes. The implemented solution did not had a measurable performance impact as the 32-node test system did not produce enough monitoring data to interfere with running applications.

97 MATHEMATICS AND COMPUTING↗

Securely Aggregated Coded Matrix Inversion

Coded computing is a method for mitigating straggling workers in a centralized computing network, by using erasure-coding techniques. Federated learning is a decentralized model for training data distributed across client devices. In this work we propose approximating the inverse of an aggregated data matrix, where the data is generated by clients; similar to the federated learning paradigm, while also being resilient to stragglers. To do so, we propose a coded computing method based on gradient coding. We modify this method so that the coordinator does not access the local data at any point; while the clients access the aggregated matrix in order to complete their tasks. Here, the network we consider is not centrally administrated, and the communications which take place are secure against potential eavesdroppers.

97 MATHEMATICS AND COMPUTING↗

Data Intensive Architecture for Scalable Cyber Analytics

Cyber analysts are tasked with the identification and mitigation of network exploits and threats. These compromises are difficult to identify due to the characteristics of cyber communication, the volume of traffic, and the duration of possible attack. In this paper, we describe a prototype implementation designed to provide cyber analysts an environment where they can interactively explore a month’s worth of cyber security data. This prototype utilized On-Line Analytical Processing (OLAP) techniques to present a data cube to the analysts. The cube provides a summary of the data, allowing trends to be easily identified as well as the ability to easily pull up the original records comprising an event of interest. The cube was built using SQL Server Analysis Services (SSAS), with the interface to the cube provided by Tableau. This software infrastructure was supported by a novel hardware architecture comprising a Netezza TwinFin® for the underlying data warehouse and a cube server with a FusionIO drive hosting the data cube. We evaluated this environment on a month’s worth of artificial, but realistic, data using multiple queries provided by our cyber analysts. As our results indicate, OLAP technology has progressed to the point where it is in a unique position to provide novel insights to cyber analysts, as long as it is supported by an appropriate data intensive architecture.

Olsen, Bryan K.↗

Label Assist: Personalized Travel Models for Longitudinal Data Collection

Understanding travel behavior is crucial to transportation decarbonization. OpenPATH is an open-source mobility platform which collects and analyzes human travel behavior at the individual level. The mobile application passively senses trips and prompts users to label them. However, users find the labeling process burdensome; less than half the trips are typically labeled, making much of the data unusable in aggregate analyses of mobility patterns. Prior work has addressed the response fatigue challenge through automated mode inference using sensor data, but sensors cannot capture all aspects of travel behavior. We explore an alternative approach in which we leverage prior user input to predict travel choices in novel trips. We first explore trip clustering methods and develop a novel two-step pipeline using DBSCAN and SVMs to extract realistic geospatial clusters. We then propose two strategies to predict trip labels: (i) clustering trips and extrapolating labels for similar trips, and (ii) random forest classification. The random forest approach is able to achieve - $70-80% accuracy (purpose: 72%, mode: 79%, replaced mode: 81%). These novel approaches to trip classification allow us to increase the rate of user labeling by suggesting predicted labels to be verified by the user. Unlabeled trips can also contribute to aggregate analyses, using label predictions and their associated confidences as a substitute. While there exist other travel survey apps with the ability to infer travel choices, to our knowledge, this is the first paper to describe such a supervised system and rigorously evaluate it.

ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗