Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County

The dataset contains hourly Anthropogenic heat (AH) from buildings in Los Angeles County, based on weather data from 2018. The hourly AH is aggregated at three spatial resolutions: 450m x 450m grid, 12km x 12km grid, and census tract. The AH is broken down into three components: building envelope surface convection, heating, ventilation, and air conditioning (HVAC) system heat release, and zone exfiltration and exhaust air heat loss. The dataset is created with the physics-based EnergyPlus building energy models to calculate individual buildings' AH considering WRF-UCM simulated microclimate conditions. Please refer to the paper "A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County" for more information about the data generation workflow and the data validation procedure. The data set contains two folders: the "output_data" folder holds the simulation results (EP_output and EP_output_csv), building metadata (building_metadata.geojson and building_metadata.csv), aggregated heat emission and energy consumption time-series data (hourly_heat_energy), and geographical data (geo_data) associated with the GEOID referenced in heat and energy consumption data. The "input_data" folder contains the raw data used to generate files in the "output_data" folder as well as data sets used in the validation. The code repository (https://github.com/IMMM-SFA/xu_etal_2022_sdata) holds the processing scripts for data curation, validation, and visualization.

Energy↗

Renewable Electricity Procurement Options Data (RE-POD)

The Renewable Electricity Procurement Options Data (RE-POD) is an aggregated dataset meant to help local jurisdictions and utility customers within those jurisdictions understand the options that may be available to them to procure renewable electricity or renewable energy credits to meet energy goals. This data is part of a suite of state and local energy profile data available at the "State and Local Energy Profile Data Suite" link below and builds on Cities-LEAP energy modeling, available at the "EERE Cities-LEAP Page" link below. Examples of how to use the data to inform energy planning can be found at the "Example Uses" link below.

Array↗

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on a quality-aware adaptive temporal data selection and reconstruction problem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we develop DeepSample , an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

Zhang, Yang↗

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

97 MATHEMATICS AND COMPUTING↗

Differentially Private Adaptive Noise Injection (DP-ANI) v1.0

Location data is collected from users continuously to understand their mobility patterns. Releasing the user trajectories may compromise user privacy. Therefore, the general practice is to release aggregated location datasets. However, private information may still be inferred from an aggregated version of location trajectories. Differential privacy (DP) protects the query output against inference attacks regardless of background knowledge. This software implements a differential privacy-based privacy model that protects the user's origins and destinations from being inferred from aggregated mobility datasets. This is achieved by injecting Planar Laplace noise to the user origin and destination GPS points. The noisy GPS points are then transformed into a link representation using a link-matching algorithm. Finally, the link trajectories form an aggregated mobility network. The injected noise level is selected using the Sparse Vector Mechanism. This DP selection mechanism considers the link density of the location and the functional category of the localized links. Compared to the different baseline models, including a k-anonymity method, our differential privacy-based aggregation model offers query responses that are close to the raw data in terms of aggregate statistics at both the network and trajectory-levels with maximum 9% deviation from the baseline in terms of network length.

Peisert, Sean [Lawrence Berkeley National Laborato↗

Skluma: An extensible metadata extraction pipeline for disorganized data

To mitigate the effects of high-velocity data expansion and to automate the organization of filesystems and data repositories, we have developed Skluma-a system that automatically processes a target filesystem or repository, extracts content-and context-based metadata, and organizes extracted metadata for subsequent use. Skluma is able to extract diverse metadata, including aggregate values derived from embedded structured data; named entities and latent topics buried within free-text documents; and content encoded in images. Skluma implements an overarching probabilistic pipeline to extract increasingly specific metadata from files. It applies machine learning methods to determine file types, dynamically prioritizes and then executes a suite of metadata extractors, and explores contextual metadata based on relationships among files. The derived metadata, represented in JSON, describes probabilistic knowledge of each file that may be subsequently used for discovery or organization. Skluma's architecture enables it to be deployed both locally and used as an on-demand, cloud-hosted service to create and execute dynamic extraction workflows on massive numbers of files. It is modular and extensible-allowing users to contribute their own specialized metadata extractors. Thus far we have tested Skluma on local filesystems, remote FTP-accessible servers, and publicly-accessible Globus endpoints. We have demonstrated its efficacy by applying it to a scientific environmental data repository of more than 500,000 files. We show that we can extract metadata from those files with modest cloud costs in a few hours.

data swamp↗

LTE Electrolyzer Data Collection

The goal for NREL is to collect, develop and publish performance metrics relative to low temperature electrolyzer installations. This will be done through the development of: Secure storage solution to house the collection of data from multiple projects Standardization of data to be collected and analyzed. This will be done using data templates developed with the help of partners involved with electrolyzer installations. Analysis that produces metrics of interest for all stakeholders Aggregation of results from multiple projects to view industry progress as a whole Publication of aggregated results in the form of composite data products (CDPs) Collaboration with Idaho National Lab and their work with high temperature electrolyzer installations will enable efficient use of storage and analysis tools.

data↗

Energy DataBus (Fact Sheet)

NREL has developed the Energy DataBus, an open-sourced software that collects massive amounts of energy-related data at second-to-second intervals; stores it in a massive, scalable database; and turns it into useful information.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Amphiphile Organization in Organic Solutions: An Alternative Explanation for Small-Angle X-ray Scattering Features in Malonamide/Alkane Mixtures

In this work, the role of different intermolecular interactions in the aggregation of amphiphiles in an organic solvent is studied for systems of relevance to liquid-liquid extraction (LLE), a chemical process used to selectively recover metals from complex mixtures. Of specific interest is the role, or lack thereof, of hydrogen bonding, which is often assumed to be a main driver of the organic phase structural organization that has been linked to separation efficacy. Toward that end, a series of malonamide extractants in n-dodecane have been studied in the absence of any extracted aqueous solutes, including water. The series of extractants includes N,N'-dimethyl-N,N'-dibutyltetradecylmalonamide (DMDBTDMA), two of its homologs, and N,N'-dimethyl-N,N'-dioctylhexylethoxymalonamide (DMDOHEMA). This simplified model LLE system enables systematic investigation of the role of dipole-dipole and alkyl tail steric interactions in amphiphile aggregation. Small-angle X-ray scattering (SAXS) profiles computed from molecular dynamics trajectories are in good agreement with experimental SAXS data. Molecular dynamics simulations show that malonamide aggregation results from dipole-driven self-association and lacks characteristic aggregate sizes. Mid-q correlation peaks in the SAXS profiles emerge at high concentration for each malonamide. In those densely packed solutions, the correlation peaks are observed to result from alkyl tail-induced spacing between electron-rich polar head groups, with peak positions determined by the different alkyl tail lengths present in the malonamide molecule. This explanation of the SAXS correlation peaks contrasts with the prevailing literature, which attributes mesoscale features observed in small-angle scattering to the formation of microemulsions. Instead, this work finds that these features are present in the absence of water or any reverse micellar organization of the malonamides. As such, molecular-scale malonamide self-association and packing, rather than microemulsion-based colloidal-scale descriptions, is a more appropriate framework for these LLE systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Vibration-Based Techniques for Concrete Structural Health Monitoring. Light Water Reactor Sustainability Program report

The objectives of this ongoing research project focus on health monitoring and data analytics of concrete slabs containing reactive aggregates and thus subjected to degradation due to alkali-silica reaction (ASR). A controlled concrete slab with four pockets of reactive aggregates (pure silica, and reactive aggregates from three different quarries) was cast at Vanderbilt University and cured in representative conditions to accelerate degradation due to ASR. A set of four concrete samples were also cast and cured at the University of Alabama for ASR testing. Of these four samples, two slabs contained reactive aggregates while the other two had the non-reactive aggregate counterparts mixed throughout the samples. Four additional slabs were cast and cured at the University of Nebraska- Lincoln. Of these four samples; one contained non-reactive aggregates and no reinforcement, one contained reactive aggregates and no reinforcement, one contained reactive aggregates and rebar reinforcement in one direction, and the last contained reactive aggregates and reinforcement in two directions. Single frequency and dual frequency tests were used on these slabs to locate ASR damage within the specimens containing reactive aggregates. The vibration tests were conducted with different sensor and actuator configurations to facilitate damage detection and localization. The results from the data analysis of the dynamic testing on concrete slabs are discussed in detail in this report. Results for damage localization are dependent on multiple parameters used in the tests. This report investigates different methods of data fusion including voting, raw averaging, damage index averaging, and Bayesian fusion.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Using Models to Provide Predicted Ranges for Building-Human Interfaces: Preprint

Most building energy consumption dashboards provide only a snapshot of building performance; whereas some provide more detailed historic data with which to compare current usage. This paper will discuss the Building Agent(tm) platform, which has been developed and deployed in a campus setting at the National Renewable Energy Laboratory as part of an effort to maintain the aggressive energyperformance achieved in newly constructed office buildings and laboratories. The Building Agent(tm) provides aggregated and coherent access to building data, including electric energy, thermal energy, temperatures, humidity, and lighting levels, and occupant feedback, which are displayed in various manners for visitors, building occupants, facility managers, and researchers. This paper focuseson the development of visualizations for facility managers, or an energy performance assurance role, where metered data are used to generate models that provide live predicted ranges of building performance by end use. These predicted ranges provide simple, visual context for displayed performance data without requiring users to also assess historical information or trends. Several energymodelling techniques were explored including static lookup-based performance targets, reduced-order models derived from historical data using main effect variables such as solar radiance for lighting performance, and integrated energy models using a whole-building energy simulation program.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Informing energy consumption uncertainty: an analysis of energy data revisions

Quality energy consumption data are important for many types of analysis, and global data sets estimate trends of county level energy consumption, derived from country reported data and regional reports. We present a novel basis for informing uncertainty in energy data by quantifying the changes in reported energy consumption as countries update their previously reported data. We use 17 editions of the British Petroleum World Energy Statistics (2001-2017) to evaluate how reported energy consumption is revised over time in aggregate coal, oil, and natural gas consumption data. We find that 70% of non-zero data points are adjusted by an average of 1.3% of a country’s total fossil fuel use in the year after their first publication. Earlier data points are revised less often, but almost half of historical trends contain some revisions in later years. The size and rate of data revisions vary over countries and fuels: coal data points have larger, less frequent revisions while oil data points have smaller more frequent revisions, with natural gas in between. A K-means cluster analysis was performed to group together countries with similar revision patterns. These groups span income, economy classification, OECD membership, and regions. Standard country groupings, therefore, do not predict the extent to which a country's energy data has undergone revisions in the past.

Hoesly, Rachel M. (ORCID:0000000309652729)↗

SOMA: Observability, monitoring, and in situ analytics for exascale applications

With the rise of exascale systems and large, data-centric workflows, the need to observe and analyze high performance computing (HPC) applications during their execution is becoming increasingly important. HPC applications are typically not designed with online monitoring in mind, therefore, the observability challenge lies in being able to access and analyze interesting events with low overhead while seamlessly integrating such capabilities into existing and new applications. We explore how our service-based observation, monitoring, and analytics (SOMA) approach to collecting and aggregating both application-specific diagnostic data and performance data addresses these needs. Furthermore, we present our SOMA framework and demonstrate its viability with LULESH, a hydrodynamics proxy application. Then we focus on Astaroth, a multi-GPU library for stencil computations, highlighting the integration of the TAU and APEX performance tools and SOMA for application and performance data monitoring.

97 MATHEMATICS AND COMPUTING↗

Generating porosity during olivine carbonation via dissolution channels and expansion cracks

Abstract. The olivine carbonation reaction, in which carbon dioxide is chemically incorporated to form carbonate, is central to the emerging carbon sequestration method using ultramafic rocks. The rate of this retrograde metamorphic reaction is controlled, in part, by the available reactive surface area: as the solid volume increases during carbonation, the feasibility of this method ultimately depends on the maintenance of porosity and the creation of new reactive surfaces. We conducted in situ dynamic X-ray microtomography and nanotomography experiments to image and quantify the porosity generation during olivine carbonation. We designed a sample setup that included a thick-walled cup (made of porous olivine aggregates with a mean grain size of either ∼ 5 or ∼ 80 µm) filled with loose olivine sands with grain sizes of 100–500 µm. The whole sample assembly was reacted with a NaHCO 3 aqueous solution at 200 °C, under a constant confining pressure of 13 MPa and a pore pressure of 10 MPa. Using synchrotron-based X-ray microtomography, the three-dimensional (3-D) pore structure evolution of the carbonating olivine cup was documented until the olivine aggregates became disintegrated. The dynamic microtomography data show a volume reduction in olivine at the beginning of the reaction, indicating a vigorous dissolution process consistent with the disequilibrium reaction kinetics. In the olivine cup with a grain size of ∼ 80 µm (coarse-grained cup), dissolution planes developed within 30 h, before any precipitation was observed. In the experiment with the olivine cup of ∼ 5 µm mean grain size (fine-grained cup), idiomorphic magnesite crystals were observed on the surface of the olivine sands. The magnesite shows a near-constant growth throughout the experiment, suggesting that the reaction is self-sustained. Large fractures were generated as the reaction proceeded and eventually disintegrated the aggregate after 140 h. Detailed analysis show that these are expansion cracks caused by the volume mismatch in the cup walls, between the expanding interior and the near-surface which keeps a nearly constant volume. Nanotomography images of the reacted olivine cup reveal pervasive etch pits and wormholes in the olivine grains. We interpret this perforation of the solids to provide continuous fluid access, which is likely key to the complete carbonation observed in nature. Reactions proceeding through the formation of nano- to micron-scale dissolution channels provide a viable microscale mechanism in carbon sequestration practices. For the natural peridotite carbonation, a coupled mechanism of dissolution and reaction-induced fracturing should account for the observed self-sustainability of the reaction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Open Energy Data Initiative (OEDI) FY22-24 (Final Technical Report)

Final technical report for the Open Energy Data Initiative (OEDI) project covering fiscal years FY22 through FY24. The DOE Open Energy Data Initiative (OEDI) is a partnership between the National Renewable Energy Laboratory (NREL), the U.S. Department of Energy (DOE), and major cloud providers including Amazon, Microsoft, and Google to provide universal access to big data in the cloud. At the heart of OEDI is a centralized repository of high-value energy research datasets aggregated from the U.S. Department of Energy's Program Offices, National Laboratories and other collaborators. It aggregates smaller, domain-specific repositories, allows direct data submissions, and includes support for big data through its energy data lakes. OEDI's data lakes make high-value data universally accessible and help researchers, collaborators and the general public overcome many of the obstacles to accessing and using big data.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DAAP (Data Analytics Application Profiling)

The Data Analytics Application Profiling (DAAP) utility is a software program comprised of a library and scripts to provide basic instrumenting of parallel scientific software applications while they are running on an HPC cluster, and transmit this data over a secure connection as individual records to aggregators. When used in conjunction with additional open source software components (Telegraf, FluentD, and RabbitMQ), the data collected by DAAP can be transmitted off-cluster to an analytics system of the user’s choice for further analysis.

Shereda, Charles↗

The Archive Solution for Distributed Workflow Management Agents of the CMS Experiment at LHC

The CMS experiment at the CERN LHC developed the Workflow Management Archive system to persistently store unstructured framework job report documents produced by distributed workflow management agents. In this paper we present its architecture, implementation, deployment, and integration with the CMS and CERN computing infrastructures, such as central HDFS and Hadoop Spark cluster. The system leverages modern technologies such as a document oriented database and the Hadoop eco-system to provide the necessary flexibility to reliably process, store, and aggregate $\mathcal{O}$(1M) documents on a daily basis. We describe the data transformation, the short and long term storage layers, the query language, along with the aggregation pipeline developed to visualize various performance metrics to assist CMS data operators in assessing the performance of the CMS computing system.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗