Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Assessing the Impacts of Two Averaging Methods on AIRS Level 3 Monthly Products and Multi-Year Monthly Means

The Atmospheric Infrared Sounder (AIRS) onboard NASA’s Aqua satellite provides more than 16 years of data. Its monthly gridded (Level 3) product has been widely used for climate research and applications. Since counts of successful soundings in a grid cell are used to derive monthly averages, this “Averaged By Observations (ABO)” approach effectively gives equal importance to all participating soundings within a month. It is conceivable, then, that days with more observations due to day-to-day orbit shift and regimes with better retrieval skills, will contribute disproportionately to the monthly average within a cell. Alternatively, the AIRS Level 3 monthly product can be produced through an "Averaged By Days (ABD)” approach, where the monthly mean in a grid cell is a simple average of the daily means. The effects of these averaging methods on the AIRS version 6 monthly product are assessed quantitatively using temperature and water vapor at surface and 500hPa. The ABO method results in a warmer (slightly colder) global mean temperature at surface (500hPa) and a drier global mean water vapor than ABD method. The AIRS multi-year monthly mean temperature and water vapor from both methods are also compared with the Modern-Era Retrospective analysis for Research and Applications – 2 (MERRA-2) product and evaluated with a simulation experiment, indicating the ABD method has less error and is more closely correlated with MERRA-2. In summary, the ABD method is recommended for future versions of the AIRS Level 3 monthly product and more data services supporting Level 3 aggregation are needed.

AIRS↗

FLUXNET-CH4: a global, multi-ecosystem dataset and analysis of methane seasonality from freshwater wetlands

Methane (CH4) emissions from natural landscapes constitute roughly half of global CH4 contributions to the atmosphere, yet large uncertainties remain in the absolute magnitude and the seasonality of emission quantities and drivers. Eddy covariance (EC) measurements of CH4 flux are ideal for constraining ecosystem-scale CH4 emissions due to quasi-continuous and high-temporal-resolution CH4 flux measurements, coincident carbon dioxide, water, and energy flux measurements, lack of ecosystem disturbance, and increased availability of datasets over the last decade. Here, we (1) describe the newly published dataset, FLUXNET-CH4 Version 1.0, the first open-source global dataset of CH4 EC measurements (available at https://fluxnet.org/data/fluxnet-ch4-community-product/, last access: 7 April 2021). FLUXNET-CH4 includes half-hourly and daily gap-filled and non-gap-filled aggregated CH4 fluxes and meteorological data from 79 sites globally: 42 freshwater wetlands, 6 brackish and saline wetlands, 7 formerly drained ecosystems, 7 rice paddy sites, 2 lakes, and 15 uplands. Then, we (2) evaluate FLUXNET-CH4 representativeness for freshwater wetland coverage globally because the majority of sites in FLUXNET-CH4 Version 1.0 are freshwater wetlands which are a substantial source of total atmospheric CH4 emissions; and (3) we provide the first global estimates of the seasonal variability and seasonality predictors of freshwater wetland CH4 fluxes. Our representativeness analysis suggests that the freshwater wetland sites in the dataset cover global wetland bioclimatic attributes (encompassing energy, moisture, and vegetation-related parameters) in arctic, boreal, and temperate regions but only sparsely cover humid tropical regions. Seasonality metrics of wetland CH4 emissions vary considerably across latitudinal bands. In freshwater wetlands (except those between 20°S to 20°N) the spring onset of elevated CH4 emissions starts 3 d earlier, and the CH4 emission season lasts 4 d longer, for each degree Celsius increase in mean annual air temperature. On average, the spring onset of increasing CH4 emissions lags behind soil warming by 1 month, with very few sites experiencing increased CH4 emissions prior to the onset of soil warming. In contrast, roughly half of these sites experience the spring onset of rising CH4 emissions prior to the spring increase in gross primary productivity (GPP). The timing of peak summer CH4 emissions does not correlate with the timing for either peak summer temperature or peak GPP. Our results provide seasonality parameters for CH4 modeling and highlight seasonality metrics that cannot be predicted by temperature or GPP (i.e., seasonality of CH4 peak). FLUXNET-CH4 is a powerful new resource for diagnosing and understanding the role of terrestrial ecosystems and climate drivers in the global CH4 cycle, and future additions of sites in tropical ecosystems and site years of data collection will provide added value to this database. All seasonality parameters are available at https://doi.org/10.5281/zenodo.4672601 (Delwiche et al., 2021). Additionally, raw FLUXNET-CH4 data used to extract seasonality parameters can be downloaded from https://fluxnet.org/data/fluxnet-ch4-community-product/ (last access: 7 April 2021), and a complete list of the 79 individual site data DOIs is provided in Table 2 of this paper.

FLUXNET-CH4↗

Sampling Biases of Space-Based Observations of XCO 2 Associated With South American Biomass Burning Events

We examine time series of OCO-2 xCO2 observations from 2017-2020 coincident with other remotely sensed and in-situ observations to better understand external sampling effects associated with biomass burning events. The GES DISC will publish How-To documents as Jupyter Notebooks to describe how to aggregate OCO-2 Level 2 data and create similar time series.

Kristan Morgan↗

Clustering of graphs with of multiple edge types.

We study clustering on graphs with multiple edge types. Our main motivation is that similarities between objects can be measured in many different metrics. For instance similarity between two papers can be based on common authors, where they are published, keyword similarity, citations, etc. As such, graphs with multiple edges is a more accurate model to describe similarities between objects. Each edge/metric provides only partial information about the data; recovering full information requires aggregation of all the similarity metrics. Clustering becomes much more challenging in this context, since in addition to the difficulties of the traditional clustering problem, we have to deal with a space of clusterings. We generalize the concept of clustering in single-edge graphs to multi-edged graphs and investigate problems such as: Can we find a clustering that remains good, even if we change the relative weights of metrics? How can we describe the space of clusterings efficiently? Can we find unexpected clusterings (a good clustering that is distant from all given clusterings)? If given the groundtruth clustering, can we recover how the weights for edge types were aggregated?

99 GENERAL AND MISCELLANEOUS↗

Cache coherence for processing in memory

A cache coherence bridge protocol provides an interface between a cache coherence protocol of a host processor and a cache coherence protocol of a processor-in-memory, thereby decoupling coherence mechanisms of the host processor and the processor-in-memory. The cache coherence bridge protocol requires limited change to existing host processor cache coherence protocols. The cache coherence bridge protocol may be used to facilitate interoperability between host processors and processor-in-memory devices designed by different vendors and both the host processors and processor-in-memory devices may implement coherence techniques among computing units within each processor. The cache coherence bridge protocol may support different granularity of cache coherence permissions than those used by cache coherence protocols of a host processor and/or a processor-in-memory. The cache coherence bridge protocol uses a shadow directory that maintains status information indicating an aggregate view of copies of data cached in a system external to a processor-in-memory containing that data.

97 MATHEMATICS AND COMPUTING↗

How does drought affect residential water demand and price elasticity?

Urban water scarcity is an important social and economic concern, particularly as the intensity, duration, and frequency of droughts is increasing in many regions. We consider whether drought induces changes to water demand and the price elasticity of demand for water that may last beyond a drought’s official end date. If drought shocks prompt long-term changes in water demand behavior, and these changes occur at broad geographic scale, they could have important implications for modeling adaptive responses to water scarcity. We assemble a novel dataset on residential water demand and pricing in the western United States to test empirically for effects of drought on water demand and price elasticity. We perform our analysis with aggregate quantity, price, and drought data, accounting for endogenous prices under increasing-block water tariffs and using both average and marginal water fees in estimating water demand functions. Results are consistent with the hypothesis that households may become less price-sensitive after exposure to drought. However, we find no systematic evidence of long-run, drought-related reductions in water demand, itself.

demand hardening↗

Compendium of photovoltaic degradation rates

Abstract Published data on photovoltaic (PV) degradation measurements were aggregated and re‐examined. The subject has seen an increased interest in recent years resulting in more than 11 000 degradation rates in almost 200 studies from 40 different countries. As studies have grown in number and size, we found an impact from sampling bias attributable to size and accuracy. Because of the correlational nature of this study we examined the data in several ways to minimize this bias. We found median degradation for x‐Si technologies in the 0.5–0.6%/year range with the mean in the 0.8–0.9%/year range. Hetero‐interface technology (HIT) and microcrystalline silicon (µc‐Si) technologies, although not as plentiful, exhibit degradation around 1%/year and resemble thin‐film products more closely than x‐Si. Several studies showing low degradation for copper indium gallium selenide (CIGS) have emerged. Higher degradation for cadmium telluride (CdTe) has been reported, but these findings could reflect a convolution of less accurate studies and longer stabilization periods for some products. Significant deviations for beginning‐of‐life measurements with respect to nameplate rating have been documented over the last 35 years. Therefore, degradation rates that use nameplate rating as reference may be significantly impacted. Studies that used nameplate rating as reference but used solar simulators showed less variation than similar studies using outdoor measurements, even when accounting for different climates. This could be associated with confounding effects of measurement uncertainty and soiling that take place outdoors. Hotter climates and mounting configurations that lead to sustained higher temperatures may lead to higher degradation in some, but not all, products. Wear‐out non‐linearities for the worst performing modules have been documented in a few select studies that took multiple measurements of an ensemble of modules during the lifetime of the system. However, the majority of these modules exhibit a fairly linear decline. Modeling these non‐linearities, whether they occur at the beginning‐of‐life or end‐of‐life in the PV life cycle, has an important impact on the levelized cost of energy. Copyright © 2016 John Wiley & Sons, Ltd.

Jordan, Dirk C.↗

A data-driven weighting scheme for multivariate phenotypic endpoints recapitulates zebrafish developmental cascades

Zebrafish have become a key alternative model for studying health effects of environmental stressors, partly due to their genetic similarity to humans, fast generation time, and the efficiency of generating high-dimensional systematic data. Studies aiming to characterize adverse health effects in zebrafish typically include several phenotypic measurements (endpoints). While there is a solid biomedical basis for capturing a comprehensive set of endpoints, making summary judgments regarding health effects requires thoughtful integration across endpoints. Here, we introduce a Bayesian method to quantify the informativeness of 17 distinct zebrafish endpoints as a data-driven weighting scheme for a multi-endpoint summary measure, called weighted Aggregate Entropy (wAggE). We implement wAggE using high-throughput screening (HTS) data from zebrafish exposed to five concentrations of all 1060 ToxCast chemicals. Our results show that our empirical weighting scheme provides better performance in terms of the Receiver Operating Characteristic (ROC) curve for identifying significant morphological effects and improves robustness over traditional curve-fitting approaches. From a biological perspective, our results suggest that developmental cascade effects triggered by chemical exposure can be recapitulated by analyzing the relationships among endpoints. Thus, wAggE offers a powerful approach for analysis of multivariate phenotypes that can reveal underlying etiological processes. - Highlights: • Introduced a data-driven weighting scheme for multiple phenotypic endpoints. • Weighted Aggregate Entropy (wAggE) implies differential importance of endpoints. • Endpoint relationships reveal developmental cascade effects triggered by exposure. • wAggE is generalizable to multi-endpoint data of different shapes and scales.

60 APPLIED LIFE SCIENCES↗

Kinetically Controlled Fabrication of Single‐Crystalline TiO 2 Nanobrush Architectures with High Energy {001} Facets

This study demonstrates that precise control of nonequilibrium growth conditions during pulsed laser deposition (PLD) can be exploited to produce single‐crystalline anatase TiO 2 nanobrush architectures with large surface areas terminated with high energy {001} facets. The data indicate that the key to nanobrush formation is controlling the atomic surface transport processes to balance defect aggregation and surface‐smoothing processes. High‐resolution scanning transmission electron microscopy data reveal that defect‐mediated aggregation is the key to TiO 2 nanobrush formation. The large concentration of defects present at the intersection of domain boundaries promotes aggregation of PLD growth species, resulting in the growth of the single‐crystalline nanobrush architecture. This study proposes a model for the relationship between defect creation and growth mode in nonequilibrium environments, which enables application of this growth method to novel nanostructure design in a broad range of materials.

36 MATERIALS SCIENCE↗

A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County

The dataset contains hourly Anthropogenic heat (AH) from buildings in Los Angeles County, based on weather data from 2018. The hourly AH is aggregated at three spatial resolutions: 450m x 450m grid, 12km x 12km grid, and census tract. The AH is broken down into three components: building envelope surface convection, heating, ventilation, and air conditioning (HVAC) system heat release, and zone exfiltration and exhaust air heat loss. The dataset is created with the physics-based EnergyPlus building energy models to calculate individual buildings' AH considering WRF-UCM simulated microclimate conditions. Please refer to the paper "A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County" for more information about the data generation workflow and the data validation procedure. The data set contains two folders: the "output_data" folder holds the simulation results (EP_output and EP_output_csv), building metadata (building_metadata.geojson and building_metadata.csv), aggregated heat emission and energy consumption time-series data (hourly_heat_energy), and geographical data (geo_data) associated with the GEOID referenced in heat and energy consumption data. The "input_data" folder contains the raw data used to generate files in the "output_data" folder as well as data sets used in the validation. The code repository (https://github.com/IMMM-SFA/xu_etal_2022_sdata) holds the processing scripts for data curation, validation, and visualization.

Energy↗

Renewable Electricity Procurement Options Data (RE-POD)

The Renewable Electricity Procurement Options Data (RE-POD) is an aggregated dataset meant to help local jurisdictions and utility customers within those jurisdictions understand the options that may be available to them to procure renewable electricity or renewable energy credits to meet energy goals. This data is part of a suite of state and local energy profile data available at the "State and Local Energy Profile Data Suite" link below and builds on Cities-LEAP energy modeling, available at the "EERE Cities-LEAP Page" link below. Examples of how to use the data to inform energy planning can be found at the "Example Uses" link below.

Array↗

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on a quality-aware adaptive temporal data selection and reconstruction problem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we develop DeepSample , an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

Zhang, Yang↗

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

97 MATHEMATICS AND COMPUTING↗

Differentially Private Adaptive Noise Injection (DP-ANI) v1.0

Location data is collected from users continuously to understand their mobility patterns. Releasing the user trajectories may compromise user privacy. Therefore, the general practice is to release aggregated location datasets. However, private information may still be inferred from an aggregated version of location trajectories. Differential privacy (DP) protects the query output against inference attacks regardless of background knowledge. This software implements a differential privacy-based privacy model that protects the user's origins and destinations from being inferred from aggregated mobility datasets. This is achieved by injecting Planar Laplace noise to the user origin and destination GPS points. The noisy GPS points are then transformed into a link representation using a link-matching algorithm. Finally, the link trajectories form an aggregated mobility network. The injected noise level is selected using the Sparse Vector Mechanism. This DP selection mechanism considers the link density of the location and the functional category of the localized links. Compared to the different baseline models, including a k-anonymity method, our differential privacy-based aggregation model offers query responses that are close to the raw data in terms of aggregate statistics at both the network and trajectory-levels with maximum 9% deviation from the baseline in terms of network length.

Peisert, Sean [Lawrence Berkeley National Laborato↗

Skluma: An extensible metadata extraction pipeline for disorganized data

To mitigate the effects of high-velocity data expansion and to automate the organization of filesystems and data repositories, we have developed Skluma-a system that automatically processes a target filesystem or repository, extracts content-and context-based metadata, and organizes extracted metadata for subsequent use. Skluma is able to extract diverse metadata, including aggregate values derived from embedded structured data; named entities and latent topics buried within free-text documents; and content encoded in images. Skluma implements an overarching probabilistic pipeline to extract increasingly specific metadata from files. It applies machine learning methods to determine file types, dynamically prioritizes and then executes a suite of metadata extractors, and explores contextual metadata based on relationships among files. The derived metadata, represented in JSON, describes probabilistic knowledge of each file that may be subsequently used for discovery or organization. Skluma's architecture enables it to be deployed both locally and used as an on-demand, cloud-hosted service to create and execute dynamic extraction workflows on massive numbers of files. It is modular and extensible-allowing users to contribute their own specialized metadata extractors. Thus far we have tested Skluma on local filesystems, remote FTP-accessible servers, and publicly-accessible Globus endpoints. We have demonstrated its efficacy by applying it to a scientific environmental data repository of more than 500,000 files. We show that we can extract metadata from those files with modest cloud costs in a few hours.

data swamp↗

The Data Acquisition and Controls Systems (DACS) of the E-Complex at the John C. Stennis Space Center, MS: A General Overview

The John C. Stennis Space Center (SSC) provides test operations services to a variety of customers including NASA, DoD, commercial enterprises, and others for the development of current next-generation rocket propulsion systems. Many of these test operations services are provided in the E-Complex series of test facilities. The E-Complex is composed of three active test stands (E1, E2, & E3), each with two or more test positions. Each test position is comprised of unique sets of data acquisition and controls hardware and software that record both facility and test article data and safely operate the test facility. The E-Complex data acquisition system (DAS) is actually composed of two separate systems, one for static data and the other for dynamic. The static DAS, otherwise known as the Low-Speed DAS (LSDAS), samples 16 bit data at 250 samples-per-second (SPS), although an aggregate sample rate of 200,000 SPS is possible. The dynamic data acquisition system, otherwise known as the high-speed DAS (HSDAS), samples 16 bit data at 100K SPS with a 45 KHz bandwidth.

Hughes, Mark S.↗

Transcriptomics-based Machine Learning (ML) Analysis Predicts Space-Exposed Murine Livers

NASA has employed high-throughput molecular assays to identify sub-cellular changes impacting human physiology during spaceflight. Machine learning (ML) methods hold the promise to improve our ability to identify important signals within highly dimensional molecular data. However, the inherent limitation of study subject numbers within a spaceflight mission minimizes the utility of ML approaches. To overcome the sample power limitations, data from multiple spaceflight missions must be aggregated while appropriately addressing intra- and inter-study variabilities. Here we describe an approach to log transform, scale and normalize data from six heterogeneous, mouse liver derived transcriptomics datasets (ntotal=137) which enabled ML-methods to perform well (AUC ≥ 0.87) in classifying spaceflown vs ground control animals rather than mission-of-origin. Concordance was found between liver-specific biological processes identified from harmonized ML-based analysis and study-by-study classical omics analysis. This work demonstrates the feasibility of applying ML methods on integrated, heterogeneous datasets of small sample size.

Machine Learning↗

LTE Electrolyzer Data Collection

The goal for NREL is to collect, develop and publish performance metrics relative to low temperature electrolyzer installations. This will be done through the development of: Secure storage solution to house the collection of data from multiple projects Standardization of data to be collected and analyzed. This will be done using data templates developed with the help of partners involved with electrolyzer installations. Analysis that produces metrics of interest for all stakeholders Aggregation of results from multiple projects to view industry progress as a whole Publication of aggregated results in the form of composite data products (CDPs) Collaboration with Idaho National Lab and their work with high temperature electrolyzer installations will enable efficient use of storage and analysis tools.

data↗