Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Multiscale and Multivariate Transportation System Visualization for Shopping District Traffic and Regional Traffic

In this paper, we present a suite of visualization techniques for sensor-based transportation system data at different scales to facilitate the exploration of interconnected traffic dynamics at intersections and highways. Additionally, these techniques are designed for analyzing multivariate traffic data from radar-based highway sensors and camera-based intersection sensors recording turn movements and vehicle speed, in the Chattanooga Metropolitan Area, with the capability of (a) revealing multiscale mobility patterns using different levels of data aggregation (e.g., individual sensor for microscale, multiple sensors along a corridor for mesoscale, and a larger number of sensors across the region for macroscale visualization) at different intervals (e.g., 5-min intervals, time of day, full day, and day-of-the-week), and (b) exploring the spatial variation of multiple traffic-related variables (e.g., volumes, speeds, turn movements, and traffic light colors) provided by the sensors. We close with a case study to demonstrate the effectiveness of our multiscale and multivariate visualization techniques. At microscale, we focused on intersection data from a shopping district around Shallowford Road in East Chattanooga. For mesoscale visualization, we studied the Shallowford Road corridor and an adjacent stretch of I-75. At macroscale, we included highway data from the Chattanooga Metropolitan Area. All visualizations were integrated into a web-based situational awareness tool to promote user access and interaction. At a minimum, each visualization provides the option for selecting dates for real-time (depending on sensor availability) and historical data, and additional information on hovering, though most provide more detailed information, including different views of the selected data, or interactive highlights.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS2007

An Intriguing Convex Break in the EGRET SED of Mrk 421

Based upon analysis of the entire EGRET data from Mrk 421, it is found that the time-averaged spectra are inconsistent with the predictions of current theoretical models that have had success in describing simultaneous X-ray/TeV observations, and suggest additional components in the GeV band, as well as complex time variability. Current theoretical pictures explain the GeV emission as comptonization of the synchrotron photons in the jet, and predict hard spectra that should join smoothly with the TeV emission. Our analysis shows that the situation is more complex. The spectrum ranges from hard to soft during individual epochs, and shows a convext break in the aggregated data. We also present the mission-averaged EGRET spectrum for PKS 2155-304, which shows a similar (but not as pronounced) convex curvature. We discuss a series of possible explanations for the 10(exp 22) - 10(exp 23) HZ declining part of the EGRET nu F(sub nu), spectrum for Mrk 421, and suggest that it is synchrotron emission from the high energy tail of the electron population that produces the X-rays during the highest X-ray states. Such multi-MeV photons are produced by electrons accelerated close to the limit of diffusive shock acceleration. Simultaneous GLAST and X-ray observations of high X-ray states will address the issue of the convex curvature in the future.

Nandikotkur, Giridhar↗
DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗
DOE OSTI2022

Prototype hardware design and testing of the global common module for the global trigger subsystem of the ATLAS phase-II upgrade

We report the High-Luminosity Large Hadron Collider (HL-LHC) will deliver more than ten times the integrated luminosity of the previous runs combined. Meeting its stricter throughput requirements poses new challenges to the Trigger and Data Acquisition (TDAQ) systems of the LHC experiments. Introduced in the framework of the ATLAS experiment’s HL upgrade, the Global Trigger (GT) is a new subsystem which will perform offline-like algorithms on full-granularity calorimeter data. The implementation of the GT’s functionality is firmware-focused and is composed of three layers: multiplexing (or data aggregating), global event processing, and demultiplexing interface to the central trigger processor. Each layer will be composed of several, similar nodes, hosted on replicas of identical hardware, the Global Common Module (GCM), an ATCA front board which is designed to be adopted throughout the entire GT subsystem. This article proceeds from the TWEPP 2021 conference and presents the GCM hardware design, performed in 2020, and focuses on some key results of its extensive testing performed in 2021.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2017

Smashing the Stovepipe: Leveraging the GMSEC Open Architecture and Advanced IT Automation to Rapidly Prototype, Develop and Deploy Next-Generation Multi-Mission Ground Systems

Satellite/Payload Ground Systems - Typically highly-customized to a specific mission's use cases - Utilize hundreds (or thousands!) of specialized point-to-point interfaces for data flows / file transfers Documentation and tracking of these complex interfaces requires extensive time to develop and extremely high staffing costs Implementation and testing of these interfaces are even more cost-prohibitive, and documentation often lags behind implementation resulting in inconsistencies down the road With expanding threat vectors, IT Security, Information Assurance and Operational Security have become key Ground System architecture drivers New Federal security-related directives are generated on a daily basis, imposing new requirements on current / existing ground systems - These mandated activities and data calls typically carry little or no additional funding for implementation As a result, Ground System Sustaining Engineering groups and Information Technology staff continually struggle to keep up with the rolling tide of security Advancing security concerns and shrinking budgets are pushing these large stove-piped ground systems to begin sharing resources - I.e. Operational / SysAdmin staff, IT security baselines, architecture decisions or even networks / hosting infrastructure Refactoring these existing ground systems into multi-mission assets proves extremely challenging due to what is typically very tight coupling between legacy components As a result, many "Multi-Mission" ops. environments end up simply sharing compute resources and networks due to the difficulty of refactoring into true multi-mission systems Utilizing continuous integration / rapid system deployment technologies in conjunction with an open architecture messaging approach allows System Engineers and Architects to worry less about the low-level details of interfaces between components and configuration of systems GMSEC messaging is inherently designed to support multi-mission requirements, and allows components to aggregate data across multiple homogeneous or heterogeneous satellites or payloads - The highly-successful Goddard Science and Planetary Operations Control Center (SPOCC) utilizes GMSEC as the hub for it's automation and situational awareness capability Shifts focus towards getting GS to a final configuration-managed baseline, as well as multi-mission / big-picture capabilities that help increase situational awareness, promote cross-mission sharing and establish enhanced fleet management capabilities across all levels of the enterprise.

GMSEC↗
DOE OSTI2021

Evaluating the Benefits of Bayesian Hierarchical Methods for Analyzing Heterogeneous Environmental Datasets: A Case Study of Marine Organic Carbon Fluxes

Large compilations of heterogeneous environmental observations are increasingly available as public databases, allowing researchers to test hypotheses across datasets. Statistical complexities arise when analyzing compiled data due to unbalanced spatial sampling, variable environmental context, mixed measurement techniques, and other reasons. Hierarchical Bayesian modeling is increasingly used in environmental science to describe these complexities, however few studies explicitly compare the utility of hierarchical Bayesian models to simpler and more commonly applied methods. Here we demonstrate the utility of the hierarchical Bayesian approach with application to a large compiled environmental dataset consisting of 5,741 marine vertical organic carbon flux observations from 407 sampling locations spanning eight biomes across the global ocean. We fit a global scale Bayesian hierarchical model that describes the vertical profile of organic carbon flux with depth. Profile parameters within a particular biome are assumed to share a common deviation from the global mean profile. Individual station-level parameters are then modeled as deviations from the common biome-level profile. The hierarchical approach is shown to have several benefits over simpler and more common data aggregation methods. First, the hierarchical approach avoids statistical complexities introduced due to unbalanced sampling and allows for flexible incorporation of spatial heterogeneitites in model parameters. Second, the hierarchical approach uses the whole dataset simultaneously to fit the model parameters which shares information across datasets and reduces the uncertainty up to 95% in individual profiles. Third, the Bayesian approach incorporates prior scientific information about model parameters; for example, the non-negativity of chemical concentrations or mass-balance, which we apply here. We explicitly quantify each of these properties in turn. We emphasize the generality of the hierarchical Bayesian approach for diverse environmental applications and its increasing feasibility for large datasets due to recent developments in Markov Chain Monte Carlo algorithms and easy-to-use high-level software implementations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Quantifying microbial control of soil organic matter dynamics at macrosystem scales

Soil organic matter (SOM) stocks, decomposition and persistence are largely the product of controls that act locally. Yet the controls are shaped and interact at multiple spatiotemporal scales, from which macrosystem patterns in SOM emerge. Theory on SOM turnover recognizes the resulting spatial and temporal conditionality in the effect sizes of controls that play out across macrosystems, and couples them through evolutionary and community assembly processes. For example, climate history shapes plant functional traits, which in turn interact with contemporary climate to influence SOM dynamics. Selection and assembly also shape the functional traits of soil decomposer communities, but it is less clear how in turn these traits influence temporal macrosystem patterns in SOM turnover. Here, we review evidence that establishes the expectation that selection and assembly should generate decomposer communities across macrosystems that have distinct functional effects on SOM dynamics. Representation of this knowledge in soil biogeochemical models affects the magnitude and direction of projected SOM responses under global change. Yet there is high uncertainty and low confidence in these projections. To address these issues, we make the case that a coordinated set of empirical practices are required which necessitate (1) greater use of statistical approaches in biogeochemistry that are suited to causative inference; (2) long-term, macrosystem-scale, observational and experimental networks to reveal conditionality in effect sizes, and embedded correlation, in controls on SOM turnover; and (3) use of multiple measurement grains to capture local- and macroscale variation in controls and outcomes, to avoid obscuring causative understanding through data aggregation. Here, when employed together, along with process-based models to synthesize knowledge and guide further empirical work, we believe these practices will rapidly advance understanding of microbial controls on SOM and improve carbon cycle projections that guide policies on climate adaptation and mitigation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

CreelCat, a Catalog of United States Inland Creel and Angler Survey Data

The United States Inland Creel and Angler Survey Catalog (CreelCat) contains a national compilation of angler and creel survey data collected by natural resource management agencies across the United States (including Washington, D.C. and Puerto Rico). These surveys are used to help inform the management of recreational fisheries, by collecting information about anglers including what they are catching and harvesting, the amount of effort they expend, their angling preferences, and demographic information. As of May 1, 2023, CreelCat houses over 14,729 surveys from 33 states, Puerto Rico, and Washington, D.C., comprising 235 data fields across 8 tables. These tables contain 235,015 records of fish catch and harvest metrics, 27,250 angler preference metrics, 14,729 records of survey characteristics, 13,576 records of effort metrics, and 409 records of angler demographics. Though individual creel surveys are often deployed to meet local science and management objectives, creel data aggregated across jurisdictions has the potential to address larger scale research and management needs.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Distributional Equity in the Employment and Wage Impacts of Energy Transitions

We use restricted-access, geocoded data on the near-universe of workers in 23 US states to quantify the impact of wind energy development on local earnings and employment, by race, ethnicity, sex, and educational attainment. We find significant impacts that persist for several years beyond the project construction phase. Our estimates are larger than those from previous studies but still small relative to typical economic multipliers for fiscal spending or investment in other industries. We find the largest percentage increases for black workers and workers who either do not have a high school diploma or who have a college degree. We also find the economic gains for men to be much larger than those for women. Finally, we find estimates from data aggregated to the county level to be significantly lower than our worker-level estimates. We suggest a number of areas for further study building off the justice implications of our findings.

employment↗
DOE OSTI2020

A Learning-based Power Management Method for Networked Microgrids Under Incomplete Information

In this paper, we present a model-free reinforcement learning (RL)-based technique to solve the power management problem of networked microgrids (MGs), when the utility has only limited or no detailed knowledge of the MG asset models.The proposed method consists of two levels: at the first level, using an approximate RL strategy and aggregated data, a utility agent determines the locational price signals to maximize its profit under incomplete information on the MGs’ private data behind their points of common coupling (PCC). The proposed RL strategy enables the utility to adapt to changing system conditions and learn from previous experiences, while respecting the data ownership of the MGs. At the second level, the MGs receive the price signals to dispatch their generation/storage assets individually over look-ahead decision time window, while considering power flow constraints. The performance of the proposed RL technique has been verified under realistic operational scenarios.

Zhang, Qianzhi↗
DOE OSTI2024

Quantum teleportation coexisting with classical communications in optical fiber

The ability for quantum and conventional networks to operate in the same optical fibers would aid the deployment of quantum network technology on a large scale. Quantum teleportation is a fundamental operation in quantum networking, but has yet to be demonstrated in fibers populated with high-power conventional optical signals. Here we report, to the best of our knowledge, the first demonstration of quantum teleportation over fibers carrying conventional telecommunications traffic. Quantum state transfer is achieved over a 30.2-km fiber carrying 400-Gbps C-band classical traffic with a Bell state measurement performed at the fiber’s midpoint. To protect quantum fidelity from spontaneous Raman scattering noise, we use optimal O-band quantum channels, narrow spectro-temporal filtering, and multi-photon coincidence detection. Fidelity is shown to be well maintained with an elevated C-band launch power of 18.7 dBm for the single-channel 400-Gbps signal, which we project could support multiple classical channels totaling many terabits/s aggregate data rates. These results show the feasibility of advanced quantum and classical network applications operating within a unified fiber infrastructure.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Manuscript Workflows from and Processed Organic Matter Composition of Experimentally Burned Open Air and Muffle Furnace Vegetation Chars across Differing Burn Severity and Feedstock Types from Pacific Northwest, USA (v3)

This dataset includes processed organic matter chemistry data from an experimental study designed to compare how the chemical composition of organic matter changes across different burn conditions and vegetation materials representative of major land cover types of the Pacific Northwest, USA. Chars were created in a closed muffle furnace or on an open burn table from four different feedstock species representing vegetation commonly impacted by fire regimes across the Pacific Northwest, USA. Source data and associated metadata (including methods and geospatial information) can be found at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1894135 (Grieger et al. 2022). This dataset provides processing scripts and processed data for both solid and dissolved phase organic matter characterization data from experimentally generated chars. These processed data can be used to compare how different burn conditions may influence resultant organic matter chemistry and help further our understanding of potential biogeochemical impacts on river corridors post-fire. The processed data were subsequently analyzed; and the results and ecological implications of the findings were published in peer-reviewed manuscripts. The scripts and workflows used to develop the manuscripts are also included in this data package.This data package was originally published June 2024. It was updated September 2024 (new and modified files) and in January 2025 (modified files). See the change history section in the readme for more details.This dataset is comprised of one data package readme, one data dictionary (dd), one file level metadata (flmd), and folders containing (A) processed data; (B) general processing scripts; and (C) additional folders with specific manuscript analysis scripts and processed data. Step-by-step instructions to assist the user in recreating the workflow used to generate the results in the manuscripts is also provided. The processed data folder includes (1) a folder of processed Parallel Factor Analysis (PARAFAC) and spectra indices outputs from excitation emissions matrix (EEM) fluorescence and absorbance data; (2) a folder of processed solid state carbon-13 (13-C NMR) integrals; (3) folder of high resolution characterization of organic matter via 21 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) generated through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory) processed data outputs from Formultitude (https://github.com/PNNL-Comp-Mass-Spec/Formultitude), blank corrections and data aggregation, and calculated molecular indices. All files are .pdf, .csv, .html, .Rmd, .R, or .RData.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

PEC Performance Measures from Literature Reports

This dataset is a capture of PEC experimental performance metrics data aggregated from publications and included here in the measurement and format provided by the authors. The measurements captured are based on rate of hydrogen production by PEC devices under illumination. Original creator of this dataset: Wen-Hui (Sophia) Cheng (National Cheng Kung University) Contributors to this dataset: Joel Ager (LBNL), H. Nishiyama et al. (Meiji University), I. Holmes-Gentle (EPFL), Yanfa Yan (University of Toledo), Aditya Mohite (Rice University), Zetian Mi (University of Michigan).

08 HYDROGEN↗
DOE OSTI2025

gaia: An R package to estimate crop yield responses to temperature and precipitation

gaia is an open-source R package designed to estimate crop yield shocks in response to annual weather variations and CO 2 concentrations at the country scale for 17 major crops. This innovative tool streamlines the workflow from raw climate data processing to projections of annual shocks to crop yields at the country level, using the response surfaces from an empirical econometric model developed and documented in Waldhoff et al. (2020), which leverages historical weather, CO 2 , and crop yield data for robust empirical fitting for 17 crops. gaia uses these response surfaces with monthly temperature and precipitation projections (e.g., from the Coupled Model Intercomparison Project Phase 6 (CMIP6) (O’Neill et al., 2016) climate data bias-adjusted and statistically downscaled by the ISIMIP3BASD approach (Lange, 2019) in the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) (Warszawski et al., 2014)) to project yield shocks that can be applied to agricultural productivity changes at the country level for use in multisectoral economic models. The historical and future projections use gridded, country-and-crop specific monthly growing season precipitation and temperature data, aggregated to the national level, and weighted by cropland area derived from the global Monthly Irrigated and Rainfed Crop Areas around the year 2000 (MIRCA2000) dataset (Portmann et al., 2010). These annual, country, and crop-specific yield shocks can be aggregated to different definitions of regions, crop commodities, and time periods, as needed by specific multisectoral economic models. gaia serves as a lightweight, powerful tool that can aid exploration of crop yield responses under a broad range of future climate projections, enhancing human-Earth system analysis capabilities.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Model run output for WM-ABM 2024 HESS Publication

Model output for computational experiments introducing WM-ABM. See associated HESS publication here: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1604/ all_nldas_cdl_data_v3.txt - Cropland Data Layer data aggregated to 1/8 degree resolution for input into the PMP calibration process. Variables include NLDAS ID, crop type, year, the count of pixels in the grid cell of the crop type, CDL crop ID, and GCAM crop ID. 20230115 ABM runs - WM-ABM model outputs for various scenarios described in HESS publication. If multiple versions exist (e.g., v2, v3, etc.) latest version indicates dataset generated for manuscript. The "baseline" run is the primary run, where as the "memXX" runs indicate sensitivity runs testing different parameters of agent memory. Output files starting with "abm_results_" include groundwater irrigated acres and surface water irrigated acres for each farm/agent combination. Other CSV outputs provide summary statistics of various agent and water shortage outcomes, used for generation of figures in the HESS manuscript.

ABM↗
DOE OSTI2025

Integrated GW Farm ABM

This Data Repository includes data used for the integrated groundwater- farm ABM model, raw model output from scenario ensemble, and processed outputs that isolate the groundwater storage depletion outcomes for the 35,000 farm cells. Model Inputs: Farm ABM Inputs: This folder contains the input data used by the integrated groundwater - farm ABM modelling script (Python file) used for the high performance computing (HPC) experiments. The sub-folder "data inputs" contains all of the farm attribute data, while the three files in the folder have the hydrogeological data lookup table (NLDAS Cost Curve Attributes.csv), a lookup table (Theis well function table.csv) for the groundwater cost curve function, and the farm indexes and corresponding NLDAS ids for all of the cells run in this experiment (nldas farms subset final.csv). NLDAS Cost curve hydrogeological data: Hydrogeological data aggregated to 1/8 degree resolution and aligned with the NLDAS grid. Parameters include: water depth below ground surface [meters], subsurface porosity [unitless], aquifer depth from ground surface to aquifer bottom [meters], annual average recharge (USGS: mm, Doll: meters), and three different hydraulic conductivity (K) values (meters/day). The three K values represent the mean value from Gleeson et al. (2018), one standard deviation above the mean from Gleeson et al. (2018), and the de Graaf et al. 2020 modifications to certain lithologies. Additional information about these datasets and their processing are documented in the supplement to Yoon et al. 2025 (in review). Output: Raw outputs: This folder contains a .zip file that has model outputs for the entire scenario ensemble. There is one csv for each farm id, using the format "farm farmid cases.csv". The relationship between the farm id and NLDAS id is defined by the "nldas farms subset final.csv" located in the Farm ABM Inputs folder. Each csv has 625 rows, corresponding to 625 combinations of different scenario parameter values. Each row (scenario) represents the outcome of a 100 year simulation. Columns define scenario settings and summary statistics for each scenario. The first four columns define the scenario settings: "hydro ratio," "econ ratio," "K scenario," and "gamma scenario." The hydro and econ ratios are values passed to the modeling script that influence multipliers for other model parameters, as documented in the supplement to Yoon et al. 2025 (in review). The gamma multiplier is a coefficient multiplier applied to the baseline gamma values (values below 1 represent lower unobserved costs compared to baseline, values above 1 represent higher costs). The K scenario names represent K values of: "low": 0.5 m/d, "int 1": 2.5 m/d, "int 2": 10 m/d, "high": 50 m/d, and "gleeson": mean Gleeson K value. "Perc vol depleted" is the fraction of groundwater depleted at the end of the 100 simulation. Processed Output: Derived depletion outcomes from raw outputs: All of the individual csv files from the Raw outputs were aggregated into a single file that has the scenario settings and fraction depletion "Perc vol depleted" for every farm cell, for every scenario. The other two files define relationships between the farm id, NLDAS id, and local and major aquifer units, used for aquifer-level depletion analysis.

Agent based modeling↗