SEARCH · Search NASA

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Field-scale dynamics of planting dates in the US Corn Belt from 2000 to 2020

Crop planting dates are a dynamic feature of agricultural systems that respond to short- and long-term climate signals, crop and cultivar selection, and technology changes. Planting date records are essential for yield gap analyses, accurate crop modeling, and tracking farmer adaptations to weather and climate change. Although planting dates have high variation at local scales due to heterogeneity in farm resources and decision-making, available long-term data on planting dates is largely restricted to aggregated regional statistics or, at best, satellite-derived datasets with limited spatiotemporal extent and at resolutions unable to distinguish individual fields (> 250 m). Here, we generated retrospective annual field-scale (30 m) planting date maps for both maize and soybeans spanning 2000-2020 across a 12 state region in the United States Corn Belt based on Landsat satellite data and a large ground sample of over 28,000 maize and soybean fields. Using training data from 2015-2020 for model selection, we found that planting date predictions improved with harmonic regression of Landsat data and additional annual weather covariates. The preferred random forests model approximately doubled performance compared to a null model based on state median planting dates, capturing 47% of field-level variation for maize (mean absolute error, MAE = 7.4 days) and 44% for soybeans (MAE = 7.5 days) against held-out ground truth test data for 2008-2014. We also evaluated the full 2000-2020 dataset with state agricultural statistics, finding strong agreement with median planting dates for maize (R 2 = 0.76, MAE = 4.4 days) and slightly lower agreement for soybeans (R 2 = 0.65, MAE = 5.4 days) when aggregated to the state level. We then used this new dataset to analyze environmental determinants of planting dates at a finer-scale than previously possible, controlling for unobserved variation at the sub-state district level. We found that during 2000-2020, each standard deviation increase in rainfall delayed planting by ~ 2.5 days, and fields with higher soil productivity ratings tended to be planted earlier. We did not find meaningful trends over the last two decades in planting dates for maize or soybeans, in contrast to trends towards earlier planting dates late last century and predicted for this period in climate adaptation studies. We hypothesize increases in early season rainfall may have inhibited these shifts towards earlier planting. Remotely sensed planting dates will be a useful tool for yield gap analyses, crop simulation modeling, and ongoing assessment of climate adaptation.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRSDate not supplied

Creating Near Real-time Hourly VIIRS Radiance Mosaic over Polar Regions from SatCORPS Global Cloud Composite (GCC)

While polar orbiting satellites are essential in providing the needed observations over polar regions for data assimilation, weather diagnoses/forecasting, and flight guidance and situation awareness in airborne field campaigns, there is limited temporal coverage. Often, polar mosaics are created by overlaying multiple satellite imagers on top of each other which can lead to transition lines and discontinuities in the data products. To help ease some of these shortcomings, we utilize the NASA Langley Satellite ClOud and Radiation Property retrieval System (SatCORPS) Mosaic algorithm for optimally fusing and merging selected imager radiances from the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument onboard the Suomi National polar-orbiting Partnership (SNPP), Joint Polar Satellite System (JPSS-1), and JPSS-2 satellites, and the MODerate resolution Imaging Spectroradiometer (MODIS) instruments onboard AQUA and TERRA satellites. Since multiple satellites overpasses and instruments are used in creating hourly 3-km composites over the polar regions, selection of the best satellite data for each 3-km composite pixel is based on aggregated ranking of satellite resolution, time, viewing zenith angle, and distance from terminator. This ranking system provides a smoother transition and improves data discontinuity in the merged output resulting in a more seamless polar region dataset. We will discuss the use of this near real-time dataset to support polar airborne field campaigns and SatCORPS satellite-derived Global Cloud Composite (GCC) generation.

Satellite composites↗

NASA NTRS1993

Strategic planning for aircraft noise route impact analysis: A three dimensional approach

The strategic routing of aircraft through navigable and controlled airspace to minimize adverse noise impact over sensitive areas is critical in the proper management and planning of the U.S. based airport system. A major objective of this phase of research is to identify, inventory, characterize, and analyze the various environmental, land planning, and regulatory data bases, along with potential three dimensional software and hardware systems that can be potentially applied for an impact assessment of any existing or planned air route. There are eight data bases that have to be assembled and developed in order to develop three dimensional aircraft route impact methodology. These data bases which cover geographical information systems, sound metrics, land use, airspace operational control measures, federal regulations and advisories, census data, and environmental attributes have been examined and aggregated. A three dimensional format is necessary for planning, analyzing space and possible noise impact, and formulating potential resolutions. The need to develop this three dimensional approach is essential due to the finite capacity of airspace for managing and planning a route system, including airport facilities. It appears that these data bases can be integrated effectively into a strategic aircraft noise routing system which should be developed as soon as possible, as part of a proactive plan applied to our FAA controlled navigable airspace for the United States.

Bragdon, C. R.↗

DOE OSTI2021

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency &lE; 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)

Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.

14 SOLAR ENERGY↗

NASA NTRS2023

Fifteen Years of Millimeter Accuracy Lunar Laser Ranging with APOLLO: Data Reduction and Calibration

The Apache Point Lunar Laser-ranging Operation (APOLLO) has been collecting lunar range measurements for 15 yr at millimeter accuracy. The median nightly range uncertainty since 2006 is 1.7 mm. A recently added Absolute Calibration System (ACS), providing an independent assessment of APOLLO system accuracy and the capability to correct lunar range data, revealed a ∼0.4% (10 ps) systematic error in the calibration of one piece of hardware that has been present for the entire history of APOLLO. The application of ACS-based timing corrections suggests systematic errors are reduced to <1 mm, such that overall data accuracy and precision are both ∼1 mm. This paper describes the processing of APOLLO/ACS data that converts photon-by-photon range measurements into the aggregated normal points that are used for science analyses. Additionally, we present methodologies to estimate timing corrections for range data lacking contemporaneous ACS photons, including range data collected prior to installation of the ACS. We also provide access to the full 15 yr archive of APOLLO normal points (2006 April 6–2020 December 27).

LLR↗

DOE OSTI2022

Assessing the large-scale drivers of precipitation in the northeastern United States via linear orthogonal decomposition

This study examines the linear orthogonal modes associated with monthly precipitation in the northeastern United States, from CESM1 LENS (35 ensemble members, 1979–2005) and two reanalysis datasets (ERA5, 1950–2018 and NOAA-CIRES-DOE 20CRv3, 1950–2015). Calendar months are aggregated together, and any linear trends in data are removed. Using region-averaged precipitation anomaly time series and monthly anomalies for several global 2D atmospheric fields, a linear orthogonal decomposition method is implemented to iteratively extract time series (based on field and geographic location) of absolute maximum correlation. Linear modes associated with this method are then projected onto the full set of 2D fields to provide physical insight into the mechanisms involved in generating precipitation. In this region, the first mode is associated with vapor transport from the Atlantic seaboard, the second mode is characterized by westward vapor transport associated with extratropical cyclones, and the third mode captures vapor transport from the Gulf of Mexico during the fall and winter. However, the third mode is less robust in the spring and summer. Results are generally consistent across the datasets, and applying multiple linear regression with the linear modes to predict the precipitation anomalies produces R-squared values of around 0.54–0.65 for CESM1 LENS, and around 0.58–0.88 for reanalysis, with the lowest values generally in the spring and late summer. The influence of low-frequency climate variability on the modes is considered for CESM1 LENS, and the modes in late winter can be predicted with some success via a combination of several, prominent large-scale teleconnection patterns.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Parametric dynamic mode decomposition for reduced order modeling

Dynamic Mode Decomposition (DMD) is a model-order reduction approach, whereby spatial modes of fixed temporal frequencies are extracted from numerical or experimental data sets. The DMD low-rank or reduced operator is typically obtained by singular value decomposition of the temporal data sets. For parameter-dependent models, as found in many multi-query applications such as uncertainty quantification or design optimization, the only parametric DMD technique developed was a stacked approach, with data sets at multiple parameter values were aggregated together, increasing the computational work needed to devise low-rank dynamical reduced-order models. Here in this paper, we present two novel approach to carry out parametric DMD: one based on the interpolation of the reduced-order DMD eigen-pair and the other based on the interpolation of the reduced DMD (Koopman) operator. Numerical results are presented for diffusion-dominated nonlinear dynamical problems, including a multiphysics radiative transfer example. All three parametric DMD approaches are compared.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Linking structural and rheological memory in disordered soft materials

Linking the macroscopic flow properties and nanoscopic structure is a fundamental challenge to understanding, predicting, and designing disordered soft materials. Under small stresses, these materials are soft solids, while larger loads can lead to yielding and the acquisition of plastic strain, which adds complexity to the task. In this work, we connect the transient structure and rheological memory of a colloidal gel under cyclic shearing across a range of amplitudes via a generalized memory function using rheo-X-ray photon correlation spectroscopy (rheo-XPCS). Our rheo-XPCS data show that the nanometer scale aggregate-level structure recorrelates whenever the change in recoverable strain over some interval is zero. The macroscopic recoverable strain is therefore a measure of the nano-scale structural memory. We further show that yielding in disordered colloidal materials is strongly heterogeneous and that memories of prior deformation can exist even after the material has been subjected to flow.

Kamani, Krutarth M. [Univ. of Illinois at Urbana-C↗

DOE OSTI2026

ComStock Measure Documentation: High-Efficiency Rooftop Unit

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - high-efficiency rooftop unit (RTU) - and briefly introduces key results. The full public data set can be accessed on the Comstock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2026

ComStock Measure Documentation: Variable-Speed Pumps

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - variable speed pumps - and briefly introduces key results. The full public data set can be accessed on the ComStock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1974

Earth Observatory Satellite system definition study. Report no. 3: Design/cost tradeoff studies. Appendix A: EOS program WBS dictionary. Appendix B: EOS mission functional analysis

The work breakdown structure (WBS) dictionary for the Earth Observatory Satellite (EOS) is defined. The various elements of the EOS program are examined to include the aggregate of hardware, computer software, services, and data required to develop, produce, test, support, and operate the space vehicle and the companion ground data management system. A functional analysis of the EOS mission is developed. The operations for three typical EOS missions, Delta, Titan, and Shuttle launched are considered. The functions were determined for the top program elements, and the mission operations, function 2.0, was expanded to level one functions. Selection of ten level one functions for further analysis to level two and three functions were based on concern for the EOS operations and associated interfaces.

Source record↗

NASA NTRS2011

What We've Learned from Approximately 12 Years of MISR Aerosol Observations

The NASA Earth Observing System's Multi-angle Imaging SpectroRadiometer (MISR) has been imaging the planet about once per week for nearly 12 years. These data contain a wealth of information about the current state, as well as the variability of Earth's surface and atmosphere. For airborne particles specifically, horizontal distributions of aerosol optical depth, aerosol type, and near-source aerosol plume height are now routinely derived. The results are being applied to a wide range of problems, from constraining aerosol direct radiative forcing of climate and assessing its seasonal and secular trends, to material transports of smoke and dust, to mapping volcanic ash plumes and near-surface aerosol pollution. The value of these data is multiplied many times by combining it with data from other sources, and using the aggregate to constrain models. This talk will summarize the latest developments, and will also discuss current work aimed at making further advancements in these areas.

Kahn, Ralph↗

NASA NTRS2012

Linking NASA Environmental Data with a National Public Health Cohort Study and a CDC On-Line System to Enhance Public Health Decision Making

The overall goal of this study is to address issues of environmental health and enhance public health decision making by utilizing NASA remotely-sensed data and products. This study is a collaboration between NASA Marshall Space Flight Center, Universities Space Research Association (USRA), the University of Alabama at Birmingham (UAB) School of Public Health and the Centers for Disease Control and Prevention (CDC) National Center for Public Health Informatics. The objectives of this study are to develop high-quality spatial data sets of environmental variables, link these with public health data from a national cohort study, and deliver the linked data sets and associated analyses to local, state and federal end-user groups. Three daily environmental data sets were developed for the conterminous U.S. on different spatial resolutions for the period 2003-2008: (1) spatial surfaces of estimated fine particulate matter (PM2.5) exposures on a 10-km grid utilizing the US Environmental Protection Agency (EPA) ground observations and NASA s MODerate-resolution Imaging Spectroradiometer (MODIS) data; (2) a 1-km grid of Land Surface Temperature (LST) using MODIS data; and (3) a 12-km grid of daily Solar Insolation (SI) and maximum and minimum air temperature using the North American Land Data Assimilation System (NLDAS) forcing data. These environmental datasets were linked with public health data from the UAB REasons for Geographic and Racial Differences in Stroke (REGARDS) national cohort study to determine whether exposures to these environmental risk factors are related to cognitive decline and other health outcomes. These environmental national datasets will also be made available to public health professionals, researchers and the general public via the CDC Wide-ranging Online Data for Epidemiologic Research (WONDER) system, where they can be aggregated to the county, state or regional level as per users need and downloaded in tabular, graphical, and map formats. The linkage of these data provides a useful addition to CDC WONDER, allowing public health researchers and policy makers to better include environmental exposure data in the context of other health data available in this online system. It also substantially expands public access to NASA data, making their use by a wide range of decision makers feasible.

Al-Hamdan, Mohammad↗

DOE OSTI2026

IntraShuffler: A Privacy Preserving Framework for Heterogeneous DP Federated Learning

Heterogeneous Differential Privacy (HDP) in Federated Learning (FL) allows clients to select individual privacy budgets () according to institutional policies and data sensitivity. In practice, many HDP-FL systems employ -aware server aggregation to improve model utility by re-weighting client updates according to their declared privacy budgets. However, gradient updates in FL retain structural patterns induced by non-independent and identically-distributed (non-IID) data, and these additional signals exposed by -aware aggregation create new opportunities for inference by an honest-but-curious server. In this work, we first show that a server equipped with gradient denoising and surrogate modeling can mount a Privacy Inference Attack that infers distributional attributes of clients and links updates from the same client across training rounds, measured via surrogate inference accuracy and linkage success, under realistic knowledge constraints. The Shuffle-Model has been widely studied as a defense against such inference risks by anonymizing update sources, but it is fundamentally incompatible with HDP-FL -aware aggregation. To address this challenge, we propose IntraShuffler, a middleware defense framework designed for HDP-FL systems. IntraShuffler introduces a privacy-aware shuffling mechanism that groups clients into privacy-compatible buckets and performs parameter-level shuffling within each bucket to disrupt persistent gradient structure while preserving -aware aggregation. Experiments across four different datasets show that IntraShuffler reduces gradient recoverability by over 60% and decreases surrogate inference accuracy from 0.78 to 0.33 while maintaining comparable model utility across multiple FL aggregation rules.

Riya, Farhin Farhad [ORNL]↗

DOE OSTI2025

Front-of-Meter Model Results

These files contains aggregations of key variables from the NREL Distributed Wind Futures Study using full parcel level data. These variables describe total technical and economic potential for distributed wind turbine deployment. Aggregations are available at the (1) county, (2) zipcode (zip code tabulation area or zcta), and (3) US Census block group level. Each scenario is coded with the scenario name (e.g., baseline) and year (e.g., 2022). Those files postfixed with 'econpot' contain results for only those parcels that are economically viable while the files postfixed with 'techpot' include results for all parcels that are technically feasible. Hence these correspond to technoeconomic and technical potential respectively. The data are available as CSV or Geopackage. Columns in the files are as follows: * geoid: geographic identifier (FIPS code or similar) * min_techpot_sum_kw: technical potential for all parcels in kW using turbines downsized to demand when appropriate * max_techpot_sum_kw: technical potential for all parcels in kW without downsizing turbines * aep_sum_kwh: annual energy production estimate in kWh * cf_mean_ratio: mean capacity factor * lcoe_mean_cents_per_kwh: mean levelized cost of energy for parcels in geography in cents per kWh * lcoe_std_cents_per_kwh: standard deviation of the above * parcel_area_sum_acres: total area of viable parcels in acres * n_turbines: number of cited turbines (one per viable parcel currently) Note: These are preliminary results from the full-parcel 2024 update of the Distributed Wind Energy Futures study. Please take care when making use of the data, and feel free to contact the team with any questions. Full documentation in support of these data is in progress and will follow.

17 WIND ENERGY↗

DOE OSTI2025

Behind-the-Meter Model Results

17 WIND ENERGY↗

DOE OSTI2024

Individual Data Sparsity in Smart Thermostat Big Data: Impacts on Modeling Thermostat Use Behavior Dynamics

This study explores the impacts of the sparsity of individual thermostat interaction data on modeling thermostat use behavior dynamics using a dataset of over 100,000 smart thermostats. In developing a data-driven model of Thermal Frustration Theory (TFT), we investigate the challenges and trade-offs in clustering occupant data to enhance predictive accuracy. Our findings reveal that a single, aggregated model fails to capture the diversity of occupant behaviors, resulting in extremely poor prediction performance. Conversely, excessive clustering exacerbates data sparsity, undermining model reliability. By identifying an optimal clustering strategy, we achieve a balance that significantly improves the prediction of manual setpoint changes during demand response (DR) events, enhancing energy management and occupant comfort

Fannon, David↗