Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗

International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)

Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.

14 SOLAR ENERGY↗

The Properties of Mortar Mixtures Blended with Natural, Crushed, and Recycled Fine Aggregates for Building Construction Materials

In this research, the possible applicability of fine aggregates blended with natural, crushed, and recycled fine aggregate are discussed. The fresh and hardened properties of mortar using blended fine aggregates are monitored depending on various blending ratio of fine aggregates. Newly developed ternary diagram was also utilized for better interpretation of the data. It was found that air content increased and unit weight decreased as recycled fine aggregate content increased. With moisture type processing of recycled fine aggregate, the mortar flow was not negatively affected by increase in the recycled fine aggregate content. The ternary diagram is found to be an effective graphical presentation tool that can be used for the quality evaluation of mortar using blended fine aggregate.

36 MATERIALS SCIENCE↗

Assessing the large-scale drivers of precipitation in the northeastern United States via linear orthogonal decomposition

This study examines the linear orthogonal modes associated with monthly precipitation in the northeastern United States, from CESM1 LENS (35 ensemble members, 1979–2005) and two reanalysis datasets (ERA5, 1950–2018 and NOAA-CIRES-DOE 20CRv3, 1950–2015). Calendar months are aggregated together, and any linear trends in data are removed. Using region-averaged precipitation anomaly time series and monthly anomalies for several global 2D atmospheric fields, a linear orthogonal decomposition method is implemented to iteratively extract time series (based on field and geographic location) of absolute maximum correlation. Linear modes associated with this method are then projected onto the full set of 2D fields to provide physical insight into the mechanisms involved in generating precipitation. In this region, the first mode is associated with vapor transport from the Atlantic seaboard, the second mode is characterized by westward vapor transport associated with extratropical cyclones, and the third mode captures vapor transport from the Gulf of Mexico during the fall and winter. However, the third mode is less robust in the spring and summer. Results are generally consistent across the datasets, and applying multiple linear regression with the linear modes to predict the precipitation anomalies produces R-squared values of around 0.54–0.65 for CESM1 LENS, and around 0.58–0.88 for reanalysis, with the lowest values generally in the spring and late summer. The influence of low-frequency climate variability on the modes is considered for CESM1 LENS, and the modes in late winter can be predicted with some success via a combination of several, prominent large-scale teleconnection patterns.

54 ENVIRONMENTAL SCIENCES↗

Parametric dynamic mode decomposition for reduced order modeling

Dynamic Mode Decomposition (DMD) is a model-order reduction approach, whereby spatial modes of fixed temporal frequencies are extracted from numerical or experimental data sets. The DMD low-rank or reduced operator is typically obtained by singular value decomposition of the temporal data sets. For parameter-dependent models, as found in many multi-query applications such as uncertainty quantification or design optimization, the only parametric DMD technique developed was a stacked approach, with data sets at multiple parameter values were aggregated together, increasing the computational work needed to devise low-rank dynamical reduced-order models. Here in this paper, we present two novel approach to carry out parametric DMD: one based on the interpolation of the reduced-order DMD eigen-pair and the other based on the interpolation of the reduced DMD (Koopman) operator. Numerical results are presented for diffusion-dominated nonlinear dynamical problems, including a multiphysics radiative transfer example. All three parametric DMD approaches are compared.

97 MATHEMATICS AND COMPUTING↗

Toward Molecule-Specific Geochemistry of Heavy Ends: Application to the Upstream Oil Industry

Detailed molecular analysis of the composition of petroleum is typically performed on only the light (volatile) fraction, using analyses involving gas chromatography (GC). Heavier (nonvolatile) components such as kerogen, bitumen, and asphaltenes, which are relevant to conventional reservoirs and particularly to unconventional gas shale and tight oil formations, are not amenable to analysis by GC and instead are typically studied by more phenomenological techniques. This report describes a suite of analytical techniques that can provide sufficient average molecular information on the heavy ends (defined as the nonvolatile fraction) to be useful for a variety of industrial applications, expanding the discipline of molecule-specific geochemistry into the non-GC amenable fractions of petroleum materials. Heavy ends are composed of the elements C, H, N, S, and O, and the bonding environments of each can be measured by various solid state spectroscopies: nuclear magnetic resonance (NMR) and infrared (IR) spectroscopies for C, H, and O; X-ray absorption near-edge structure (XANES) spectroscopy for S and N. Additionally, advances in mass spectrometry, including techniques such as laser desorption laser ionization mass spectrometry (L2MS), can be used to measure the molecular weight and dominant molecular architecture of asphaltenes. Moreover, higher-order structures can be measured: samples of kerogen with representative pore geometry can be prepared, and those pore systems are analyzed with techniques such as X-ray diffraction and neutron scattering. Colloidal structures of asphaltenes can be measured with a variety of techniques, which reveal a hierarchical aggregation structure. These analyses provide sufficient structural data to enable structure–property and structure–function relationships to be determined, which can be applied to the characterization of conventional and unconventional resources. In gas shale and tight oil formations, which contain abundant inorganic minerals as well as kerogen, diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS) is used for simultaneous analysis of mineralogy, kerogen content, and maturity, typically performed at the wellsite. In conventional reservoirs, spatial variations in the asphaltene concentration measured by downhole fluid analysis logging tools are being interpreted with a recently developed equation of state to address upstream issues such as reservoir connectivity, tar mat formation, and fault block migration. Furthermore, as more heavy end analyses are developed, further scientific knowledge and industrial applications of heavy end geochemistry are anticipated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Linking structural and rheological memory in disordered soft materials

Linking the macroscopic flow properties and nanoscopic structure is a fundamental challenge to understanding, predicting, and designing disordered soft materials. Under small stresses, these materials are soft solids, while larger loads can lead to yielding and the acquisition of plastic strain, which adds complexity to the task. In this work, we connect the transient structure and rheological memory of a colloidal gel under cyclic shearing across a range of amplitudes via a generalized memory function using rheo-X-ray photon correlation spectroscopy (rheo-XPCS). Our rheo-XPCS data show that the nanometer scale aggregate-level structure recorrelates whenever the change in recoverable strain over some interval is zero. The macroscopic recoverable strain is therefore a measure of the nano-scale structural memory. We further show that yielding in disordered colloidal materials is strongly heterogeneous and that memories of prior deformation can exist even after the material has been subjected to flow.

Kamani, Krutarth M. [Univ. of Illinois at Urbana-C↗

ULTRAVIOLET EXTINCTION AT HIGH GALACTIC LATITUDES

In order to study the properties and effects of high Galactic latitude dust, we present an analysis of 373,303 galaxies selected from the Galaxy Evolution Explorer All-Sky Survey and Wide-field Infrared Explorer All-Sky Data Release. By examining the variation in aggregate ultraviolet colors and number density of these galaxies, we measure the extinction curve at high latitude. We additionally consider a population of spectroscopically selected galaxies from the Sloan Digital Sky Survey to measure extinction in the optical. We find that dust at high latitude is neither quantitatively nor qualitatively consistent with standard reddening laws. Extinction in the FUV and NUV is {approx}10% and {approx}35% higher than expected, with significant variation across the sky. We find that no single R{sub V} parameter fits both the optical and ultraviolet extinction at high latitude, and that while both show detectable variation across the sky, these variations are not related. We propose that the overall trends we detect likely stem from an increase in very small silicate grains in the interstellar medium.

79 ASTRONOMY AND ASTROPHYSICS↗

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Iglesias, Felix↗

MO-G-18C-05: Real-Time Prediction in Free-Breathing Perfusion MRI

Purpose: The aim is to minimize frame-wise difference errors caused by respiratory motion and eliminate the need for breath-holds in magnetic resonance imaging (MRI) sequences with long acquisitions and repeat times (TRs). The technique is being applied to perfusion MRI using arterial spin labeling (ASL). Methods: Respiratory motion prediction (RMP) using navigator echoes was implemented in ASL. A least-square method was used to extract the respiratory motion information from the 1D navigator. A generalized artificial neutral network (ANN) with three layers was developed to simultaneously predict 10 time points forward in time and correct for respiratory motion during MRI acquisition. During the training phase, the parameters of the ANN were optimized to minimize the aggregated prediction error based on acquired navigator data. During realtime prediction, the trained ANN was applied to the most recent estimated displacement trajectory to determine in real-time the amount of spatial Results: The respiratory motion information extracted from the least-square method can accurately represent the navigator profiles, with a normalized chi-square value of 0.037±0.015 across the training phase. During the 60-second training phase, the ANN successfully learned the respiratory motion pattern from the navigator training data. During real-time prediction, the ANN received displacement estimates and predicted the motion in the continuum of a 1.0 s prediction window. The ANN prediction was able to provide corrections for different respiratory states (i.e., inhalation/exhalation) during real-time scanning with a mean absolute error of < 1.8 mm. Conclusion: A new technique enabling free-breathing acquisition during MRI is being developed. A generalized ANN development has demonstrated its efficacy in predicting a continuum of motion profile for volumetric imaging based on navigator inputs. Future work will enhance the robustness of ANN and verify its effectiveness with human subjects. Research supported by National Institutes of Health National Cancer Institute Grant R01 CA159471-01.

60 APPLIED LIFE SCIENCES↗

LBNL-Load

This software package is a set of Python-based tools for forecasting future time series of electrical power demand for a granular set of customer types in California, based on historical data on individual customers' demand, combined with aggregate forecasts of demand growth.

Gerke, Brian↗

Technological Cost-Reduction Pathways for Axial-Flow Turbines in the Marine Hydrokinetic Environment.

This report considers and prioritizes potential technical cost reduction pathways for axial flow turbines designed for tidal, river, and ocean current resources. This report focuses on technical research and development cost reduction pathways related to the device technology rather than environmental monitoring or permitting opportunities. Three sources of information were utilized to understand current cost drivers and develop a list of potential cost reduction pathways: a literature review of technical work related to axial flow turbines, the U.S. Department of Energy Reference Model effort, and informal webinars and other targeted interactions with industry developers. Data from these various information sources were aggregated and prioritized with respect to potential impact on the lifetime levelized cost of energy. The four most promising cost reduction pathways include structural design optimization; improved deployment, maintenance, and recovery; system simplicity and reliability; and array optimization.

Laird, Daniel L.↗

ComStock Measure Documentation: High-Efficiency Rooftop Unit

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - high-efficiency rooftop unit (RTU) - and briefly introduces key results. The full public data set can be accessed on the Comstock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

ComStock Measure Documentation: Variable-Speed Pumps

Building on the 3-year End-Use Load Profiles project to calibrate and validate the U.S. Department of Energy's ResStock and ComStock models, this work produces national data sets that enable cities, states, utilities, and other stakeholders to answer a broad range of questions regarding their commercial building stock. ComStock is a highly granular, bottom-up model that uses various data sources, statistical sampling methods, and advanced building energy simulations to estimate the annual subhourly energy consumption of the commercial building stock across the United States. The "baseline" model intends to represent the U.S. commercial building stock as it existed in 2018. The methodology of the baseline model is discussed in the ComStock Reference Documentation. The goal of this work is to develop energy efficiency and demand flexibility measures that cover market-ready technologies and study their mass adoption impact on the baseline building stock. "Measures" refers to various "what-if" scenarios that can be applied to buildings. The results for the baseline and measure scenario simulations are published in public data sets that provide insights into building stock characteristics, operational behaviors, utility bill impacts, and annual and sub-hourly energy usage by fuel type and end use. This report describes the modeling methodology for a single ComStock measure scenario - variable speed pumps - and briefly introduces key results. The full public data set can be accessed on the ComStock data lake or via the Data Viewer at comstock.nlr.gov. The public data set enables users to create custom aggregations of results for their use case (e.g., filter to a specific county or building type).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Novel capabilities for examining and summarizing hierarchical and geographic resiliency profiles

The geospatial research and development team in the National and Homeland Security Division at Idaho National Laboratory was tasked with providing tools to derive insight from the substantial amount of data currently available - and continuously being produced - associated with the critical infrastructure of the US. This effort is in support of the Department of Homeland Security, whose mission includes the protection of this infrastructure and the enhancement of its resilience to hazards, both natural and human. We present geovisual-analytics-based approaches for analysis of vulnerabilities and resilience of critical infrastructure, designed so that decision makers, analysts, and infrastructure owners and managers can manage risk, prepare for hazards, and direct resources before and after an incident that might result in an interruption in service. Our designs are based on iterative discussions with DHS leadership and analysts, who in turn will use these tools to explore and communicate data in partnership with utility providers, law enforcement, and emergency response and recovery organizations, among others. In most cases these partners desire summaries of large amounts of data, but increasingly, our users seek the additional capability of focusing on, for example, a specific infrastructure sector, a particular geographic region, or time period, or of examining data in a variety of generalization or aggregation levels. These needs align well with tenets of in-formation-visualization design; in this paper, selected applications among those that we have designed are described and positioned within geovisualization, geovisual analytical, and information visualization frameworks.

97 MATHEMATICS AND COMPUTING↗

IntraShuffler: A Privacy Preserving Framework for Heterogeneous DP Federated Learning

Heterogeneous Differential Privacy (HDP) in Federated Learning (FL) allows clients to select individual privacy budgets () according to institutional policies and data sensitivity. In practice, many HDP-FL systems employ -aware server aggregation to improve model utility by re-weighting client updates according to their declared privacy budgets. However, gradient updates in FL retain structural patterns induced by non-independent and identically-distributed (non-IID) data, and these additional signals exposed by -aware aggregation create new opportunities for inference by an honest-but-curious server. In this work, we first show that a server equipped with gradient denoising and surrogate modeling can mount a Privacy Inference Attack that infers distributional attributes of clients and links updates from the same client across training rounds, measured via surrogate inference accuracy and linkage success, under realistic knowledge constraints. The Shuffle-Model has been widely studied as a defense against such inference risks by anonymizing update sources, but it is fundamentally incompatible with HDP-FL -aware aggregation. To address this challenge, we propose IntraShuffler, a middleware defense framework designed for HDP-FL systems. IntraShuffler introduces a privacy-aware shuffling mechanism that groups clients into privacy-compatible buckets and performs parameter-level shuffling within each bucket to disrupt persistent gradient structure while preserving -aware aggregation. Experiments across four different datasets show that IntraShuffler reduces gradient recoverability by over 60% and decreases surrogate inference accuracy from 0.78 to 0.33 while maintaining comparable model utility across multiple FL aggregation rules.

Riya, Farhin Farhad [ORNL]↗

Front-of-Meter Model Results

These files contains aggregations of key variables from the NREL Distributed Wind Futures Study using full parcel level data. These variables describe total technical and economic potential for distributed wind turbine deployment. Aggregations are available at the (1) county, (2) zipcode (zip code tabulation area or zcta), and (3) US Census block group level. Each scenario is coded with the scenario name (e.g., baseline) and year (e.g., 2022). Those files postfixed with 'econpot' contain results for only those parcels that are economically viable while the files postfixed with 'techpot' include results for all parcels that are technically feasible. Hence these correspond to technoeconomic and technical potential respectively. The data are available as CSV or Geopackage. Columns in the files are as follows: * geoid: geographic identifier (FIPS code or similar) * min_techpot_sum_kw: technical potential for all parcels in kW using turbines downsized to demand when appropriate * max_techpot_sum_kw: technical potential for all parcels in kW without downsizing turbines * aep_sum_kwh: annual energy production estimate in kWh * cf_mean_ratio: mean capacity factor * lcoe_mean_cents_per_kwh: mean levelized cost of energy for parcels in geography in cents per kWh * lcoe_std_cents_per_kwh: standard deviation of the above * parcel_area_sum_acres: total area of viable parcels in acres * n_turbines: number of cited turbines (one per viable parcel currently) Note: These are preliminary results from the full-parcel 2024 update of the Distributed Wind Energy Futures study. Please take care when making use of the data, and feel free to contact the team with any questions. Full documentation in support of these data is in progress and will follow.

17 WIND ENERGY↗

Behind-the-Meter Model Results

These files contains aggregations of key variables from the NREL Distributed Wind Futures Study using full parcel level data. These variables describe total technical and economic potential for distributed wind turbine deployment. Aggregations are available at the (1) county, (2) zipcode (zip code tabulation area or zcta), and (3) US Census block group level. Each scenario is coded with the scenario name (e.g., baseline) and year (e.g., 2022). Those files postfixed with 'econpot' contain results for only those parcels that are economically viable while the files postfixed with 'techpot' include results for all parcels that are technically feasible. Hence these correspond to technoeconomic and technical potential respectively. The data are available as CSV or Geopackage. Columns in the files are as follows: * geoid: geographic identifier (FIPS code or similar) * min_techpot_sum_kw: technical potential for all parcels in kW using turbines downsized to demand when appropriate * max_techpot_sum_kw: technical potential for all parcels in kW without downsizing turbines * aep_sum_kwh: annual energy production estimate in kWh * cf_mean_ratio: mean capacity factor * lcoe_mean_cents_per_kwh: mean levelized cost of energy for parcels in geography in cents per kWh * lcoe_std_cents_per_kwh: standard deviation of the above * parcel_area_sum_acres: total area of viable parcels in acres * n_turbines: number of cited turbines (one per viable parcel currently)

17 WIND ENERGY↗