Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

A riming‐dependent parameterization of scattering by snowflakes using the self‐similar Rayleigh–Gans approximation

Abstract Riming is a key process of precipitation formation in ice‐containing clouds, but quantifying riming from observations is challenging, limiting our ability to evaluate the riming process in numerical weather models. One challenge for radar observations is that riming changes both the physical properties (mass, area cross‐section) and scattering properties of ice particles. These changes need to be implemented consistently as a function of riming in radar forward operators, which are required for retrievals and model evaluation in observation space. In this study, mass–size, cross‐section area–size, and backscattering cross‐section relations are developed as a function of the normalized rime mass for aggregates composed of various monomer types (columns, dendrites, needles, plates, and rosettes). The proposed framework allows us to simulate scattering properties of aggregated ice particles consistently as a function of riming in retrievals and radar forward operators. The parameterizations are developed from a large data set of simulated rimed aggregates of different sizes and monomer crystal types. The backscattering cross‐section parameterization (the “riming‐dependent parameterization”) is evaluated for radar frequencies of 35.6 and 94.0 GHz and is based on the Self‐Similar Rayleigh–Gans approximation (SSRGA), which is increasingly used to calculate microwave scattering of ice crystals and snowflakes. Compared with parameterizations from the literature that do not consider riming, the riming‐dependent parameterization leads to significantly smaller biases in terms of backscattering cross‐section. When using the particle masses and scattering properties of the individual particles simulated by the aggregation and riming model as a reference, the bias of our parameterization is below 1 dB when integrating over an exponential particle size distribution with sizes from 0.1–10 mm.

54 ENVIRONMENTAL SCIENCES↗

Transferable Kinetic Monte Carlo Models with Thousands of Reactions Learned from Molecular Dynamics Simulations

Molecular dynamics (MD) simulation of complex chemistry typically involves thousands of atoms propagating over millions of time steps, generating a wealth of data. Traditionally these data are used to calculate some aggregate properties of the system and then discarded, but we propose that these data can be reused to study related chemical systems. Using approximate chemical kinetic models and methods from statistical learning, we study hydrocarbon chemistries under extreme thermodynamic conditions. We discover that a single MD simulation can contain sufficient information about reactions and rates to predict the dynamics of related yet different chemical systems using kinetic Monte Carlo (KMC) simulation. Our learned KMC models identify thousands of reactions and run four orders of magnitude faster than MD. In conclusion, the transferability of these models suggests that we can viably reuse data from existing MD simulations to accelerate future simulation studies and reduce the number of new MD simulations required.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Quota management in dCache or making a perfectly normal file system normal

dCache (https://dcache.org) is a highly scalable storage system providing location-independent access to data. The data are stored across multiple data servers as complete files presented to the end-user via a single-rooted namespace. From its inception, dCache has been designed as a caching disk buffer to a tertiary tape storage system with the assumption that the latter has virtually unlimited capacity. dCache can also be configured as a disk-only storage system with no tape backend. Owing to the idea that a tape resource is infinite, or purely physically limited by budget considerations, the system has never provided for any restrictions on how much data can be stored on tape. Likewise, in the disk-only configuration, the capacity of the system is only limited by the aggregate disk capacity of the data servers. In a multi-user environment, however, this has become problematic. This presentation will describe the design and implementation of a user- and group-based quota system, that allows to manage tape and disk space allocations, as part of dCache namespace.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Use of Semantic Technology to Create Curated Data Albums

One of the continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available online. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the data sets they need can obtain the specific files using these systems. However, in cases where researchers are interested in studying an event of research interest, they must manually assemble a variety of relevant data sets by searching the different distributed data systems. Consequently, there is a need to design and build specialized search and discover tools in Earth science that can filter through large volumes of distributed online data and information and only aggregate the relevant resources needed to support climatology and case studies. This paper presents a specialized search and discovery tool that automatically creates curated Data Albums. The tool was designed to enable key elements of the search process such as dynamic interaction and sense-making. The tool supports dynamic interaction via different modes of interactivity and visual presentation of information. The compilation of information and data into a Data Album is analogous to a shoebox within the sense-making framework. This tool automates most of the tedious information/data gathering tasks for researchers. Data curation by the tool is achieved via an ontology-based, relevancy ranking algorithm that filters out nonrelevant information and data. The curation enables better search results as compared to the simple keyword searches provided by existing data systems in Earth science.

Ramachandran, Rahul↗

Use of Semantic Technology to Create Curated Data Albums

One of the continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available online. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the data sets they need can obtain the specific files using these systems. However, in cases where researchers are interested in studying an event of research interest, they must manually assemble a variety of relevant data sets by searching the different distributed data systems. Consequently, there is a need to design and build specialized search and discovery tools in Earth science that can filter through large volumes of distributed online data and information and only aggregate the relevant resources needed to support climatology and case studies. This paper presents a specialized search and discovery tool that automatically creates curated Data Albums. The tool was designed to enable key elements of the search process such as dynamic interaction and sense-making. The tool supports dynamic interaction via different modes of interactivity and visual presentation of information. The compilation of information and data into a Data Album is analogous to a shoebox within the sense-making framework. This tool automates most of the tedious information/data gathering tasks for researchers. Data curation by the tool is achieved via an ontology-based, relevancy ranking algorithm that filters out non-relevant information and data. The curation enables better search results as compared to the simple keyword searches provided by existing data systems in Earth science.

Ramachandran, Rahul↗

A Lightweight I/O Scheme to Facilitate Spatial and Temporal Queries of Scientific Data Analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data post-processing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Temporal Queries↗

A Suite of Mechanistic Epidemiological Decision Support Tools

Objective: We will demonstrate tools that allow mechanistic contraints on disease progression and epidemic spread to play off against interventions, mitigation, and control measures. The fundamental mechanisms of disease progression and epidemic spread provide important constraints on interpreting changing epidemic cases counts with time and geography in the context of on-going interventions, mitigations, and controls. Models such as these that account for the effects of human actions can also allow evaluation of the importance of categories of epidemic and disease controls. Introduction: We present the EpiEarly, EpiGrid, and EpiCast tools for mechanistically-based biological decision support. The range of tools covers coarse-, medium-, and fine-grained models. The coarse-grained, aggregated time-series only data tool (EpiEarly) provides a statistic quantifying epidemic growth potential and associated uncertainties. The medium grained, geographically-resolved model (EpiGrid) is based on differential equation type simulations of disease and epidemic progression in the presence of various human interventions geared toward understanding the role of infection control, early vs. late diagnosis, vaccination, etc. in outbreak control. A fine-grained hybrid-agent epidemic model (EpiCast) with diurnal agent travel and contagion allows the analysis of the importance of contact-networks, travel, and detailed intervention strategies for the control of outbreaks and epidemics. Methods: We use three types of methods for simulation and analysis. They are: (1) Bayesian and regression methods allowing estimation of the basic reproductive number from case-count data; (2) ordinary-differential equation integration with modifications to account for discreteness of disease spread when case counts are small (we include space- and time-dependent effects); and (3) methods that hybridize agent-based travel, mixing, and disease progression with nested-mass action contagion (i.e. not fully agent-based). From the perspective of decision support, the crucial feature of mechanistic infectious epidemiological models is a way to capture the human interventions that determine epidemic outcome. Categorizing types of mitigation into those that change the force of infection, and those that branch disease progression allows a common framework that can be extended from medium-grained models through fine-grained. Our canonical example is our EpiGrid tool which allows for the modulation of the force of infection (i.e. contagion) with time (and potentially space), the vaccination of a susceptible population in a geographically-targeted manner, movement controls, and branching our disease progression model to account for early- vs. late-intervention during host disease progression.ResultsWe will present analysis of diseases that exemplify the various aspects of analysis in support of outbreak and epidemic control. Human and animal diseases relevant to this demonstration include rinderpest, avian influenza, and measles. We will begin with EpiEarly’s estimate of epidemic potential using aggregated time-dependent case-count data. The key observation for EpiEarly is that under a wide range of situations a disease's reproductive number should be generalized to a distribution of possibilities to account for inherent randomness and other factors (including the variability of a disease contact network). We will then continue with a demonstration of EpiGrid’s capabilities for understanding and modelling the role of interventions including contagion control (the force of infection), treatment (changing disease progression and infectiousness depending on treatment), vaccination, culling, and movement controls. We will briefly touch on the capabilities of EpiCast for more detailed analysis of specific intervention strategies. Conclusions: We will demonstrate examples where modeling either contributed or plausibly would contribute to informing epidemic and outbreak control constrained by the possibilities of the underlying epidemic and disease dynamics.

Geospatial↗

A lightweight I/O scheme to facilitate spatial and temporal queries of scientific data analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data postprocessing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Tian, Yuan↗

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗

Subtle Effects of Aliphatic Alcohol Structure on Water Extraction and Solute Aggregation in Biphasic Water/ n -Dodecane

Organic phase aggregation behavior of 1-octanol and its structural isomer, 2-ethylhexanol, in a biphasic n-dodecane water system is studied with a combination of physical measurement, small-angle X-ray scattering (SAXS), and atomistic molecular dynamic simulations. Physical properties of the organic phases are probed following their mixing and equilibration with immiscible water phases. Studies reveal that the interfacial tension decreases as a function of increasing alcohol concentration over the solubility range of the alcohol with no evidence for a critical aggregate concentration (cac). An uptake of water into the organic phases is quantified, as a function of alcohol content, by Karl Fischer titrations. The extraction of water into dodecane was further assessed as a function of alcohol concentration via the slope-analysis method sometimes employed in chemical separations. This provides a qualitative understanding of solute (water/alcohol) aggregation in the organic phase. The physical results are supported by analyses of SAXS data that reveals an emergence of aggregates in n-dodecane at elevated alcohol concentrations. The observed aggregate structure is dependent on the alcohol tail group geometry, consistent with surfactant packing parameter. The formation of these aggregates is discussed at a molecular level, where alcohol-alcohol and alcohol-water H-bonding interactions likely dominate the occurrence and morphology of the aggregates.

Knight, Andrew W.↗

Needs for registration and rectification of satellite imagery for land use and land cover and hydrologic applications

The use of satellite imagery and data for registration of land use, land cover and hydrology was discussed. Maps and aggregations are made from existing the data in concert with other data in a geographic information system. Basic needs for registration and rectification of satellite imagery related to specifying, reformatting, and overlaying the data are noted. It is found that the data are sufficient for users who must expand much effort in registering data.

Gaydos, L.↗

A numerical study of soot aggregate formation in a laminar coflow diffusion flame

Soot aggregate formation in a two-dimensional laminar coflow ethylene/air diffusion flame is studied with a pyrene-based soot model, a detailed sectional aerosol dynamics model, and a detailed radiation model. The chemical kinetic mechanism describes polycyclic aromatic hydrocarbon formation up to pyrene, the dimerization of which is assumed to lead to soot nucleation. The growth and oxidation of soot particles are characterized by the HACA surface mechanism and pyrene-soot surface condensation. The mass range of the solid soot phase is divided into thirty-five discrete sections and two equations are solved in each section to model the formation of the fractal-like soot aggregates. The coagulation model is improved by implementing the aggregate coagulation efficiency. Several physical processes that may cause sub-unitary aggregate coagulation efficiency are discussed. Their effects on aggregate structure are numerically investigated. The average number of primary soot particles per soot aggregate n{sub p} is found to be a strong function of the aggregate coagulation efficiency. Compared to the available experimental data, n{sub p} is well reproduced with a constant 20% aggregate coagulation efficiency. The predicted axial velocity, OH mole fraction, and C{sub 2}H{sub 2} mole fraction are validated against experimental data in the literature. Reasonable agreements are obtained. Finally, a sensitivity study of the effects of particle coalescence on soot volume fraction and soot aggregate nanostructure is conducted using a coalescence cutoff diameter method. (author)

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Decoding Ethiopian Abodes: Towards Classifying Buildings by Occupancy Type Using Footprint Morphology

Building occupancy classification plays a crucial role in urban planning, disaster management, and population modeling. Traditional methods often require extensive field surveys or detailed datasets, which can be time-consuming, expensive, and may yield incomplete or erroneous data. In this paper, we present a novel approach for classifying buildings as residential or non-residential using only building footprint data. By extracting geometric shape derivatives that characterize building morphology, we developed a high-accuracy classification model employing a combination of unsupervised and supervised learning methods. We utilized open-source data from Open Street Map, aggregating it to create binary labels for buildings based on their respective human use type. Our approach demonstrates the potential for scalability without the need for additional data sources other than building footprints and labels, offering a more efficient solution for building occupancy classification.

Adams, Daniel↗

The Manufacturing Data and Machine Learning Platform: Enabling Real-time Monitoring and Control of Scientific Experiments via IoT

IoT devices and sensor networks present new opportunities for measuring, monitoring, and guiding scientific experiments. Sensors, cameras, and instruments can be combined to provide previously unachievable insights into the state of ongoing experiments. However, IoT devices can vary greatly in the type, volume, and velocity of data they generate, making it challenging to fully realize this potential. Indeed, synergizing diverse IoT data streams in near-real time can require the use of machine learning (ML). In addition, new tools and technologies are required to facilitate the collection, aggregation, and manipulation of sensor data in order to simplify the application of ML models and in turn, fully realize the utility of IoT devices in laboratories. Here we will demonstrate how the use of the Argonne-developed Manufacturing Data and Machine Learning (MDML) platform can analyze and use IoT devices in a manufacturing experiment. MDML is designed to standardize the research and operational environment for advanced data analytics and AI-enabled automated process optimization by providing the infrastructure to integrate AI in cyber-physical systems for in situ analysis. We will show that MDML is capable of processing diverse IoT data streams, using multiple computing resources, and integrating ML models to guide an experiment.

Machine Learning↗

Comparing Convective Self‐Aggregation in Idealized Models to Observed Moist Static Energy Variability Near the Equator

Abstract Idealized convection‐permitting simulations of radiative‐convective equilibrium have become a popular tool for understanding the physical processes leading to horizontal variability of tropical water vapor and rainfall. However, the applicability of idealized simulations to nature is still unclear given that important processes are typically neglected, such as lateral water vapor advection by extratropical intrusions, or interactive ocean coupling. Here, we exploit spectral analysis to compactly summarize the multiscale processes supporting convective aggregation. By applying this framework to high‐resolution reanalysis data and satellite observations in addition to idealized simulations, we compare convective‐aggregation processes across horizontal scales and data sets. The results affirm the validity of the radiative‐convective equilibrium simulations as an analogy to the real world. Column moist static energy tendencies share similar signs and scale selectivity in convection‐permitting models and observations: Radiation increases variance at wavelengths above 1,000 km, while advection damps variance across wavelengths, and surface fluxes mostly reduce variance between 1,000 and 10,000 km.

Beucler, Tom↗

De-Risking Large-Scale PV Systems Through Data Analytics

Large-scale photovoltaic system performance analysis is being conducted within the US Department of Energy's-sponsored PV Fleet Performance Data Initiative. This collaboration with commercial PV system owners collects and evaluates PV field performance data, and provides reports on aggregated results. Drawing on over 2200 sites across the US and over 24,000 separate PV inverters we have collected in excess of 8.3 gigawatts (GW) of performance data, representing 6-7% of the entire US installed PV capacity. A mixture of utility-scale and large commercial systems are represented, averaging 4.1 megawatts (MW) in size and 5 years in age. Initial results show average system degradation rates at -0.75% / year, which is slightly higher than historically reported module-level values of -0.5%/year. We also found that the availability of systems averaged 97.7%, which is lower than the typical 99% uptime assumed by many project economic forecasts. Given these results, we compared monthly performance with expected production values, based on satellite weather data and a simple PVWatts performance model. We found that systems were performing within 10% of monthly expectation over 90% of the time, with a fleet average vs expected monthly value of 0.994. We also evaluated the impact of extreme weather events on system performance, and found a range of short-term and longer-term performance effects ranging from grid outage, system downtime, module damage and accelerated long-term degradation rate.

analysis↗

Short-Term Electric Load Forecasting for a Residential Household in Alaska

Accurate short-term load forecasting at a fine scale is essential for demand response programs, peak shaving, and load-shedding strategies [1]. While traditionally, only aggregate short-term consumption data was available, advanced metering infrastructure (AMI) now provides data at the individual consumer level [1]. There is increasing interest in utilizing this data for short-term load forecasting (from an hour to a few days) to optimize grid operations. Electricity consumption in individual households is highly influenced by residents’ personal behaviors [2]. As a result, unlike aggregate loads, electrical power usage in single households often shows significant volatility, making meter-level load forecasting for individual users particularly challenging [3], [4]. Deep learning methods, with their strong ability to model nonlinear data, have become popular for improving the accuracy of household electricity consumption forecasting [4]. Notably, the Long ShortTerm Memory (LSTM) has attracted significant attention [5], [6].

42 ENGINEERING↗

Ion aggregation in high salt solutions: Ion network versus ion cluster

The critical aggregation phenomena are ubiquitous in many self-assembling systems. Ions in high salt solutions could also spontaneously form larger ion aggregates, but their effects on hydrogen-bond structures in water have long been controversial. Here, carrying out molecular dynamics (MD) simulation studies of high salt solutions and comparing the MD simulation results with infrared absorption and pump-probe spectroscopy of O–D stretch mode of HDO in highly concentrated salt solutions and {sup 13}C-NMR chemical shift of S{sup 13}CN{sup −} in KSCN solutions, we find evidence on the onset of ion aggregate and large-scale ion-ion network formation that concomitantly breaks water hydrogen-bond structure in certain salt solutions. Despite that these experimental results cannot provide direct evidence on the three-dimensional morphological structures of ion aggregates, they serve as reference data for verifying MD simulation methods. The MD results suggest that disrupted water hydrogen-bond network is intricately intertwined with ion-ion network. This further shows morphological variation of ion aggregate structures from ion cluster to ion network in high salt solutions that are interrelated to the onset of macroscopic aggregate formation and the water hydrogen-bond structure making and breaking processes induced by Hofmeister ions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗