Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Installation Assignment Durations and Patterns of Army Personnel

The purpose of this document is to report on the development of personnel assignment duration statistical metrics and statistical descriptions for groups of Army personnel aggregated by rank, career management field, and other descriptors for 32 Continental United States (CONUS) Army installations. This information can be used for generating evidence-based measures of personnel exposure times associated with environmental health considerations at the installation and career management field levels. Assignment duration statistical descriptions were developed to provide temporal metrics for groups of personnel at Army installations as foundational data to support a variety of assessments. Statistical results of personnel assignment durations at the installation level are provided in several formats. The 50th percentile assignment duration represents the median number of days personnel in groups aggregated by rank and career management field spend at the 32 CONUS Army installations in this study. Likewise, other duration percentiles (90th) and quartiles of assignment durations for groups aggregated by rank and career management field spend at each installation were developed. Assignment path analysis provide statistical descriptions of assignment durations at Army installations throughout increments of time (e.g., 5 years, 10 years). Key groupings of assignment paths were achieved by analyzing the duration of assignments at installations in chronological order from entry throughout specified increments of military service to generate the most common assignment paths across the 32 CONUS Army installations in this study. The developed assignment duration analytics and duration statistical descriptions provide foundational temporal metrics for groups of personnel at 32 CONUS Army installations to support a variety of assessments including those by human resource planners and environmental health specialists.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Short-term electricity load forecasting: Application-driven evaluation of machine learning models across spatial and temporal scales

As we transition towards a decarbonized economy, the integration of variable renewable energy resources and new demands (e.g., electric vehicles, heat pumps) into the electricity grid places unprecedented pressure on grid operators to effectively anticipate and manage peak load. In this context, machine learning algorithms are proving to be indispensable for accurate short-term load forecasting, a crucial task to address these challenges. This study benchmarks 6 machine learning algorithms, including three neural networks and three tree-based algorithms, across various levels of spatial aggregation and time horizons (1, 4, 8, 24, and 48 h). The central contribution of this work is the comparison and analysis of load forecasting models not only based on statistical metrics, but also based on a novel error metric, which evaluates the cost implications of forecast errors for power system stakeholders. Results show that tree-based models outperform neural networks, based on statistical metrics, and yield less skewed error distributions for most spatial scales. However, through the lens of the novel error metric, neural networks are the more competitive choice, especially for forecast horizons that exceed 8 h. The study concludes with actionable recommendations to grid operators and highlights the need for the development of error metrics that link forecasting accuracy to operational costs. To promote transparency and open science, the datasets and Python code are open-sourced via a supplementary repository.

Houben, Nikolaus↗

Exploring topography-based methods for downscaling subgrid precipitation for use in Earth System Models

Topography exerts major control on land surface processes. To improve representation of topographic impacts on land surface processes, a new topography-based subgrid structure has been introduced to the Energy Exascale Earth System Model to represent the subgrid heterogeneity of surface elevation. To take advantage of the new subgrid structure for improving land surface modeling, this study explores four variations of topography-based methods for downscaling grid precipitation to the corresponding subgrids. In the first three methods, the deviation of the subgrid precipitation from the grid’s is equal to the grid precipitation multiplied by the ratio of the elevation difference between the subgrid and grid mean to a specified elevation equals to the grid elevation, the difference between the maximum and minimum subgrid elevation, and the maximum subgrid elevation, respectively. The second method limits the ratio to 0.5 to avoid extreme values on mountains and the third method accounts for the slope effect. The fourth method is similar to the third method except that the Froude Number is used to limit the numerator in a blocking regime of the ambient flow. The downscaled precipitation is evaluated using the PRISM precipitation data over the U.S. using statistical metrics. Results show that by accounting for topographic slope besides elevation, the third and fourth methods show clear advantages over the first and second methods. Furthermore, introducing the Froude Number in the fourth method improves downscaling skill and shows consistent advantages over the third method in areas with larger subgrid heterogeneity across different grids sizes.

Tesfa, Teklu K.↗

Implementation of disruptive designs for gas turbine components using direct energy deposition additive manufacturing

This research aims to develop a framework for establishing the correlation between in-situ monitoring data, process parameters, and microstructure evolution in blown-powder laser-directed energy deposition (DED) additive manufacturing (AM). To achieve this, a comprehensive manufacturing framework has been developed, spanning from in-situ data acquisition, melt-pool simulation, microstructure modeling, and statistical microstructure quantification. A machine learning-based surrogate model is constructed to predict melt pool geometry directly from in-situ coaxial camera data. The surrogate model is trained using outputs from a high-fidelity melt pool simulation, which provides accurate melt pool dimension data under varying process conditions. The predicted melt pool geometry is then used as input to a microstructure model to predict microstructural features. To rigorously compare and analyze microstructures, the project introduces statistical metrics that quantify differences based on key features such as morphology and texture. Microstructures are represented using advanced statistical descriptors including angular chord length distribution, two-point spatial statistics, orientation distribution function, and global spherical harmonic. These representations are used to compute four distinct “dissimilarity scores” that quantitatively capture differences in texture and morphology. This framework is demonstrated to enable automated calibration of simulation parameters by minimizing discrepancies between simulated and target microstructures. The technology developed in this project enables direct correlation between in-situ monitoring data and resulting microstructure, paving the way for adaptive microstructure control in metal AM. This capability strengthens the connection between process parameters and final material properties, facilitating more precise and reliable material design.

36 MATERIALS SCIENCE↗

Quantitative metrics for evaluating thermonuclear design codes and physics models applied to the National Ignition Campaign

In this study, physics models and design codes for hot dense plasmas undergoing thermonuclear burn are evaluated objectively using statistical metrics that compare the difference between calculations and data relative to the experimental uncertainties. The analysis is applied to the National Ignition Campaign (NIC) because it is relevant, comprehensive, and well documented. The statistics confirm that a key process afflicting NIC performance is mix driven by hydrodynamic instabilities as approximated here using the KL model [G. Dimonte and R. Tipton, Phys. Fluids 18, 085101 (2006)]. New physics models are also presented for instability-driven magnetic fields [B. Srinivasan et al., Phys. Rev. Lett. 108, 165002 (2012)] and the Coulomb logarithm for electron–ion thermal relaxation [G. Dimonte and J. Daligault, Phys. Rev. Lett. 101, 135001 (2008)]. The plasma-generated magnetic fields improve the agreement between code and data in a statistically significant manner by reducing the electron thermal conduction in the hot-spot via the Hall term. The Coulomb logarithm is presented in a numerically practical form that incorporates recent theoretical advances. However, it does not improve the agreement between the code and data because the thermal relaxation is so fast in non-ignited NIC plasmas that the typical 20% error does not change the tight coupling between electrons and ions. Even with these improved physics models, the one-dimensional simulations presented here are not able to describe the high-convergence NIC implosions in a statistically acceptable manner. Of the heroic multi-dimensional simulations of Clark et al. [Phys. Plasmas 23, 056302 (2016)], only those in three-dimensions satisfy the statistical acceptance criterion.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Streaming Statistics

In the context of a larger effort for in situ data analytics, there is a need to calculate basic statistics metrics (e.g., count, mean, median) online as new data points become available. Originally, the code for such online, or in other words streaming, statistics was part of the TALASS (Topological Analysis of Large- Scale Simulations) library. We isolated the relevant code and created a standalone library from it called Streaming Statistics. We also added an ability to serialize and deserialize the statistics objects so that the library can be used in distributed, task-based processing. To use the Streaming Statistics library, the user chooses a statistic, constructs an object for it, and then "adds" values to it, which means the statistic is augmented.

Shudler, Sergei↗

EVI-EnSitePy (Electric Vehicle Infrastructure – Energy Estimation and Site Optimization Tool in Python) [EVI-X Modeling Suite] [SWR-25-07]

EVI-EnSitePy is a comprehensive agent-based tool designed for the analysis and design of high-power charging sites, encompassing a wide array of site agents including Electric Vehicles (EVs), chargers, energy storage units (ESS), renewable energy resources (DER), and loads. This versatile tool offers diverse functionalities and a modular modeling approach, allowing detailed configuration of agents based on power ratings, port numbers, energy capacities, demand requirements, charger interfaces, and flexibility to customize the tool for project specific goals. By simulating agent interactions and employing various metrics, EVI-EnSitePy enables the assessment of site performance, exploration of energy management systems (EMS), and implementation of innovative EV charging policies. Utilizing EV charge schedules and arrival states, the tool performs thorough charging site simulations, with outputs consisting of agent and site-level power profiles and statistical metrics. Employing a tree graph structure, EVI-EnSitePy supports nested site structures and power distribution modeling. The tool's ability to generate charging schedules deterministically or via stochastic analysis further enhances its versatility. Through its features and capabilities, EVI-EnSitePy offers a powerful platform for informed decision-making in the realm of high-power charging site design and operation.

Jackson, Derek [National Renewable Energy Laborato↗

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗

Statistical Tests for Diagnosing Fission Source Convergence and Undersampling in Monte Carlo Criticality Calculations [Slides]

There is a very strong need for statistical testing to determine fission source convergence in Monte Carlo criticality calculations. Automation of such tests will greatly streamline and support the work carried out by NCS practitioners. Recent R&D work has shown that no single statistical test for convergence is sufficiently reliable, robust, and “guaranteed.” However, a combination of several standard statistical tests for the similarity of distributions, coupled with a high-fidelity estimate of the fission-matrix source is sufficiently robust, reliable, and repeatable that convergence can be “guaranteed.” During the course of the EG-AMCT studies, a number of statistical metrics and tests were proposed for diagnosing clustering and undersampling. None of these was robust and reliable enough for practical use in production codes. However, the expert group efforts came close. Some recent R&D work stemming from those past efforts has been very successful and promising. This Sub-Group will provide international input and collaboration on the development and implementation of statistical tests for convergence, with the primary goal of having the MC codes automatically detect convergence (or lack thereof). Newly proposed statistical tests to detect undersampling (after convergence) will also be reviewed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Stochastic multiscale modeling for quantifying statistical and model errors with application to composite materials

This paper provides a coherent and efficient computational framework for stochastic multiscale analysis of material systems in the presence of parametric uncertainties and modeling errors. Uncertainty in those model parameters that are not deduced as upscaled quantities is attributed to an uncertainty “germ”. While such parameters can appear at any scale, they are predominant at the finest analysis scale. Additional uncertainties stemming from statistical estimation, attributed to lack of data and model error, are associated with each submodel contributing to the multiscale system. Here, a robust and efficient framework based on a generalized extended polynomial chaos expansion (gEPCE) is proposed to simultaneously propagate all these uncertainties in order to provide a probabilistic representation of specific quantities of interest (QoI). We characterize the full probability distribution of the QoI and the uncertainty in the failure probability pertaining to its tails. By combining gEPCE with kernel density estimation (KDE) and directional derivatives, we construct sensitivity measures that connect these statistical metrics of QoI to the various sources of uncertainty to assess their individual and combined impacts. An illustrative problem featuring three-point bending of a composite beam is investigated to demonstrate the presented approach.

36 MATERIALS SCIENCE↗

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗

High-Resolution South American Wind Resource Data Downscaled with Generative Machine Learning Conditioned on Near-Surface Observations

High-resolution historical wind data was developed for the entirety of South America using the innovative Super-Resolution for Renewable Resource Data (sup3r) machine learning framework. The publicly available Sup3rWind South America dataset represents a significant advancement in wind resource data generation, leveraging generative machine learning conditioned on near-surface observations from the Meteorological Assimilation Data Ingest System (MADIS) to efficiently and accurately downscale coarse reanalysis data from the European Centre for Medium-Range Weather Forecasts (ERA5). This approach produces fine-scale, spatially and temporally coherent wind and meteorological fields hundreds of times more computationally efficient than traditional numerical weather modeling methods, enabling access to high-fidelity wind information across both continental and offshore regions. Sup3rWind South America builds on the earlier Sup3rWind Ukraine dataset through improvements in model architecture and outputs conditioned on near-surface observation inputs. As with the Ukraine data release, this dataset includes wind speed, wind direction, temperature, relative humidity, and pressure at a horizontal resolution of ~2 km, representing a 15x spatial enhancement relative to the 31 km ERA5 grid. Wind speed and direction are provided at 5-minute resolution, a 12x temporal refinement compared to the hourly ERA5 data, while temperature, relative humidity, and pressure remain at hourly resolution. The data covers all years from 2005 to 2024. Before downscaling, ERA5 inputs were bias-corrected using long-term monthly means and a limited number of quality-controlled observations to align large-scale statistics with regional conditions. The resulting dataset is the first publicly available high-resolution timeseries wind record that provides full spatial coverage of South America. Model validation demonstrates strong agreement with observations across several statistical metrics, consistent with other state-of-the-art high-resolution wind resource datasets. The potential applications of Sup3rWind South America span renewable energy resource assessment, energy system modeling, and grid resilience analysis. The 20-year record and high spatial and temporal resolution support accurate estimation of long-term energy yield and the economic feasibility of potential wind development sites. Continuous coverage across both continental and offshore regions enables comprehensive site prospecting within exclusive economic zones. The 2 km, 5-minute resolution data provide the spatial and temporal variability required for power system simulation, operational planning, and regional risk assessments.

17 WIND ENERGY↗

A tool for assessing demand side management and operating strategies for isolated microgrids

Globally, power system operators are exploring ways to leverage the capabilities of smart meters to implement fine-grained demand side management (DSM). Here this paper presents a new simulation tool to evaluate operating strategies for smart-meter-enabled (SME) islanded microgrids being advanced for sustainable rural electrification in emerging economies. Based on MATLAB/Simulink, the tool's component-level models of small microgrids can be easily configured. Alternative rule-based operating strategies are implemented with controllable supply and storage components as well as other components driven by external factors (e.g., weather and time of day). To account for the wide range of possible operating conditions in real-world applications, statistical metrics can be evaluated using Monte Carlo (MC) methods based on user-defined, time-dependent probability distributions for demands, supplies, and environmental variables. Features of the tool are demonstrated with case studies for a representative microgrid in rural Rwanda.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Are Terrestrial Biosphere Models Fit for Simulating the Global Land Carbon Sink?

The Global Carbon Project estimates that the terrestrial biosphere has absorbed about one-third of anthropogenic CO 2 emissions during the 1959–2019 period. This sink-estimate is produced by an ensemble of terrestrial biosphere models and is consistent with the land uptake inferred from the residual of emissions and ocean uptake. The purpose of our study is to understand how well terrestrial biosphere models reproduce the processes that drive the terrestrial carbon sink. One challenge is to decide what level of agreement between model output and observation-based reference data is adequate considering that reference data are prone to uncertainties. To define such a level of agreement, we compute benchmark scores that quantify the similarity between independently derived reference data sets using multiple statistical metrics. Models are considered to perform well if their model scores reach benchmark scores. Our results show that reference data can differ considerably, causing benchmark scores to be low. Model scores are often of similar magnitude as benchmark scores, implying that model performance is reasonable given how different reference data are. While model performance is encouraging, ample potential for improvements remains, including a reduction in a positive leaf area index bias, improved representations of processes that govern soil organic carbon in high latitudes, and an assessment of causes that drive the inter-model spread of gross primary productivity in boreal regions and humid tropics. The success of future model development will increasingly depend on our capacity to reduce and account for observational uncertainties.

58 GEOSCIENCES↗

Application of research reactor environments for validation of the IRDFF-II dosimetry cross section library

Activation data from seven different reactor-based reference neutron fields are examined to provide enhanced validation evidence for the newly released IRDFF-II library. A least-squares based spectrum adjustment methodology is used and rigorous statistical metrics demonstrate the consistency of the set of IRDFF-II dosimetry cross sections. The use of these reactor-based neutron fields provides validation evidence for nine more reactions than have been addressed in the 252 Cf(sf) and 235 U(th) benchmark validation testing. The use of covers to shift their energy response increases the power of the response validation but makes it challenging to properly capture response correlations.

Griffin, Patrick↗