Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Modernizing GlideinWMS Factory Monitoring with Prometheus & Grafana

Large-scale scientific experiments like CMS and DUNE rely on the distributed workload management system GlideinWMS to efficiently utilize computing resources across heterogeneous computing environments. GlideinWMS currently records Factory statistics using Round Robin Databases (RRDBs), XML, and JSON files, and these statistics are displayed via custom monitoring Web pages, thereby limiting integration with modern observability platforms. This project investigates the use of Prometheus-based instrumentation to expose Factory metrics using OpenTelemetry principles. Factory statistics related to Glidein submission and job execution are exported as Prometheus metrics through the Prometheus Python Client Library and are served via an HTTP metrics endpoint. The collected metrics are inspected using the Prometheus web-based interface and are visualized through Grafana dashboards within the Landscape monitoring infrastructure at Fermilab. This project significantly streamlines the integration of modern monitoring technologies into GlideinWMS and establishes a framework for extending observability across additional system components.

Appiah, Gideon [Grambling State U.]↗
DOE OSTI2022

Evaluating wind speed and power forecasts for wind energy applications using an open-source and systematic validation framework

Building on the verification and validation work developed under the Second Wind Forecast Improvement Project, this work exhibits the value of a consistent procedure to evaluate wind power forecasts. We established an open-source Python code base tailored for wind speed and wind power forecast validation, WE-Validate. The code base can evaluate model forecasts with observations in a coherent manner. To demonstrate the systematic validation framework of WE-Validate, we designed and hosted a forecast evaluation benchmark exercise. We invited forecast providers in industry and academia to participate and submit forecasts for two case studies. We then evaluated the submissions with WE-Validate. Our findings suggest that ensemble means have reasonable skills in time series forecasting, whereas they are often inferior to single ensemble members in wind ramp forecasting. Adopting a voting scheme in ramp forecasting that allows ensemble members to detect ramps independently leads to satisfactory skill scores. Throughout this document, we also emphasize the importance of using statistically robust and resistant metrics as well as equitable skill scores in forecast evaluation.

17 WIND ENERGY↗
DOE OSTI2023

Reliable Devices Yield Stable Quantum Computations

Stable quantum computation requires noisy results to remain bounded even in the presence of noise fluctuations. Yet non-stationary noise processes lead to drift in the varying characteristics of a quantum device that can greatly influence the circuit outcomes. Here we address how temporal and spatial variations in noise relate device reliability to quantum computing stability. First, our approach quantifies the differences in statistical distributions of characterization metrics collected at different times and locations using Hellinger distance. We then validate an analytical bound that relates this distance directly to the stability of a computed expectation value. Our demonstration uses numerical simulations with models informed by the washington superconducting transmon device. We find that the stability metric is consistently bounded from above by the corresponding Hellinger distance, which can be cast as a specified tolerance level. These results underscore the significance of reliable quantum computing devices and the impact for stable quantum computation.

Dasgupta, Samudra↗
NASA NTRS1997

Systems Engineering Metrics: Organizational Complexity and Product Quality Modeling

Innovative organizational complexity and product quality models applicable to performance metrics for NASA-MSFC's Systems Analysis and Integration Laboratory (SAIL) missions and objectives are presented. An intensive research effort focuses on the synergistic combination of stochastic process modeling, nodal and spatial decomposition techniques, organizational and computational complexity, systems science and metrics, chaos, and proprietary statistical tools for accelerated risk assessment. This is followed by the development of a preliminary model, which is uniquely applicable and robust for quantitative purposes. Exercise of the preliminary model using a generic system hierarchy and the AXAF-I architectural hierarchy is provided. The Kendall test for positive dependence provides an initial verification and validation of the model. Finally, the research and development of the innovation is revisited, prior to peer review. This research and development effort results in near-term, measurable SAIL organizational and product quality methodologies, enhanced organizational risk assessment and evolutionary modeling results, and 91 improved statistical quantification of SAIL productivity interests.

Mog, Robert A.↗
NASA NTRS2012

Prediction of Traffic Complexity and Controller Workload in Mixed Equipage NextGen Environments

Controller workload is a key factor in limiting en route air traffic capacity. Past efforts to quantify and predict workload have resulted in identifying objective metrics that correlate well with subjective workload ratings during current air traffic control operations. Although these metrics provide a reasonable statistical fit to existing data, they do not provide a good mechanism for estimating controller workload for future air traffic concepts and environments that make different assumptions about automation, enabling technologies, and controller tasks. One such future environment is characterized by en route airspace with a mixture of aircraft equipped with and without Data Communications (Data Comm). In this environment, aircraft with Data Comm will impact controller workload less than aircraft requiring voice communication, altering the close correlation between aircraft count and controller workload that exists in current air traffic operations. This paper outlines a new trajectory-based complexity (TBX) calculation that was presented to controllers during a human-in-the-loop simulation. The results showed that TBX accurately estimated the workload in a mixed Data Comm equipage environment and the resulting complexity values were understood and readily interpreted by the controllers. The complexity was represented as a "modified aircraft account" that weighted different complexity factors and summed them in such a way that the controllers could effectively treat them as aircraft count. The factors were also relatively easy to tune without an extensive data set. The results showed that the TBX approach is well suited for presenting traffic complexity in future air traffic environments.

Lee, Paul U.↗
NASA NTRS2016

Assimilation of Gridded Terrestrial Water Storage Observations from GRACE into a Land Surface Model

Observations of terrestrial water storage (TWS) from the Gravity Recovery and Climate Experiment (GRACE) satellite mission have a coarse resolution in time (monthly) and space (roughly 150,000 km(sup 2) at midlatitudes) and vertically integrate all water storage components over land, including soil moisture and groundwater. Data assimilation can be used to horizontally downscale and vertically partition GRACE-TWS observations. This work proposes a variant of existing ensemble-based GRACE-TWS data assimilation schemes. The new algorithm differs in how the analysis increments are computed and applied. Existing schemes correlate the uncertainty in the modeled monthly TWS estimates with errors in the soil moisture profile state variables at a single instant in the month and then apply the increment either at the end of the month or gradually throughout the month. The proposed new scheme first computes increments for each day of the month and then applies the average of those increments at the beginning of the month. The new scheme therefore better reflects submonthly variations in TWS errors. The new and existing schemes are investigated here using gridded GRACE-TWS observations. The assimilation results are validated at the monthly time scale, using in situ measurements of groundwater depth and soil moisture across the U.S. The new assimilation scheme yields improved (although not in a statistically significant sense) skill metrics for groundwater compared to the open-loop (no assimilation) simulations and compared to the existing assimilation schemes. A smaller impact is seen for surface and root-zone soil moisture, which have a shorter memory and receive smaller increments from TWS assimilation than groundwater. These results motivate future efforts to combine GRACE-TWS observations with observations that are more sensitive to surface soil moisture, such as L-band brightness temperature observations from Soil Moisture Ocean Salinity (SMOS) or Soil Moisture Active Passive (SMAP). Finally, we demonstrate that the scaling parameters that are applied to the GRACE observations prior to assimilation should be consistent with the land surface model that is used within the assimilation system.

GRACE↗
NASA NTRS2018

Impacts of NASA's Earth Observations on Subseasonal and Seasonal Forecasts

NASA's suite of Earth-observing satellites provides a unique view of many processes on Earth, with relevance on timescales ranging from hours to weeks and even years. NASA's observations span all parts of the Earth system: atmospheric, ocean, land and cryosphere, and include physical, chemical and biological components. This presentation explores use of NASA observations in extended-range prediction, from many days to months, using the Goddard Earth Observing System (GEOS) assimilation and predictive modeling capabilities. The skill of a weather forecast is linked to the fidelity of the initialization process (data assimilation) and the realistic representation of "fast" physical processes in the model. At longer time horizons, the slower "feedback" processes in the Earth System begin to take a more prominent role in the accuracy of the forecast. Simultaneously, forecast-skill attribution transitions away from feature-based metrics that emphasize smaller scales (e.g., representations of fronts and vortices) to metrics that emphasize the statistical distributions of large-scale features (e.g., ENSO diagnostics and teleconnections). This presentation will summarize studies performed using the GEOS-S2S (subseasonal to seasonal) system that explore the impacts of NASA observations on the fidelity of the forecasts. The GEOS-S2S system is configured for the atmosphere-ocean-land-ice model and is initialized using in-situ and space-based observations, including atmospheric aerosols and ozone which are not typically analyzed in such systems. The GEOS-S2S model routinely includes aerosol feedbacks, which systematically impact the realism of the forecasts, providing a first example of how suitable NASA observations impact the performance of the GEOS-S2S system. Studies in which a stratospheric chemistry module is activated in the GEOS-S2S system allow the impacts of ozone radiative feedbacks to be isolated. Space-based observations of sea-surface temperature and altimetry are routinely analyzed for the initialization of the GEOS-S2S system; recent advances allow the use of NASA's sea-surface salinity data, which are shown to impact the long-range skill of the forecasts.

Pawson, Steven↗
NASA NTRS2023

Split-Step Simulations to Assess the Effects of Atmospheric Boundary Layer Turbulence on the Dose Variability of N-Waves and Shaped Booms

The effects of atmospheric boundary layer turbulence on the loudness variability of a sonic boom N-wave and shaped boom are examined with split-step simulations. The shaped boom is representative of a design iteration of the NASA X-59 aircraft. Inhomogeneous atmospheric boundary layer turbulence is generated in the computational domain by a Fourier synthesis method. The N-wave and shaped boom are propagated through turbulent fields representing eight different convection levels measured at the NASA Kennedy Space Center and the NASA Armstrong Flight Research Center. Probability density functions of the formation of caustic regions along the propagation direction are computed from the N-wave results, and a parameter to collapse the caustic PDFs that accounts for both fluctuation intensities and length scales is proposed. Statistical results concerning loudness metric variability are presented, and the standard deviations of several metrics are shown to collapse across different convection levels of turbulence for small nondimensional propagation distances. The loudness metric distributions are observed to be well approximated by a normal distribution for a given range of propagation distances, and become increasingly skewed as distance increases. A model function for the dose variability is proposed, and the function parameters are found to be related to the convection level of the turbulence. The model for the dose variability distribution is compared to simulation data that were not used to find the regression parameters of the model. At several nondimensional propagation distances, agreement is observed between the model and the simulation data. These results indicate that the model may be suitable for providing quick estimates of noise dose variability in the primary carpet region across a wide range of atmospheric boundary layer conditions.

Sonic boom↗
NASA NTRSDate not supplied

Uncertainty Quantification and Sensitivity Analysis in Process-Structure-Property Simulations for Laser Powder Bed Fusion Additive Manufacturing

Process variations and process-induced defects like porosity cause significant uncertainty in the microstructure and mechanical behavior of additively manufactured metals. Establishing process-structure-property (PSP) relationships and quantifying uncertainty using experiments alone is costly, especially for structural applications where mechanical allowables must be established for qualification and certification. This work presents a PSP simulation framework for laser powder bed fusion with a focus on uncertainty quantification through probabilistic calibration and multi-fidelity uncertainty propagation. Motivated by phenomenological input parameters related to grain nucleation and growth that are difficult to characterize, a global sensitivity analysis (GSA) is completed. Through GSA, the most important input parameters are identified based on their influence on the statistical distributions of microstructural metrics that influence mechanical behavior, including grain size, morphology, and crystallographic texture. The results provide insight on what experiments are necessary to quantify and control PSP uncertainties, particularly those associated with the more challenging input parameters.

additive manufacturing↗
NASA NTRS2020

Recent Developments in Tolerancing Methods for Imaging Spectrometers

We discuss detailed tolerancing methods developed for imaging spectrometers at NASA Jet Propulsion Laboratory, California Institute of Technology using the Earth Surface Mineral Dust Source Investigation (EMIT) imaging spectrometer as an illustrative example. We tolerance five metrics simultaneously: along-track response function, cross-track response function, spectral response function, spectral centroid uniformity, and spatial centroid uniformity. A method to calculate tolerancing sensitivities for each metric directly, a method to statistically combine Monte Carlo files from multiple tolerancing runs, and an example summary error budgets that communicate the key and driving tolerances for each metric are discussed. These methods facilitate rapid and semi-automated assessment of the predicted performance of imaging spectrometer systems from design through to assembly and launch life cycle, using metrics that are directly relevant to the extraction of accurate spectroscopic data from these instruments.

Mouroulis, Pantazis↗
DOE OSTI2021

Quantifying the impacts of compound extremes on agriculture

Agricultural production and food prices are affected by hydroclimatic extremes. There has been a growing amount of literature measuring the impacts of individual extreme events (heat stress or water stress) on agricultural and human systems. Yet, we lack a comprehensive understanding of the significance and the magnitude of the impacts of compound extremes. This study combines a fine-scale weather product with outputs of a hydrological model to construct functional metrics of individual and compound hydroclimatic extremes for agriculture. Then, a yield response function is estimated with individual and compound metrics, focusing on corn in the United States during the 1981–2015 period. Supported by statistical evidence, the findings suggest that metrics of compound hydroclimatic extremes are better predictors of corn yield variations than metrics of individual extremes. The results also confirm that wet heat is more damaging than dry heat for corn. This study shows the average yield damage from heat stress has been up to four times more severe when combined with water stress.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1987

Robust statistical methods for automated outlier detection

The computational challenge of automating outlier, or blunder point, detection in radio metric data requires the use of nonstandard statistical methods because the outliers have a deleterious effect on standard least squares methods. The particular nonstandard methods most applicable to the task are the robust statistical techniques that have undergone intense development since the 1960s. These new methods are by design more resistant to the effects of outliers than standard methods. Because the topic may be unfamiliar, a brief introduction to the philosophy and methods of robust statistics is presented. Then the application of these methods to the automated outlier detection problem is detailed for some specific examples encountered in practice.

Jee, J. R.↗
DOE OSTI2022

Evaluating Synthetic Smart Meter Locations For Communication System Modeling

An important part of good communication models for smart grid applications, particularly when wireless protocols are used, is the location information of the nodes. We’ve developed metrics to assist in evaluating whether a given model with such information is representative of feeders found in the world. These metrics are applied to a set of feeder models with known good location information for the smart meter locations and compared to a set of feeder models with similar location information that is not expected to be representative of real-world feeders. The comparison reveals that the suspect models do not pass statistical tests utilizing the developed metrics, providing initial validation of the analysis technique.

smart meter, feeder model, GridLAB-D↗
DOE OSTI2025

X-Ray Microtomography Measurements of Coevolving Particle Size and Shape

This study measures simultaneous changes in particle size and shape during grain crushing and proposes quantitative metrics to track their coevolving statistical distribution. Idealized granular materials with widely different initial particle shapes (i.e., spheres, quasi-ellipsoids, plates, and rods) are compressed oedometrically at vertical stress levels sufficient to induce particle breakage. X-ray microtomography and digital image analysis are then used to track changes in particle-scale characteristics. The observations indicate that more irregular particles tend to exhibit a higher degree of breakage compared to more spherical grains subjected to an equivalent stress level. Despite these differences, the measurements also indicate that all the tested particle sets approach a common attractor (expressed in terms of aspect ratio, flatness, and elongation), regardless of their initial morphology. A new shape evolution index is proposed to quantify such trends. With this new metric, it is shown that in all the tested materials, the particle shape evolves faster than size, especially for the more regular morphologies, thus implying that the shape tends to reach its attractor point before the grading reaches its ultimate particle size distribution. Furthermore, this finding underpins an ultimate stage of crushing-driven compression that occurs with particle size reduction but minimal shape changes (i.e., self-replicating shapes due to fractures).

Oedometric compression↗
DOE OSTI2023

Stochastic Framework for Optimal Control of Planetary Reentry Trajectories Under Multilevel Uncertainties

We present a novel stochastic optimal control framework that accounts for various types of uncertainties, with application to reentry trajectory planning. The formulation of the optimal trajectory control problem is presented in the context of an indirect method where a functional objective associated with the terminal vehicle speed is to be minimized. Uncertain input parameters in the optimal trajectory control model, including aerodynamic parameters and initial and terminal conditions, are modeled as aleatory random variables, while the statistical parameters of these aleatory distributions are themselves random variables. The parametric and model uncertainties are simultaneously propagated through an extended polynomial chaos expansion (EPCE) formalism. Several metrics are described to evaluate response statistics and presented as insightful tools for robust decision making. Specifically, the response probability density function (PDF) reflecting influence of both epistemic and aleatory uncertainties is obtained. By sampling over the random variables representing model error, an ensemble of response PDFs is generated and the associated failure probability is estimated as a random variable with its own polynomial chaos expansion. Besides, the sensitivity index functions of response PDF with respect to the statistical parameters are evaluated. Coupling parametric and model uncertainties within the EPCE framework leads to a robust and efficient paradigm for multilevel uncertainty propagation and PDF characterization in general optimal control problems.

Engineering↗
DOE OSTI2025

Relationships Between Mesoscale Convective System Properties and Midlevel Dynamic Perturbations

Abstract Past studies implicate dynamic anomalies operating on subsynoptic scales as a possible initiation source of summertime (July–August) mesoscale convective systems (MCSs) in the central United States during northwesterly flow regimes. To improve our understanding of warm season MCSs occurring over a variety of flow regimes, we track midlevel (600 hPa) vorticity perturbations (“MPs”) as 2D objects comprising wavelengths of 500–2,500 km over the central US from May–August of 2004–2021. We perform statistical analysis of relationships between metrics of MP objects (e.g., duration, size, intensity, and origin) and high‐resolution MCS precipitation characteristics (e.g., duration, total rainfall, rain coverage area, and motion) that occur while collocated with or in the absence of MPs to discern predictive capability of background dynamic features on storm precipitation potential. Although the majority of MPs collocated with MCS initiation occur during July–August, a significant number (40%) occur between May and June. Northwesterly flow MPs comprise a relative minority of our events, suggesting that MPs can affect MCSs across a variety of warm season flow regimes. MPs affecting MCSs initiated primarily over the high plains near the central Rockies. Only approximately 20% of tracked MCS initiation events were collocated with MPs, but these storms produced ∼25% greater lifetime rainfall and coverage area, and ∼29% more stratiform rain than non‐MP‐induced MCSs. In general, larger and more vigorous MPs resulted in more hydrologically impactful MCSs. The most directly attributable benefit to MCS initiation was from MP‐enhanced background vertical motion and thermodynamic instability (e.g., increased CAPE).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Learning plasma dynamics and robust rampdown trajectories with predict-first experiments at TCV

The rampdown phase of a tokamak pulse is difficult to simulate and often exacerbates multiple plasma instabilities. To reduce the risk of disrupting operations, we leverage advances in Scientific Machine Learning (SciML) to combine physics with data-driven models, developing a neural state-space model (NSSM) that predicts plasma dynamics during Tokamak à Configuration Variable (TCV) rampdowns. The NSSM efficiently learns dynamics from a modest dataset of 311 pulses with only five pulses in a reactor-relevant high-performance regime. The NSSM is parallelized across uncertainties, and reinforcement learning (RL) is applied to design trajectories that avoid instability limits. High-performance experiments at TCV show statistically significant improvements in relevant metrics. A predict-first experiment, increasing plasma current by 20% from baseline, demonstrates the NSSM’s ability to make small extrapolations. The developed approach paves the way for designing tokamak controls with robustness to considerable uncertainty and demonstrates the relevance of SciML for fusion experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Extensive analysis of reconstruction algorithms for DESI 2024 baryon acoustic oscillations

Reconstruction of the baryon acoustic oscillation (BAO) signal has been a standard procedure in BAO analyses over the past decade and has helped to improve the BAO parameter precision by a factor of ∼2 on average. The Dark Energy Spectroscopic Instrument (DESI) BAO analysis for the first year (DR1) data uses the “standard” reconstruction framework, in which the displacement field is estimated from the observed density field by solving the linearized continuity equation in redshift space, and galaxy and random positions are shifted in order to partially remove non-linearities. There are several approaches to solving for the displacement field in real survey data, including the multigrid (MG), iterative Fast Fourier Transform (iFFT), and iterative Fast Fourier Transform particle (iFFTP) algorithms. In this work, we analyze these algorithms and compare them with various metrics including two-point statistics and the displacement itself using realistic DESI mocks. We focus on three representative DESI samples, the emission line galaxies (ELG), quasars (QSO), and the bright galaxy sample (BGS), which cover the extreme redshifts and number densities, and potential wide-angle effects. We conclude that the MG and iFFT algorithms agree within 0.4% in post-reconstruction power spectrum on BAO scales with the RecSym convention, which does not remove large-scale redshift space distortions (RSDs), in all three tracers. The RecSym convention appears to be less sensitive to displacement errors than the RecIso convention, which attempts to remove large-scale RSDs. However, iFFTP deviates from the first two; thus, we recommend against using iFFTP without further development. In addition, we provide the optimal settings for reconstruction for five years of DESI observation. The analyses presented in this work pave the way for DESI DR1 analysis as well as future BAO analyses.

79 ASTRONOMY AND ASTROPHYSICS↗