Search NASA⌕ Search

SEARCH · Search NASA

Results for “test statistics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Machine Learning Approaches for Nuclear Material Accounting Data from Irradiation and Reprocessing

We are currently exploring data analysis methods for their ability to strengthen the synthesis and evaluation of information generated within domestic and international safeguard regimes. Safeguard data typically includes rich heterogenous datasets amenable to advanced data analytics methods, such as machine learning. We have converted transactional data containing both numerical and categorical attributes from a domestic nuclear material control and accountability (NMC&A) system into a low-dimensional numerical structure via linear principal component analysis. This data representation allows for global structure discovery via cluster analysis, which can characterize the typical behavior of each of the primary types of transaction events. Furthermore, the structure of the top principal components captures the “typical behavior” of the data and is thus amenable to anomaly detection through statistical hypothesis testing. We explored this capability by generating erroneous permutations of the data and computing the Q-residual quantity increase associated with the information loss when these data are projected into the low-dimensional principal component analysis space representative of typical transactions.Future work will focus on identifying data transformations (e.g., graph networks) that more closely align with the inherent structure of these transactions to explain more salient information and disambiguate the underlying nuclear process—in this case, irradiation and reprocessing—from artifacts of NMC&A system transactional record keeping data.

Drescher, Adam↗
DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗
DOE OSTI2022

Probing the role of solids loading and mix procedure on the properties of acoustically mixed materials for additive manufacturing

We report resonant acoustic mixing has been of particular interest for use in additive manufacturing since viscous, solids-loaded materials can be difficult to mix and inhomogeneity has adverse effects on print quality. In this study, we detail a method to iterate through different formulations and mix procedures and assess mixture quality. The approach utilizes a constant pressure-driven flow test to collect statistics on flow rate through a standard geometry. This test is first applied to well mixed formulations containing particulate solids and find that the results are sensitive to viscosity produced by solids content. We then consider the formulations at various stages of mixing and find that the spread in the volume measurements is indicative of the mixing quality; poorly mixed material yields measurements with wide distributions. We believe this testing approach can be useful when screening new formulations, developing mixing processes, or for quality control.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Overall Performance Losses and Activated Mechanisms in Double Glass and Glass-backsheet Photovoltaic Modules with Monofacial and Bifacial PERC Cells, under Accelerated Exposures

Commercial PV modules have various packaging choices nowadays, which influence their long-term reliability. This study compared the degradation behaviors of sixteen module variants from two brands with varying encapsulant materials (EVA or POE), encapsulant types, module architectures (GB or DG), and cell types (monofacial or bifacial) using null hypothesis testing to determine statistical significant findings. The modules were exposed for 2,520 hours under two accelerated exposures: modified damp heat (mDH) and modified damp heat with full-spectrum light (mDH+FSL). For both brands, two DG module variants with UV-Cutoff rear encapsulant are found to have significantly lower average power loss than the module variants of EVA+GB with opaque rear encapsulant after each accelerated exposure. Metallization interconnect corrosion is identified as the primary degradation mechanism. Furthermore, unsupervised hierarchical clustering finds that the degradation behaviors of modules from one brand with a more strict manufacturing quality control depends on module architectures only.

14 SOLAR ENERGY↗
DOE OSTI2023

Statistical chronometry of meteorites. I. A Test of 26 Al homogeneity and the Pb-Pb age of the solar system’s $t=0$

Here, we use rapidly cooled achondrites to test the assumption of 26 Al homogeneity in the solar nebula, by checking if there is a single value of t SS , the absolute “Pb-Pb” age of the Solar System’s t=0, that makes concordant their ages from the Al-Mg and Pb-Pb systems. We find that values t SS =4568.42 ± 0.24 Myr do make these ages concordant, and therefore the hypothesis of homogeneous 26 Al is not falsified. This age, defined to be when the solar nebula had ( 26 Al/ 27 Al) = 5.23 × 10 –5 , is significantly older than the ≈ 4567.3 Myr inferred from direct measurements of Pb-Pb ages in CAIs. Discrepancies between the Al-Mg and Pb-Pb chronometers in chondrules and CAIs have previously been interpreted as arising from heterogeneities in 26 Al, under the presumption that the Al-Mg and Pb-Pb systems in CAIs closed simultaneously. We examine this assumption and show that resetting is to be expected in CAIs. In particular, we quantitatively demonstrate that it is plausible that Pb-Pb ages of CAIs were reset at late times, without resetting the earlier Al-Mg ages, if they were transiently heated in the same manner as chondrules. We critically examine Pb-Pb isochrons, refining data and suggesting best practices for their calculation and reporting. We advocate reporting chronometry as times of formation after t=0 rather than absolute ages, as only the former is useful for astrophysical models of the solar nebula. We advocate averaging of multiple samples, rather than anchoring to individual meteorites, to improve precision.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Atacama Cosmology Telescope: DR6 gravitational lensing and SDSS BOSS cross-correlation measurement and constraints on gravity with the 𝐸 𝐺 statistic

We derive new constraints on the 𝐸 𝐺 statistic as a test of gravity, combining the cosmic microwave background (CMB) lensing map estimated from Data Release 6 (DR6) of the Atacama Cosmology Telescope with Sloan Digital Sky Survey III Baryon Oscillation Spectroscopic Survey (SDSS BOSS) CMASS and LOWZ galaxy data. We develop an analysis pipeline to measure the cross-correlation between CMB lensing maps and galaxy data, following a blinding policy and testing the approach through null and consistency checks. By testing the equivalence of the spatial and temporal gravitational potentials, the 𝐸 𝐺 statistic can distinguish Λ⁢ CDM from alternative models of gravity. We find 𝐸 𝐺 ⁡(𝑧 eff = 0.555) = 0.3⁢1$^{+0.06}_{−0.05}$ for Atacama Cosmology Telescope (ACT) and CMASS data at 68.28% confidence level, and 𝐸 𝐺 ⁡(𝑧 eff = 0.316) = 0.4⁢9$^{+0.14}_{−0.11}$ for the ACT and LOWZ. Systematic errors are estimated to be 3% and 4%, respectively. Including CMB lensing information from Planck PR4 results in 𝐸 𝐺 ⁡(𝑧 eff = 0.555) = 0.3⁢4$^{+0.05}_{−0.05}$ with CMASS and 𝐸 𝐺 ⁡(𝑧 eff = 0.316) = 0.4⁢3$^{+0.11}_{−0.09}$ with LOWZ. These are consistent with predictions for the Λ⁢ CDM model that best fits the Planck CMB anisotropy and SDSS BOSS baryon acoustic oscillations (BAO), where 𝐸$^{GR}_{𝐺⁡}$(𝑧 eff =0.555) =0.401 ± 0.005 for CMB lensing combined with CMASS and 𝐸$^{GR}_{𝐺}$⁡(𝑧 eff = 0.316) = 0.452 ± 0.005 combined with LOWZ. We also find 𝐸 𝐺 to be scale independent, with probability to exceed >5%, as predicted by general relativity. The methods developed in this work are also applicable to improved future analyses with upcoming spectroscopic galaxy samples and CMB lensing measurements.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Monte Carlo goodness-of-fit tests for degree corrected and related stochastic blockmodels

Abstract We construct Bayesian and frequentist finite-sample goodness-of-fit tests for three different variants of the stochastic blockmodel for network data. Since all of the stochastic blockmodel variants are log-linear in form when block assignments are known, the tests for the latent block model versions combine a block membership estimator with the algebraic statistics machinery for testing goodness-of-fit in log-linear models. We describe Markov bases and marginal polytopes of the variants of the stochastic blockmodel and discuss how both facilitate the development of goodness-of-fit tests and understanding of model behaviour. The general testing methodology developed here extends to any finite mixture of log-linear models on discrete data, and as such is the first application of the algebraic statistics machinery for latent-variable models.

Karwa, Vishesh↗
DOE OSTI2021

Towards testing the theory of gravity with DESI: summary statistics, model predictions and future simulation requirements

Shortly after its discovery, General Relativity (GR) was applied to predict the behavior of our Universe on the largest scales, and later became the foundation of modern cosmology. Its validity has been verified on a range of scales and environments from the Solar system to merging black holes. However, experimental confirmations of GR on cosmological scales have so far lacked the accuracy one would hope for — its applications on those scales being largely based on extrapolation and its validity there sometimes questioned in the shadow of the discovery of the unexpected cosmic acceleration. Future astronomical instruments surveying the distribution and evolution of galaxies over substantial portions of the observable Universe, such as the Dark Energy Spectroscopic Instrument (DESI), will be able to measure the fingerprints of gravity and their statistical power will allow strong constraints on alternatives to GR. In this paper, based on a set of N-body simulations and mock galaxy catalogs, we study the predictions of a number of traditional and novel summary statistics beyond linear redshift distortions in two well-studied modified gravity models — chameleon f(R) gravity and a braneworld model — and the potential of testing these deviations from GR using DESI. These summary statistics employ a wide array of statistical properties of the galaxy and the underlying dark matter field, including two-point and higher-order statistics, environmental dependence, redshift space distortions and weak lensing. We find that they hold promising power for testing GR to unprecedented precision. The major future challenge is to make realistic, simulation-based mock galaxy catalogs for both GR and alternative models to fully exploit the statistic power of the DESI survey (by matching the volumes and galaxy number densities of the mocks to those in the real survey) and to better understand the impact of key systematic effects. Using these, we identify future simulation and analysis needs for gravity tests using DESI.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Durability testing of actual Hanford waste glasses and their non-radioactive simulant glasses

The low-activity waste (LAW) fraction of Hanford tank waste will be converted to glass at the Waste Treatment and Immobilization Plant (WTP) and disposed on the Hanford site. The chemical durability of LAW glasses has been researched for decades to satisfy contract requirements. Most LAW glass durability data has been generated on non-radioactive simulant glasses fabricated via crucible melts. These non-radioactive glasses were chosen due to safety and cost reasons with confidence that radioactive waste glasses would exhibit similar behavior. To reduce the risk of significant differences in laboratory test response data between WTP melter waste glass and simulant glass, Product Consistency Tests (PCT, i.e., ASTM C1285-21) and Environmental Protection Agency (EPA) 1313 durability tests were performed on actual and simulant LAW glasses fabricated using laboratory-scaled melters. Actual and simulant glass durability test results are presented and statistically compared. Finally, differences in test responses were found to be within experimental uncertainty.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

STAT7 v2.0 Verification and Validation Report

This report documents the software testing which has been performed for STAT7 Version 2.0 (hereafter STAT7 v2.0), a code for the statistical propagation of uncertainties in steady-state thermal hydraulics (TH) analysis of plate-fueled research and test reactors. The key capabilities addressed in this testing were selected based on the use of the software for analysis work supporting the conversion to low-enriched uranium fuel of U.S. High-Performance Research Reactors (USHPRR) such as the Massachusetts Institute of Technology Research Reactor (MITR-II). This testing scope includes comparison of the code’s statistical processing capabilities using statistical tests, comparison of coolant property evaluations to NIST data tables, and systematic comparison of key TH solver capabilities to the results of hand calculations and to predictions from PLTEMP/ANL Version 4.4. The

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Verification of Maximal Oxygen Uptake in Active Military Personnel During Treadmill Running

Figueiredo, PS, Looney, DP, Pryor, JL, Doughty, EM, McClung, HL, Vangala, SV, Santee, WR, Beidleman, BA, and Potter, AW. Verification of maximal oxygen uptake in active military personnel during treadmill running. J Strength Cond Res 36(4): 1053–1058, 2022—It is unclear whether verification tests are required to confirm “true” maximal oxygen uptake ($\dot{V}_{o2}$max) in modern warfighter populations. Our study investigated the prevalence of $\dot{V}_{o2}$max attainment in U.S. Army soldiers performing a traditional incremental running test. In addition, we examined the utility of supramaximal verification testing as well as repeated trials for familiarization for accurate $\dot{V}_{o2}$max assessment. Sixteen U.S. Army soldiers (1 woman, 15men; age, 21±2 years; height, 1.73±0.06m; body mass, 71.6±10.1 kg) completed 2 laboratory visits, each with an incremental running test (modified Astrand protocol) and a verification test (110% maximal incremental test speed) on a motorized treadmill. We evaluated $\dot{V}_{o2}$max attainment during incremental testing by testing for the definitive $\dot{V}_{o2}$ plateau using a linear least-squares regression approach. Peak oxygen uptake ($\dot{V}_{o2}$peak) was considered statistically equivalent between tests if the 90% confidence interval around the mean difference was within 62.1 ml·kg -1 ·min -1 . Oxygen uptake plateaus were identified in 14 of 16 volunteers for visit 1 (87.5%) and all 16 volunteers for visit 2 (100%). Peak oxygen uptake was not statistically equivalent, apparent from the mean difference in $\dot{V}_{o2}$peak measures between the incremental test and verification test on visit 1 (2.3 ml·kg -1 ·min -1 , [1.3–3.2]) or visit 2 (1.1 ml·kg -1 ·min -1 [0.2–2.1]). Interestingly, $\dot{V}_{o2}$peak was equivalent, apparent from the mean difference in $\dot{V}_{o2}$peak measures between visits for the incremental tests (0.0ml·kg -1 ·min -1 [-0.8to 0.9]) but not the verification tests (-1.2 ml·kg -1 ·min -1 [-2.2 to -0.2]). Modern U.S. Army soldiers can attain $\dot{V}_{o2}$max by performing a modified Astrand treadmill running test. Additional familiarization and verification tests for confirming $\dot{V}_{o2}$max in healthy active military personnel may be unnecessary.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

STAT7 v1.2 Verification and Validation Report

This report documents the software testing which has been performed for the STAT7 Version 1.2 (hereafter STAT7 v1.2) software, a code for the statistical propagation of uncertainties in steady-state thermal hydraulics (TH) analysis of plate-fueled research and test reactors. The key capabilities addressed in this testing were selected based on the use of the software for analysis work supporting the conversion to low-enriched uranium fuel of U.S. High-Performance Research Reactors (USHPRR) such as the Massachusetts Institute of Technology Research Reactor (MITR-II). This testing scope includes comparison of the code’s statistical processing capabilities using statistical tests, comparison of coolant property evaluations to NIST data tables, and systematic comparison of key TH solver capabilities to the results of hand calculations and to predictions from PLTEMP/ANL Version 4.4. The good agreement achieved in this testing campaign, within the acceptance criteria of a 1% relative difference, also demonstrates that these capabilities have been implemented in STAT7 v1.2 correctly.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Estimation of process steady state with autoregressive models and Bayesian inference

To improve efficiency, separations engineers will typically design process circuits containing recirculating streams, which mix one or more of the process outputs with the feed material. Doing so can improve efficiency, but will cause a delay in the system reaching steady state conditions until the recirculating load mass flows stabilize. In testing separation circuits, engineers will often test a variety of factors and complete an analysis from sample results. Knowledge of if a process is at steady state, as well as the steady state conditions of a process, is essential for a valid techno-economic analysis. However, the definition of process steady state is often poorly defined, or does not include uncertainty quantification. If the performance of a process operating under two different sets of conditions are compared, an engineer who does not test for steady state or quantify steady state conditions risks producing a faulty analysis. In this work, a Bayesian statistical method for testing if all streams are at steady state is further motivated and then derived. Then after testing for steady state, the same model is used with a prior distribution that enforces a steady state assumption to estimate steady state conditions. Further, these methods were validated in a solvent extraction pilot plant where steady state conditions for all outflows were inferred with uncertainty quantification. Analysis is completed with functions available to the reader as part of the BayesMassBal (V 1.1.0) software package written in R.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2023

The Potential Benefits of Handling Mixture Statistics via a Bi-Gaussian EnKF: Tests With All-Sky Satellite Infrared Radiances

The meteorological characteristics of cloudy atmospheric columns can be very different from their clear counterparts. Thus, when a forecast ensemble is uncertain about the presence/absence of clouds at a specific atmospheric column (i.e., some members are clear while others are cloudy), that column's ensemble statistics will contain a mixture of clear and cloudy statistics. Such mixtures are inconsistent with the ensemble data assimilation algorithms currently used in numerical weather prediction. Hence, ensemble data assimilation algorithms that can handle such mixtures can potentially outperform currently used algorithms. In this study, we demonstrate the potential benefits of addressing such mixtures through a bi-Gaussian extension of the ensemble Kalman filter (BGEnKF). The BGEnKF is compared against the commonly used ensemble Kalman filter (EnKF) using perfect model observing system simulated experiments (OSSEs) with a realistic weather model (the Weather Research and Forecast model). Synthetic all-sky infrared radiance observations are assimilated in this study. In these OSSEs, the BGEnKF outperforms the EnKF in terms of the horizontal wind components, temperature, specific humidity, and simulated upper tropospheric water vapor channel infrared brightness temperatures. This study is one of the first to demonstrate the potential of a Gaussian mixture model EnKF with a realistic weather model. Our results thus motivate future research toward improving numerical Earth system predictions though explicitly handling mixture statistics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Ensuring statistical reproducibility of ocean model simulations in the age of hybrid computing

Novel high performance computing systems that feature hybrid architectures require large scale code refactoring to unravel underlying exploitable parallelism. Such redesign can often be accompanied with machine-precision changes as the order of computation cannot always be maintained. For chaotic systems like climate models, these round-off level differences can grow rapidly. Systematic errors may also manifest initially as machine-precision differences. Isolating genuine round off level differences from such errors remains a challenge. Here, we apply two-sample equality of distribution tests to evaluate statistical reproducibility of the ocean model component of US Department of Energy's Energy Exascale Earth System Model (E3SM). A 2-year control simulation ensemble is compared to a modified ensemble as a test case - after a known non-bit-for-bit change in a model component is introduced - to evaluate the null hypothesis that the two ensembles are statistically indistinguishable. To quantify the false negative rates of these tests, we conduct a formal power analysis using a targeted suite of short simulation ensembles. The ensemble suite contains several perturbed ensembles, each with a progressively different climate than the baseline ensemble - obtained by perturbing the magnitude of a single model tuning parameter, the Gent and McWilliams κ, in a controlled manner. The null hypothesis is evaluated for each of perturbed ensembles using these tests. The power analysis informs on the detection limits of the tests for given ensemble size allowing model developers to evaluate the impact of an introduced non-bit-for-bit change to the model.

Mahajan, Salil↗
DOE OSTI2024

Improved Subseasonal Forecasting of Extreme Polar Vortices Using Machine Learning

Our research was focused on forecasting the position and shape of the winter stratospheric polar vortex at a subseasonal timescale of 15 days in advance. To achieve this, we employed both statistical and neural network machine learning techniques. The analysis was performed on 42 winter seasons of reanalysis data provided by NASA giving us a total of 6,342 days of data. The state of the polar vortex for determined by using geometric moments to calculate the centroid latitude and the aspect ratio of an ellipse fit onto the vortex. Timeseries for thirty additional precursors were calculated to help improve the predictive capabilities of the algorithm. Feature importance of these precursors was performed using random forest to measure the predictive importance and the ideal number of precursors. Then, using the precursors identified as important, various statistical methods were tested for predictive accuracy with random forest and nearest neighbor performing the best. An echo state network, a type of recurrent neural network that features sparsely connected hidden layer and a reduced number of trainable parameters that allows for rapid training and testing, was also implemented for the forecasting problem. Hyperparameter tuning was performed for each methods using a subset of the training data. The algorithms were trained and tuned on the first 41 years of data, then tested for accuracy on the final year. In general, the centroid latitude of the polar vortex proved easier to predict than the aspect ratio across all algorithms. Random forest outperformed other statistical forecasting algorithms overall but struggled to predict extreme values. Forecasting from echo state network suggested a strong predictive capability past 15 days, but further work is required to fully realize the potential of recurrent neural network approaches.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

High-Throughput Statistical Interrogation of Mechanical Properties with Build Plate Location and Powder Reuse in AlSi10Mg

Abstract Additive manufacturing (AM) allows agile, rapid manufacturing of geometrically complex components that would otherwise be impossible through traditional manufacturing methods. With this maturing manufacturing technology comes the need to adopt testing methods that are commensurate with the speed of additive manufacturing and take advantage of its geometric flexibility. High-throughput tensile testing (HTT) is a technique that allows a large number of tensile bars to be tested in a short amount of time. In the present study, HTT is used to evaluate AM AlSi10Mg produced using powder bed fusion with a Renishaw AM250 machine. Three parameters were varied in this study: (1) powder reuse history, (2) location on the build plate, and (3) size of the tensile specimen. For all parameter combinations, at least 22 specimens were tested; in several cases, over 40 were tested. This large dataset, consisting of over 500 tensile tests, permits Weibull statistical analysis and provides sufficient fidelity to isolate subtle trends that would have likely been missed in smaller, traditional datasets. The observed trends are rationalized in terms of the role of porosity and surface crust on mechanical response.

Carroll, Jay D. (ORCID:0000000258184709)↗