Search NASA⌕ Search

SEARCH · Search NASA

Results for “error testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An Explainable Machine-Learning Model for Compensatory Reserve Measurement: Methods for Feature Selection and the Effects of Subject Variability

Tracking vital signs accurately is critical for triaging a patient and ensuring timely therapeutic intervention. The patient’s status is often clouded by compensatory mechanisms that can mask injury severity. The compensatory reserve measurement (CRM) is a triaging tool derived from an arterial waveform that has been shown to allow for earlier detection of hemorrhagic shock. However, the deep-learning artificial neural networks developed for its estimation do not explain how specific arterial waveform elements lead to predicting CRM due to the large number of parameters needed to tune these models. Alternatively, we investigate how classical machine-learning models driven by specific features extracted from the arterial waveform can be used to estimate CRM. More than 50 features were extracted from human arterial blood pressure data sets collected during simulated hypovolemic shock resulting from exposure to progressive levels of lower body negative pressure. A bagged decision tree design using the ten most significant features was selected as optimal for CRM estimation. This resulted in an average root mean squared error in all test data of 0.171, similar to the error for a deep-learning CRM algorithm at 0.159. By separating the dataset into sub-groups based on the severity of simulated hypovolemic shock withstood, large subject variability was observed, and the key features identified for these sub-groups differed. This methodology could allow for the identification of unique features and machine-learning models to differentiate individuals with good compensatory mechanisms against hypovolemia from those that might be poor compensators, leading to improved triage of trauma patients and ultimately enhancing military and emergency medicine.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Shock-ramp analysis test problem

Quasi-isentropic (ramp) compression is now a well-established experimental method and so are the analysis techniques to give Lagrangian sound speed, pressure, and density along the sample material's isentrope. A shock followed by ramp compression is a natural extension to investigate, for example, shock melt and refreeze on compression, or isentropes of states off the Hugoniot or principal isentrope. In practice, graded-density impactors produce initial shocks, compression by shaped laser pulses may be unable to produce a smooth pressure increase from zero, and incidental perturbations on the drive pulse may also give rise to shocks, so robust shock-ramp analysis methods will be needed. Appropriate analysis methods are needed for shock-ramp experiments, based on those for quasi-isentropic compression, and these require validation. This paper describes three different analyses of a shock-ramp test problem, including an assessment of their estimated errors. The methods tested were based on hydrodynamic characteristics or integration backward in space. All methods gave the known Lagrangian sound speed to within ~1%, and pressure and volume to within less than 2% and 1%, demonstrating that the analysis methods of isentropic compression experiments can be confidently extended to the analysis of shock and ramp compression.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Novel application of neutrinos to evaluate U.S. nuclear weapons performance

There is a growing realization that neutrinos can be used as a diagnostic tool to better understand the inner workings of a nuclear weapon. Robust estimates demonstrate that an Inverse Beta Decay (IBD) neutrino scintillation detector built at the Nevada Test Site with a 1000-ton active target mass at a standoff distance of 500 m would detect thousands of antineutrino events per nuclear test. This would provide less than 4% statistical error on the measured antineutrino rate and 5% error on antineutrino energy. Extrapolating this to an error on the test device explosive yield requires knowledge from evaluated nuclear databases, non-equilibrium fission rates, and assumptions on internal neutron fluxes. Initial calculations demonstrate that the total number of neutrinos emitted per fission in the first 10 3 s after a short pulse of 239 Pu fission is about a factor of two less than that from Pu fissioning under steady state conditions. Furthermore, there are significant energy spectral differences as a function of time after the pulse that must be considered. These and other model dependencies will be discussed in the paper. In the absence of nuclear weapons testing, many of the technical and theoretical challenges of a full nuclear test could be mitigated with a low cost smaller scale 20 ton fiducial mass IBD demonstration detector placed near a pulsed reactor. Potential reactors include the Texas A&M University TRIGA 1 GW–10 ms pulsed facility or the Sandia Annular Core Research Reactor. The short duty cycle and repeatability of pulses would provide critical real environment testing and measurements, which would be valuable for planning a possible real test shot in the future. Furthermore, the antineutrino rate as a function of time data would provide unique constraints on fission databases and model assumptions. Finally, there are impactful science drivers such as sensitive searches for ∼1 eV 2 sterile neutrinos and ∼MeV scale axions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collectedunder structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore,their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distanceto the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensembleof trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fittingobserved in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inversephysical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of thesource strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations ofunderlying regression methods.

Rao, Nageswara↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collected under structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore, their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distance to the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensemble of trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fitting observed in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inverse physical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of the source strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations of underlying regression methods.

Rao, Nageswara↗
DOE OSTI2020

A Step-Down Test Procedure for Wavelet Shrinkage Using Bootstrapping

Wavelet thresholding (or shrinkage) attempts to remove the noises existing in the signals while preserving inherent pattern characteristics in the reconstruction of true signals. For data-denoising purpose, we present a new wavelet thresholding procedure which employs the step-down testing idea of identifying active contrasts in unreplicated fractional factorial experiments. The proposed method employs bootstrapping methods to a step-down test for thresholding wavelet coefficients. By introducing the concept of a false discovery error rate in testing wavelet coefficients, we shrink the wavelet coefficients with p -values higher than the error rate. The error rate controls the expected proportion of wrongly accepted coefficients among chosen wavelet coefficients. Bootstrap samples are used to approximate the p -value for computational efficiency. We also present some guidelines for selecting the values of hyper-parameters which affect the performance in the step-down thresholding procedure. Based on some common testing signals and an air-conditioner sounds example, the comparison of our proposed procedure with other thresholding methods in the literature is performed. The analytical results show that the proposed procedure has a potential in data-denoising and data-reduction in a variety of signal reconstruction applications.

42 ENGINEERING↗
DOE OSTI2025

OLCF Test Harness

Acceptance and regression testing of a High Performance Computing (HPC) system requires an automated and reproducible framework and tool for running and logging results. Manually running tests across a system is labor intensive and prone to reproducibility errors. The OLCF Test Harness (OTH) provides a framework in which to document required tests for a HPC system. The OTH then provides tools to execute and log results of these tests in an automated fashion.

Dietz, Dan [Oak Ridge National Laboratory (ORNL), ↗
DOE OSTI2023

High Impedance Fault Detection Through Quasi-Static State Estimation: A Parameter Error Modeling Approach

This paper presents a model for detecting high impedance faults using parameter error modeling and a two step per-phase weighted-least squares state estimation process. The proposed scheme leverages the use of Phasor Measurement Units and synthetic measurements to identify per-phase power flow and injection measurements which indicate a parameter error through ?2 Hypothesis Testing applied to the composed measurement error. Although current and voltage waveforms are commonly analyzed for high-impedance fault detection, wide area power flow and injection measurements, which are already inherent to the state estimation process, also show promise for real-world high-impedance fault detection applications. The error distributions after detection share the measurement function error spread observed in proven parameter error diagnostics and can be applied to high-impedance fault identification. Further, this error spread across measurement functions related to the fault will be clearly discerned from measurement error. Case studies are performed on the IEEE 33-Bus Distribution System along with the proposed model in Simulink.

Cooper, Austin↗
DOE OSTI2023

Sources of error in detonation calorimeters and error analysis for neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB)

Here, a calorimeter for measuring heats of detonation at Lawrence Livermore National Laboratory is described. A calibration precision of 0.2 % at the 95 % confidence interval (CI) is reported. Sources of uncertainty are discussed, including nonequivalent sources, which are those arising from differences between calibration and experimental tests. The systematic error due to nonequivalent sources is bounded to 0.19–0.22 % of the measured heat for a standard detonation test where the confinement material is gold. The recommendation is to correct for the systematic error by adding 0.19 % to the reported value and adding 0.03 % to the uncertainty. It is demonstrated that the precision of a detonation test is variable with testing duration because a source of uncertainty resides in the thermodynamic correction factor k6, which accounts for the contribution to heat by stirring; the generated power is additive and therefore highly impacted by total test time. The relative proportion also varies with the magnitude of heat release and sample size, adding variance to the weight of the error arising from the correction factor. A full error analysis based on the described sources of uncertainty is developed. The methodology is applied to a test series on neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB), demonstrating an ultimate precision of 0.7 % (single test) for materials testing and a relative standard deviation of 1.8 %.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Experimental Considerations for Estimating Degradation in PV Modules

Carefully controlled laboratory experiments and measurements can enable the determination of acceleration factors suitable for extrapolation to durability and performance of a fielded PV module. Ideally, a single mechanism can be identified with appropriate acceleration factors for extrapolation to the field. However, even with a single mechanism, the inherent uncertainty in these factors leads to uncertainty in the extrapolation which is greater the higher the acceleration factor. This course will explain how because of the wide range of acceleration factors for a given degradation mode, utilizing acceleration factors greater than about 10x will typically lead to unacceptable uncertainty in the results. Therefore, if even just a rank ordering of materials is desired, acceleration factors must be minimized which requires a good general understanding of the scale of the different acceleration factors for the degradation mode of interest. In this tutorial we will discuss what the different purposes are for many of the accelerated stress tests used today. E.g., what is a qualification test, a highly accelerated stress test, a rank ordering test, or a service life prediction test. We will discuss how one can understand the relationship between test results and expected field performance. A single accelerated stress test condition cannot duplicate outdoor exposure for all possible degradation pathways; therefore, one must use targeted evaluation of material properties at different stress levels to determine the relevant acceleration factors and fit it to a model. We will also discuss how to interpret the results of experiments understanding what is relevant/not relevant, or not e valuated in a test. There are many common error people make in their test interpretations because they push the stress levels to be too harsh. This creates biases and can mask the relevant failure modes and mechanisms or will erroneously lead one to over design materials against things that aren't relevant. Several case studies will be presented to illustrate appropriate interpretation of accelerated stress testing results.

degradation↗
DOE OSTI2021

Early Battery Performance Prediction for Mixed Use Charging Profiles Using Hierarchal Machine Learning

A key step limiting how fast batteries can be deployed is the time necessary to provide evaluation and validation of performance. Using data analysis approaches, such as machine learning, the validation process can be accelerated. However, questions on the validity of projecting models trained on limited data or simple cycling profiles, such as constant current cycling, to real-world scenarios with complex loads remains. Here, we present the ability to predict performance with less than 1.2% mean absolute percent error when trained on cells aged using complex electric vehicle discharge profiles, and either AC Level 2 charge or DC Fast charge profiles, using only the first 45 cycles, namely 5% of the total testing time. While error is low across the projections, this study also highlights that battery lifetime analysis using only cycling data may not extrapolate safely to certain real-world conditions due to the impact of calendar degradation.

25 ENERGY STORAGE↗
DOE OSTI2022

Predicting the Cetane Number, Sooting Tendency, and Energy Density of Terpene Fuel Additives

Abstract Discovering renewable fuels and fuel additives is paramount in reducing carbon emissions from internal combustion engines. Terpenes, a group of compounds that can be synthesized from plant matter and microorganisms, have gained significant interest in recent years as promising candidates for fuels/additives. Terpenes are a diverse class of compounds that contain rings and methyl branches, resulting in high energy densities and optimal cold weather behavior. Their variation in bond order, carbon chains, and functional groups lead to varying degrees of soot formation and performance in existing engines. The present work leverages predictive models, namely artificial neural networks, to predict the cetane number (CN), sooting tendency (quantified with yield sooting index, YSI), and energy density (quantified with lower heating value, LHV) of terpenes and hydrogenated terpenes whose sooting propensities were previously determined through experimental means. Predicted sooting propensities of these terpenes are compared with experimental values, and predicted cetane numbers and energy densities are used to comment on the compounds’ ability to act as fuels/additives. Expected prediction errors for CN, YSI, and LHV, defined by blind test set median absolute error, are within 5.56 cetane units, 3.63 yield sooting index units, and 0.77 MJ/kg respectively. Additionally, the present work investigates a variety of correlation/dependence metrics for property-property relationships, furthering our understanding of how combustion-relevant properties are related.

09 BIOMASS FUELS↗
DOE OSTI2025

Development of a test system for data links of the ATLAS Inner Tracker (ITk) upgrade silicon pixel detector

This contribution introduces a novel test system developed to evaluate the signal transmission quality in high-speed data links for the 2026 Inner Tracker (ITk) upgrade of the ATLAS experiment. Using an FPGA-based data acquisition (DAQ) framework, the setup can run simultaneous Bit Error Rate (BER) tests for up to 64 channels and generate virtual eye diagrams, for qualifying the ∼26k electrical links at the ATLAS ITk data rate of 1.28 Gb/s. The paper includes results from system calibration, yielding its contribution to the measured losses, and preliminary results from tests of prototype and pre-production assemblies of on-detector links of the three ATLAS ITk Pixel subsystems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Combining compositional data sets introduces error in covariance network reconstruction

Microbial communities are diverse biological systems that include taxa from across multiple kingdoms of life. Notably, interactions between bacteria and fungi play a significant role in determining community structure. However, these statistical associations across kingdoms are more difficult to infer than intra-kingdom associations due to the nature of the data involved using standard network inference techniques. We quantify the challenges of cross-kingdom network inference from both theoretical and practical points of view using synthetic and real-world microbiome data. We detail the theoretical issue presented by combining compositional data sets drawn from the same environment, e.g. 16S and ITS sequencing of a single set of samples, and we survey common network inference techniques for their ability to handle this error. We then test these techniques for the accuracy and usefulness of their intra- and interkingdom associations by inferring networks from a set of simulated samples for which a ground-truth set of associations is known. We show that while the two methods mitigate the error of cross-kingdom inference, there is little difference between techniques for key practical applications including identification of strong correlations and identification of possible keystone taxa (i.e. hub nodes in the network). Furthermore, we identify a signature of the error caused by transkingdom network inference and demonstrate that it appears in networks constructed using real-world environmental microbiome data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Processing Aleatory and Epistemic Uncertainties in Experimental Data From Sparse Replicate Tests of Stochastic Systems for Real-Space Model Validation

This paper presents a practical methodology for propagating and processing uncertainties associated with random measurement and estimation errors (that vary from test-to-test) and systematic measurement and estimation errors (uncertain but similar from test-to-test) in inputs and outputs of replicate tests to characterize response variability of stochastically varying test units. Also treated are test condition control variability from test-to-test and sampling uncertainty due to limited numbers of replicate tests. These aleatory variabilities and epistemic uncertainties result in uncertainty on computed statistics of output response quantities. The methodology was developed in the context of processing experimental data for “real-space” (RS) model validation comparisons against model-predicted statistics and uncertainty thereof. The methodology is flexible and sufficient for many types of experimental and data uncertainty, offering the most extensive data uncertainty quantification (UQ) treatment of any model validation method the authors are aware of. It handles both interval and probabilistic uncertainty descriptions and can be performed with relatively little computational cost through use of simple and effective dimension- and order-adaptive polynomial response surfaces in a Monte Carlo (MC) uncertainty propagation approach. A key feature of the progressively upgraded response surfaces is that they enable estimation of propagation error contributed by the surrogate model. Sensitivity analysis of the relative contributions of the various uncertainty sources to the total uncertainty of statistical estimates is also presented. Finally, the methodologies are demonstrated on real experimental validation data involving all the mentioned sources and types of error and uncertainty in five replicate tests of pressure vessels heated and pressurized to failure. Simple spreadsheet procedures are used for all processing operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Combining compositional data sets introduces error in covariance network reconstruction

Microbial communities are diverse biological systems that include taxa from across multiple kingdoms of life. Notably, interactions between bacteria and fungi play a significant role in determining community structure. However, these statistical associations across kingdoms are more difficult to infer than intra-kingdom associations due to the nature of the data involved using standard network inference techniques. We quantify the challenges of cross-kingdom network inference from both theoretical and practical points of view using synthetic and real-world microbiome data. We detail the theoretical issue presented by combining compositional data sets drawn from the same environment, e.g. 16S and ITS sequencing of a single set of samples, and we survey common network inference techniques for their ability to handle this error. We then test these techniques for the accuracy and usefulness of their intra- and inter-kingdom associations by inferring networks from a set of simulated samples for which a ground-truth set of associations is known. We show that while the two methods mitigate the error of cross-kingdom inference, there is little difference between techniques for key practical applications including identification of strong correlations and identification of possible keystone taxa (i.e. hub nodes in the network). Furthermore, we identify a signature of the error caused by transkingdom network inference and demonstrate that it appears in networks constructed using real-world environmental microbiome data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Insights into Methodologies and Operational Details of Resource Adequacy Assessment: A Case Study with Application to a Broader Flexibility Framework

Assessing and maintaining resource adequacy (RA) is a core pillar of power systems. However, recent changes in the physical makeup of these systems and the conditions under which these systems must operate have yielded a renewed interest in the methods, metrics, and assumptions that underpin RA assessments. In this paper, we systematically explore a wide range of RA modeling dimensions, including: the objective function and level of operational detail in the underlying model formulation; the quantity (look-ahead) and quality (accuracy) of data that is available for making operational decisions within those models; and the physical configuration of solar photovoltaics (PV) with battery storage hybrid resources. We apply a set of probabilistic RA tools and production cost modeling tools to a realistic test system based loosely on a future Electric Reliability Council of Texas power system dominated by solar PV resources. Under the assumptions of our system and models, we find that multi-stage probabilistic assessments may provide a more robust evaluation of RA by capturing a wider range of operational and system interactions, but this comes at a computational cost of 1-2 orders of magnitude longer run time depending on the specific configuration. In addition, the information on thermal generator availability impacts RA performance by an order of magnitude more than solar resource forecasts, which is driven by the comparatively larger magnitude of thermal outages than solar forecast errors within our test system. Lastly, the flexibility provided by hybrid and other resources can help reduce system load-shedding event frequencies and enable the system to be more robust to inaccurate forecast information, and alternative hybrid inverter sizes can impact RA levels by 1-2 orders of magnitude. Our results point to the importance of a broader flexibility framework to describe the interaction between (1) flexibility "supply" from both physical resource capabilities and operational constraints considered in the modeling, and (2) flexibility "demand" from forecast errors, thermal generator outages, and other sources of uncertainty, as well as their RA impacts. Results are likely sensitive to the system buildout explored; future work could consider additional system configurations and conditions.

ENERGY PLANNING, POLICY, AND ECONOMY,POWER TRANSMI↗
DOE OSTI2022

Group structure selection with random forests

Choosing an appropriate group structure for multigroup transport is far from an exact science. For some applications, one blindly uses a group structure developed years ago by forgotten methods. Furthermore, one sometimes uses the same group structure for a variety of problems, even if the group structure was originally developed with a certain application in mind. In this work, we create optimized group structures with simulated annealing for critical assembly test problems and apply a random forest regressor with bagging to choose the best group structure based on parameters of the different test problems. The optimized group structures were generated using a simulated annealing optimizer for several simple, spherical, and unreflected problems. The optimization was performed to minimize a cost function that included fission rate, absorption rate, leakage, and k{sub eff}. A random forest regressor was then trained on a set of International Criticality Safety Benchmark Evaluation Project inputs and used to select one of these six group structures. The trained machine learning model chose the best group structure 65% of the time, and one of the three best 89% of the time. Furthermore, it decreased the L2 error over all test problems by a factor of 25 when compared to the standard Los Alamos 70-group structure. In other words, the model chose group structure that were far more appropriate for the test problems than the traditional LANL group structure. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗