Search NASA⌕ Search

SEARCH · Search NASA

Results for “error testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Improvements to CTF Code Verification and Unit Testing (FY2020)

In 2010, the U.S. Department of Energy created its first Energy Innovation Hub, which focuses on improving Light Water Reactors (LWRs) through Modeling and Simulation. This hub, named the Consortium for the Advanced Simulation of LWRs (CASL), attempts to characterize and understand LWR behavior under normal operating conditions and use any gained insights to improve their efficiency. In collaboration with North Carolina State University (NCSU), CASL has worked extensively on the thermal-hydraulic subchannel code Coolant Boiling in Rod Arrays—Three Field (COBRA-TF). The NCSU/CASL version of COBRA-TF has been rebranded as CTF. This document focuses on code verification test problems that ensure CTF converges to the correct answer for the intended application. The suite of code verification tests are mapped to the underlying conservation equations of CTF, and significant gaps are addressed. Convergence behavior and numerical errors are quantified for each of the tests. Tests that converge at the correct rate to the corresponding analytic solution are incorporated into the CTF automated regression suite. A new verification utility is created for this purpose, which enables code verification by generalizing the process. For problems that do not behave correctly, the results are reported but the problem is not included in the regression suite. In addition to verification studies, this document also quantifies the existing tests of constitutive models. A few existing gaps are addressed by adding new unit tests.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

FY20 Improvements to CTF Code Verification and Unit Testing

In 2010, the U.S. Department of Energy created its first Energy Innovation Hub, which focuses on improving Light Water Reactors (LWRs) through Modeling and Simulation. This hub, named the Consortium for the Advanced Simulation of LWRs (CASL), attempts to characterize and understand LWR behavior under normal operating conditions and use any gained insights to improve their efficiency. In collaboration with North Carolina State University (NCSU), CASL has worked extensively on the thermal-hydraulic subchannel code Coolant Boiling in Rod Arrays–Three Field (COBRA-TF). The NCSU/CASL version of COBRA-TF has been rebranded as CTF. This document focuses on code verification test problems that ensure CTF converges to the correct answer for the intended application. The suite of code verification tests are mapped to the underlying conservation equations of CTF, and significant gaps are addressed. Convergence behavior and numerical errors are quantified for each of the tests. Tests that converge at the correct rate to the corresponding analytic solution are incorporated into the CTF automated regression suite. A new verification utility is created for this purpose, which enables code verification by generalizing the process. For problems that do not behave correctly, the results are reported but the problem is not included in the regression suite. In addition to verification studies, this document also quantifies the existing tests of constitutive models. A few existing gaps are addressed by adding new unit tests.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Z-Sequence: photometric redshift predictions for galaxy clusters with sequential random k-nearest neighbours

ABSTRACT We introduce Z-Sequence, a novel empirical model that utilizes photometric measurements of observed galaxies within a specified search radius to estimate the photometric redshift of galaxy clusters. Z-Sequence itself is composed of a machine learning ensemble based on the k-nearest neighbours algorithm. We implement an automated feature selection strategy that iteratively determines appropriate combinations of filters and colours to minimize photometric redshift prediction error. We intend for Z-Sequence to be a standalone technique but it can be combined with cluster finders that do not intrinsically predict redshift, such as our own DEEP-CEE. In this proof-of-concept study, we train, fine-tune, and test Z-Sequence on publicly available cluster catalogues derived from the Sloan Digital Sky Survey. We determine the photometric redshift prediction error of Z-Sequence via the median value of |Δ$z$|/(1 + $z$) (across a photometric redshift range of 0.05 ≤ $z$ ≤ 0.6) to be ∼0.01 when applying a small search radius. The photometric redshift prediction error for test samples increases by 30–50 per cent when the search radius is enlarged, likely due to line-of-sight interloping galaxies. Eventually, we aim to apply Z-Sequence to upcoming imaging surveys such as the Legacy Survey of Space and Time to provide photometric redshift estimates for large samples of as yet undiscovered and distant clusters.

Chan, Matthew C.↗
DOE OSTI2021

Post-landing major element quantification using SuperCam laser induced breakdown spectroscopy

The SuperCam instrument on the Perseverance Mars 2020 rover uses a pulsed 1064 nm laser to ablate targets at a distance and conduct laser induced breakdown spectroscopy (LIBS) by analyzing the light from the resulting plasma. SuperCam LIBS spectra are preprocessed to remove ambient light, noise, and the continuum signal present in LIBS observations. Prior to quantification, spectra are masked to remove noisier spectrometer regions and spectra are normalized to minimize signal fluctuations and effects of target distance. In some cases, the spectra are also standardized or binned prior to quantification. To determine quantitative elemental compositions of diverse geologic materials at Jezero crater, Mars, we use a suite of 1198 laboratory spectra of 334 well-characterized reference samples. The samples were selected to span a wide range of compositions and include typical silicate rocks, pure minerals (e.g., silicates, sulfates, carbonates, oxides), more unusual compositions (e.g., Mn ore and sodalite), and replicates of the sintered SuperCam calibration targets (SCCTs) onboard the rover. For each major element (SiO 2 , TiO 2 , Al 2 O 3 , FeO T , MgO, CaO, Na 2 O, K 2 O), the database was subdivided into five “folds” with similar distributions of the element of interest. One fold was held out as an independent test set, and the remaining four folds were used to optimize multivariate regression models relating the spectrum to the composition. We considered a variety of models, and selected several for further investigation for each element, based primarily on the root mean squared error of prediction (RMSEP) on the test set, when analyzed at 3 m. In cases with several models of comparable performance at 3 m, we incorporated the SCCT performance at different distances to choose the preferred model. Shortly after landing on Mars and collecting initial spectra of geologic targets, we selected one model per element. Subsequently, with additional data from geologic targets, some models were revised to ensure results that are more consistent with geochemical constraints. The calibration discussed here is a snapshot of an ongoing effort to deliver the most accurate chemical compositions with SuperCam LIBS.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Study of Overfitting by Machine Learning Methods Using Generalization Equations

The training error of Machine Learning (ML) methods has been extensively used for performance assessment, and its low values have been used as a main justification for complex methods such as estimator fusion and ensembles, and hyper parameter tuning. We present two practical cases where independent tests indicate that the low training error is more of a reflection of over-fitting rather than the generalization ability. We derive a generic form of the generalization equations that separates the training error terms of ML methods from their epistemic terms that correspond to approximation and learnability properties. It provides a framework to separately account for both terms to ensure an overall high generalization performance. For regression estimation tasks, we derive conditions for performance enhancements achieved by hyper parameter tuning, and fusion and ensemble methods over their constituent methods. We present experimental measurements and ML estimates that illustrate the analytical results for the throughput profile estimation of a data transport infrastructure.

Rao, Nageswara↗
DOE OSTI2020

Applying Multi-Frequency AC LOCO for Finding Sextupole Errors

A fast and precise technique of multi-frequency AC LOCO has been developed and tested for finding the sextupole alignment and field-offset errors at NSLS-II. This technique is based on sine-wave beam excitation simultaneously at multiple frequencies via different fast correctors. The unique combination of the fast speed and high measurement accuracy opens the door for two different types of potential applications: finding the sextupole alignment and field-offset errors. Beam excitation via fast correctors results in the elimination of the systematic error caused by the hysteresis effect. Systematic errors of the sextupole alignment caused by orbit drift and power supply calibration are also eliminated because the measurement takes only two minutes and does not depend on the current-to-field conversion of sextupole magnets. Furthermore, the simulation studies indicate that multi-frequency AC LOCO together with an orbit bump bigger than 0.5 mm can be applied to measure the relative field offset of a sextupole magnet with a precision better than 10%. The measurement technique is described and the results of two proof-of-principle experiments carried out at NSLS-II are presented.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

A Variational Method for Analyzing Vortex Flows in Radar-Scanned Tornadic Mesocyclones. Part III: Sensitivities to Vortex Center Location Errors

Abstract When the vortex center location is estimated from a radar-scanned tornadic mesocyclone, the estimated location is not error-free. This raises an important issue concerning the sensitivities of analyzed vortex flow (VF) fields by the VF-Var (formulated in Part I of this paper series and tested in Part II) to vortex center location errors, denoted by Δ x c . Numerical experiments are performed to address this issue with the following findings: The increase of |Δ x c | from zero to a half of vortex core radius causes large analysis error increases in the vortex core but the increased analysis errors decrease rapidly away from the vortex core especially for dual-Doppler analyses. The increased horizontal-velocity errors in the vortex core are mainly in the Δ x c -normal component, because this component varies much more rapidly than the other component along the Δ x c direction in the vortex core. The vertical variations of Δ x c distort the vertical correlation structure of Δ x c -dislocated VF-dependent background error covariance, which can increase the analysis errors in the vortex core. The dual-Doppler analyses have adequate accuracies outside the vortex core even when |Δ x c | increases to a half of vortex core radius, while single-Doppler analyses can also have adequate accuracies outside the vortex core mainly for the single-Doppler-observed velocity component. The sensitivities to Δ x c are largely unaffected by the vortex slanting. The above findings are important and useful for assessing the accuracies of analyzed VFs for real radar-observed tornadic mesocyclones. Significance Statement When the vortex center location is estimated from a radar-scanned tornadic mesocyclone, the estimated location is not error-free. This raises an issue concerning the sensitivity of analyzed vortex flow (VF) by the VF-Var (formulated in Part I of this paper series and tested with simulated radar observations in Part II) to vortex center location error. This issue and its required investigations are very important for the VF-Var to be applied to real radar-observed tornadic mesocyclones, especially in an operational setting with the WSR-88Ds. Numerical experiments are performed to address this issue. The findings from these experiments are important and useful for assessing the accuracies of VF-Var analyzed VF fields for real radar-observed tornadic mesocyclones.

Xu, Qin↗
DOE OSTI2022

Machine learning for continuous quantum error correction on superconducting qubits

Abstract Continuous quantum error correction has been found to have certain advantages over discrete quantum error correction, such as a reduction in hardware resources and the elimination of error mechanisms introduced by having entangling gates and ancilla qubits. We propose a machine learning algorithm for continuous quantum error correction that is based on the use of a recurrent neural network to identify bit-flip errors from continuous noisy syndrome measurements. The algorithm is designed to operate on measurement signals deviating from the ideal behavior in which the mean value corresponds to a code syndrome value and the measurement has white noise. We analyze continuous measurements taken from a superconducting architecture using three transmon qubits to identify three significant practical examples of non-ideal behavior, namely auto-correlation at temporal short lags, transient syndrome dynamics after each bit-flip, and drift in the steady-state syndrome values over the course of many experiments. Based on these real-world imperfections, we generate synthetic measurement signals from which to train the recurrent neural network, and then test its proficiency when implementing active error correction, comparing this with a traditional double threshold scheme and a discrete Bayesian classifier. The results show that our machine learning protocol is able to outperform the double threshold protocol across all tests, achieving a final state fidelity comparable to the discrete Bayesian classifier.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Neural networks for large eddy simulations of wall-bounded turbulence: numerical experiments and challenges

Here, we examine the application of neural network-based methods to improve the accuracy of large eddy simulations of incompressible turbulent flows. The networks are trained to learn a mapping between flow features and the subgrid scales, and applied locally and instantaneously—in the same way as traditional physics-based subgrid closures. Models that use only the local resolved strain rate are poorly correlated with the actual subgrid forces obtained from filtering direct numerical simulation data. We see that highly accurate models in a priori testing are inaccurate in forward calculations, owing to the preponderance of numerical errors in implicitly filtered large eddy simulations. A network that accounts for the discretization errors is trained and found to be unstable in a posteriori testing. We identify a number of challenges that the approach faces, including a distribution shift that affects networks that fail to account for numerical errors.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Excerpt from Oxygen Monitor Clean Testing Report Performed in 2010

The Rosemount X-STREAM Series X2GP is a paramagnetic oxygen analyzer with a 0-25% oxygen measurement range. It claims a drift less than 2% of span (0.10% volume oxygen) per week, and a response time of less than 5 seconds (for the analyzer to report 90% of a step change). Our test equipment was set-up such that the same gas was running through each oxygen analyzer at the same time. This allowed Calibration Error, Calibration Drift, and Loss of Power Testing to be run on all monitors at the same time, eliminating any variability between the analyzers from tolerances in the gas cylinder, ambient temperature, ambient pressure, test duration, etc. An APTech pressure regulator was placed on the gas cylinder manifold and a 23 psia Paroscientific pressure transducer measured the regulator’s outlet pressure. This did not give the inlet pressure on any one analyzer but outlet pressure from the manifold before the stream was split. The APTech pressure regulator was used to keep operating conditions consistent. A DWYER rotameter was installed on each oxygen monitor’s outlet piping. With the exception of Variation of Flow Rate Testing, the rotameter was left fully open for all testing. The flow rate through each oxygen monitor was controlled by a Swagelok needle valve that was installed on each analyzer’s inlet piping. Additionally, a filter was installed in the each monitor’s inlet piping to keep particles from the measurement cells and a Swagelok ball valve was installed on each monitor’s outlet piping to isolate each monitor between tests. No pumps were installed. Each of the cylinders used was at a pressure greater than ambient room barometric pressure. This pressure differential was used to move gas through the testing equipment. The flow rate through each analyzer was controlled by the Swagelok needle valve installed on each analyzer’s inlet piping. Finally, for all tests, a Paroscientific Digiquartz Barometer was used to measure ambient room barometric pressure and a Fluke Thermo-hygrometer was used to measure ambient room temperature.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Stability boundary and optimal operating parameter identification in milling using Bayesian learning

Here, this paper describes a novel Bayesian learning approach for stability boundary and optimal parameter identification in milling without the knowledge of the underlying tool dynamics or material cutting force coefficients. The paper is divided into two parts. First, a Bayesian learning method for stability lobe identification using test results is described. Each axial depth and spindle speed combination is characterized by a probability of stability which is updated using Bayes’ rule when a test result (stable or unstable) is made available. A novel likelihood function is defined which incorporates knowledge of the stability behavior. Numerical results show convergence to the analytical stability lobe diagram. Second, an adaptive experimental strategy to identify optimal operating parameters that maximize material removal rate is described. Numerical evaluation shows convergence to the optimal operating point with error less than 15 % within ten tests on average. The approach is validated using experimental results. Results show that the proposed method is an efficient and robust learning method to identify the stability lobe diagram and optimal operating parameters with a limited number of tests/data points.

Bayesian machine learning↗
DOE OSTI2024

Sensitive and error-tolerant annotation of protein-coding DNA with BATH

We present BATH, a tool for highly sensitive annotation of protein-coding DNA based on direct alignment of that DNA to a database of protein sequences or profile hidden Markov models (pHMMs). BATH is built on top of the HMMER3 code base, and simplifies the annotation workflow for pHMM-based translated sequence annotation by providing a straightforward input interface and easy-to-interpret output. BATH also introduces novel frameshift-aware algorithms to detect frameshift-inducing nucleotide insertions and deletions (indels). BATH matches the accuracy of HMMER3 for annotation of sequences containing no errors, and produces superior accuracy to all tested tools for annotation of sequences containing nucleotide indels. These results suggest that BATH should be used when high annotation sensitivity is required, particularly when frameshift errors are expected to interrupt protein-coding regions, as is true with long-read sequencing data and in the context of pseudogenes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Performance and automatic calibration scheme of the waveform sampler in the ETROC2 ASIC chip

The waveform sampler in the CMS ETROC2 chip for LGAD gain aging monitoring is a 2.56-GS/s 12-bit 8x-Interleaved ADC that consists of a coarse SAR stage, and a fine stage. This architecture delivers high performance on a relatively modest 65 nm process, while requires finding up to 24 calibration constants through calibration. We developed an automatic calibration method using charge injection test data. After calibration, the baseline random error is reduced by a factor of 2.5–3 compared to the default calibration, and a 5% charge measurement precision is achieved in 15 fC charge injection tests.

Fu, Tao [Unlisted, US]↗
DOE OSTI2020

Development and Implementation of a CTF Code Verification Suite.

CTF is a thermal hydraulic subchannel code developed to predict light water reactor (LWR) core behavior. It is a version of Coolant Boiling in Rod Arrays (COBRA) developed by Oak Ridge National Laboratory (ORNL) and North Carolina State University (NCSU) and used in the Consortium for the Advanced Simulation of LWRs (CASL). Here, the existing CTF code verification matrix is expanded, which ensures that the code is a faithful representation of the underlying mathematical model. The suite of code verification tests are mapped to the underlying conservation equations of CTF and significant gaps are addressed. As such, five new problems are incorporated: isokinetic advection, conduction, pressure drop, convection, and pipe boiling. Convergence behavior and numerical errors are quantified for each of the tests and all tests converge at the correct rate to their corresponding analytic solution. A new verification utility that generalizes the code verification process is used to incorporate these problems into the CTF automated test suite.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Predicting the von Neumann entanglement entropy using a graph neural network

Calculating the von Neumann entanglement entropy from experimental data is challenging due to its dependence on the complete wavefunction, forcing reliance on approximations such as classical mutual information (MI). We propose a machine learning approach using a graph neural network to predict the von Neumann entropy directly from experimentally accessible bitstrings. We test this approach on a Rydberg ladder system and achieve a mean absolute error of $3.6\,\times 10^{-3}$ when evaluating within the training range on a dataset with entropy values ranging from 0 to 1.9. The model achieves a mean absolute percentage error of 1.44% and outperforms MI-based bounds. When tested beyond the training range, the model maintains reasonable accuracy. Furthermore, we demonstrate that fine-tuning the model with small datasets significantly improves performance on data outside the original training range.

graph neural networks↗
DOE OSTI2025

Marine and continental stratocumulus cloud microphysical properties obtained from routine ARM Cimel sunphotometer observations

This study investigates marine and continental stratocumulus (Sc) cloud properties obtained from an automated implementation of a multispectral photometer retrieval. Photometer methods simultaneously retrieve cloud optical depth (τ) and cloud droplet effective radius (r e ), with estimates for liquid water path (LWP) calculated on the availability of those quantities. These applied methods evaluate retrieved cloud properties for Sc identified during a recent 6 year period over the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program sites in Oklahoma, USA (SGP) and in the Azores, Portugal (ENA). Modest agreement in key quantity retrievals is found between the routine photometer products and multisensor collocated profiling references. Cumulative breakdowns contingent on cloud thickness indicate increases in all retrieved quantities in thicker clouds, with larger discrepancies in the relative performance between the retrievals collected in the presence of drizzle. Under continental cloud conditions, the clouds of a similar thickness and r e to those sampled under marine conditions report a factor of 1.5 larger τ and LWP. An r 2 ≅0.65 is found between photometer τ retrievals and shadowband radiometer measurements, with photometer retrievals reporting a high (relative) bias. The τ intercomparisons indicate that variability between retrievals is a factor of three larger than errors reported from individual retrieval input perturbation tests. Photometer r e retrievals suggest a low r 2 (< 0.1) having a standard deviation ≅ 3 µm when compared to ARM baseline multi-sensor radar/radiometer references (accounting for offsets in the cloud droplet number concentration assumptions of the latter). However, photometer LWP calculations remain relatively unbiased in non-drizzling conditions, with errors O (50 g m −2 ) and r 2 ≅0.5 to collocated radiometer and interferometer references. Additional sensitivity tests for island influences on marine Sc properties suggest that while island-influenced winds may promote larger cloud LWP or thickness, the influence could be within retrieval method uncertainty and/or collocated instrument variability.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

State-Space Model to Estimate Salmon Escapement Using Multiple Data Sources

Abstract Accurate estimates of salmonids passing Lower Granite Dam on the Snake River, by species and origin, are a critical input to assessing the status and trends of various populations as well as successful management of fisheries in the Snake River basin. Here, we describe a state-space model that estimates such escapement past a dam by using window counts, PIT tag observations, and data from an adult fish trap, accounting for issues such as nighttime passage, fallback and reascension, potential observation error at the window, and uncertainty in the adult trap rate. We tested the approach using a simulation framework that mimicked several levels of observation error, differences between nighttime passage and reascension rates, and the possibility of the adult trap being closed for some period of time. Our results demonstrate that the model produced unbiased estimates across all tested scenarios. We also applied this model to empirical data from Lower Granite Dam to produce estimates of wild, clipped hatchery, and unclipped hatchery spring/summer-run Chinook Salmon Oncorhynchus tshawytscha and steelhead O. mykiss from spawn years 2010–2019.

See, Kevin E. (ORCID:0000000297626442)↗
DOE OSTI2020

Simcenter TestLab Error in uploading reference profiles

Our team exports data from Matlab to a Simcenter TestLab formatted .mat file. The .mat file is then uploaded as a reference profile for Environmental Random Control vibration testing. Operations are applied to the data before export that introduce floating point errors. We are experiencing errors uploading the generated reference profiles if the frequency values have a certain level of floating point error (1 Hz vs 1.000000000000001 Hz). I generated some test cases of reference profiles that were giving us trouble, and outlined the errors we were seeing. Two different errors, depending on how the control frequency range is defined. Error is fixed by copy/pasting the frequency column in the Profile Editor.

97 MATHEMATICS AND COMPUTING↗