Search NASA⌕ Search

SEARCH · Search NASA

Results for “solution verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Code Verification and Solution Verification framework in pin-resolved neutron transport code MPACT

Program verification in scientific computing encompasses the application of formal and mathematical techniques to a scientific computing code for its credibility, accuracy, and validity. Code Verification identifies bugs and performance issues in the software development stage. Solution Verification assesses the applicability of the code and the accuracy of the solution to problems of interest. Both activities utilize application cases and quantify the error against prescribed acceptance criteria. However, simply executing more application cases does not guarantee stronger or more comprehensive credibility. Here, we establish a verification framework that involves Code Verification and Solution Verification, both of which work together such that the overarching goal of “converge to the correct answer for the intended application” can be reasonably inferred. The application of such a verification framework is demonstrated using the pin-resolved neutron transport code MPACT, where standard unit tests and regression tests are covered, and where the Method of Exact Solutions and the Method of Manufactured Solutions are successfully used. Additionally, the applicability of Method of Manufactured Solutions is extended to the OECD/NEA C5G7 benchmark problems of practical material and geometric configurations. Solution Verification activities are demonstrated on a practical hierarchy of application models of increasing complexity ranging from 2D pin cell problems to 3D assembly problems. The convergence behavior and rate of convergence with respect to each individual variable are studied and provided. This framework can be adapted broadly to other fields involving scientific computing codes.

97 MATHEMATICS AND COMPUTING↗

CTF Code Verification and Solution Verification

The CTF code is part of the suite of codes developed with DOE funding under the CASL Program to bring the benefits of high-performance computing to issues of importance to the US nuclear power industry. This document primarily focuses on code and solution verification test problems that have been developed to ensure that CTF converges to the correct answer for the intended application. Convergence behavior and numerical errors are quantified for each of the tests. The observed convergence behavior in all available tests is consistent with expectations for the numerical methods employed by CTF, providing evidence that there are no significant code bugs or algorithm deficiencies in the code. The suite of code verification and solution verification tests are mapped to core capabilities of CTF. Gaps were identified in the code verification test suite and they are being addressed. Two new code verification problems were developed to address gaps that were identified.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Verification and Validation Propensity Improvements via Solution Verification Automation: Presentation Package (Part 1 of 2) (FY15 ASC L2 milestone #5320)

A new tool, called Mesh Scaling, for producing series of hexahedral meshes suitable for solution verification was enhanced and hardened by this milestone. In addition, solution verification using the meshes produced from Mesh Scaling was performed and documented. We conclude that Mesh Scaling now produces meshes suitable for solution verification, while offering a substantial decrease in the computational cost of solution verification.

97 MATHEMATICS AND COMPUTING↗

Code and Solution Verification Assessment of the CTF Thermal Hydraulic Subchannel Code

CTF is a thermal-hydraulics subchannel code jointly developed by Oak Ridge National Laboratory and North Carolina State University. Over the past seven years, the Consortium for Advanced Simulation of Light Water Reactors (CASL) has made a significant investment in developing CTF so it can be used to model light water reactors, including nominal operating conditions, departure from nucleate boiling analysis, and transients ranging from loss of flow to reactivity insertion accidents. In addition to implementing new modeling capabilities and developing the user input and output interface, extensive work has been performed to improve the code’s quality assurance program, resulting in a development process that conforms with NQA-1 requirements. The CASL program follows the Predictive Capability Maturity Model (PCMM) approach for assessing code quality, which emphasizes performing code verification(ensuring the code converges to the correct answer) and solution verification (ensuring the code converges for the intended application). Code and solution verification are used to identify uncertainty errors introduced by numerical approximations in the code and are important for demonstrating that the model is coded without error, which is an important aspect of the Best Estimate plus Uncertainty method. This paper presents a comprehensive overview of the code and solution verification testing that has been performed on CTF. A top-down approach is taken in which the intended CTF applications are presented, followed by the code features required for their modeling. These features are then linked to the applicable code and solution verification tests that demonstrate proper functioning. Past testing efforts are summarized, and new tests are added to help close gaps in the presented test matrix. Rather than performing “one-off” exercises, these tests are added to the automated CTF regression test suite to ensure continual code quality.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗

An Investigation into Solution Verification for CFD-DEM

This report presents the study of the convergence behavior of the computational fluid dynamicsdiscrete element method (CFD-DEM) method, specifically National Energy Technology Laboratory’s (NETL) open source MFiX code (MFiX-DEM) with a diffusion based particle-tocontinuum filtering scheme. In particular, this study focused on determining if the numerical method had a solution in the high-resolution limit where the grid size is smaller than the particle size. To address this uncertainty, fixed particle beds of two primary configurations were studied: i) fictitious beds where the particles are seeded with a random particle generator, and ii) instantaneous snapshots from a transient simulation of an experimentally relevant problem. Both problems considered a uniform inlet boundary and a pressure outflow. The CFD grid was refined from a few particle diameters down to 1/6 th of a particle diameter. The pressure drop between two vertical elevations, averaged across the bed cross-section was considered as the system response quantity of interest. A least-squares regression method was used to extrapolate the grid-dependent results to an approximate “grid-free” solution in the limit of infinite resolution. The results show that the diffusion based scheme does yield a converging solution. However, the convergence is more complicated than encountered in simpler, single-phase flow problems showing strong oscillations and, at times, oscillations superimposed on top of globally non-monotonic behavior. The challenging convergence behavior highlights the importance of using at least four grid resolutions in solution verification problems so that (over-determined) regression-based extrapolation methods may be applied to approximate the grid-free solution. The grid-free solution is very important in solution verification and VVUQ exercise in general as the difference between it and the reference solution largely determines the numerical uncertainty. By testing different randomized particle configurations of the same general problem (for the fictitious case) or different instances of freezing a transient simulation, the numerical uncertainties appeared to be on the same order of magnitude as ensemble or time averaging uncertainties. By testing different drag laws, almost all cases studied show that model form uncertainty in this one, very important closure relation was larger than the numerical uncertainty, at least with a reasonable CFD grid, roughly five particle diameters. In this study, the diffusion width (filtering length scale) was mostly set at a constant of six particle diameters. A few exploratory tests were performed to show that similar convergence behavior was observed for diffusion widths greater than approximately two particle diameters. However, this subject was not investigated in great detail because determining an appropriate filter size is really a validation question which must be determined by comparison to experimental or highly accurate numerical data. Future studies are being considered targeting solution verification of transient simulations as well as validation of the filter size with direct numerical simulation data.

97 MATHEMATICS AND COMPUTING↗

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗

An Open-Source Python Package for CFD Solution Verification

Informed decision-making using computational fluid dynamics (CFD) results requires quantifying the errors and uncertainties of a simulation. Verification, validation, and uncertainty quantification (VVUQ) methods were developed to address this need and have matured. However, these VVUQ analyses are often non-trivial and require CFD analysts and practitioners to have specific skill sets. This has led to the uneven adoption of VVUQ analyses, in part, based on the availability of software tools to aid CFD analysts and practitioners. Solution verification, a procedure to evaluate the accuracy of a simulation by estimating potential errors arising from the computational model and computing the uncertainties without comparing to results from a physical system, is one of the lagging VVUQ analyses as the absence of software has forced CFD analysts and practitioners to develop their own codes or piece together incomplete software from across the internet. This work presents an opensource Python package, CFDverify, to lower the barrier of entry and fill in the technological gap in solution verification. CFDverify also provides a streamlined framework to remove some potential errors in post-processing CFD results. The hope is that CFDverify can improve the quality and quantity of CFD solution verification in scientific and research studies and attract interest in developing a communal tool. This paper describes the design, features, and an example use of CFDverify.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗

Computational solution verification and validation applied to a thermal model of a ruggedized instrumentation package

This study details a methodology for quantification of errors and uncertainties of a finite element heat transfer model applied to a Ruggedized Instrumentation Package (RIP). The proposed verification and validation (V&V) process includes solution verification to examine errors associated with the code's solution techniques, and model validation to assess the model's predictive capability for quantities of interest. The model was subjected to mesh resolution and numerical parameters sensitivity studies to determine reasonable parameter values and to understand how they change the overall model response and performance criteria. To facilitate quantification of the uncertainty associated with the mesh, automatic meshing and mesh refining/coarsening algorithms were created and implemented on the complex geometry of the RIP. Automated software to vary model inputs was also developed to determine the solution’s sensitivity to numerical and physical parameters. The model was compared with an experiment to demonstrate its accuracy and determine the importance of both modelled and unmodelled physics in quantifying the results' uncertainty. An emphasis is placed on automating the V&V process to enable uncertainty quantification within tight development schedules.

97 MATHEMATICS AND COMPUTING↗

Solution verification, goal-oriented adaptive methods for stochastic advection–diffusion problems

A goal-oriented analysis of linear, stochastic advection–diffusion models is presented which provides both a method for solution verification as well as a basis for improving results through adaptation of both the mesh and the way random variables are approximated. A class of model problems with random coefficients and source terms is cast in a variational setting. Specific quantities of interest are specified which are also random variables. A stochastic adjoint problem associated with the quantities of interest is formulated and a posteriori error estimates are derived. These are used to guide an adaptive algorithm which adjusts the sparse probabilistic grid so as to control the approximation error. Numerical examples are given to demonstrate the methodology for a specific model problem.

97 MATHEMATICS AND COMPUTING↗

CFD-DEM solution verification: Fixed-bed studies

Fixed (frozen) particle configurations are used to study CFD-DEM convergence behavior in the limit of infinite grid resolution. A filtering scheme based on solving a diffusion equation, i.e., a Gaussian kernel, is used to interpolate particle information to the continuum grid, free of cell size limitations. A regression method is used for Richardson extrapolation of the discrete solutions to the grid-free solution. The regression method is shown to predict accurate extrapolation curves in the presence of the observed oscillatory convergence behavior. The extrapolation curve is used to complete the solution verification process by quantifying the numerical uncertainty present in the calculations. Several numerical and model parameters are studied including the spatial discretization method, the kernel width and the mean drag law. Here, a manageable level of discretization error is found with relative discretization errors generally <4%.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Modeling and Simulation Credibility Assessment of Finite Element Models for Fall From Heights Injury Scenarios

Finite element (FE) computational models of the whole human body are being used to investigate the injury modalities and mechanisms arising from suited astronauts’ ExtraVehicular Activities (EVAs). These FE whole body models – simplified Elemance and THUMS pedestrian versions – have undergone a series of verification and validation (V&V) assessments for applications in automotive, domestic, and contact sports arenas. However, the FE models’ Modeling and Simulation (M&S) credibility assessments for simulating suited astronaut injuries due to fall from heights in altered gravity environments are limited. The current study evaluates the M&S credibility of simplified Elemance and THUMS pedestrian FE models based on eight credibility factors – data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty, and results robustness – as per NASA-STD-7009A [1]. These credibility factors were assigned ordinal scores ranging between 0 to 4, with a score of 0 in the case of insufficient evidence and 4 when all data for the model is known, used appropriately, and traceable to the Real-World System (RWS) with acceptable accuracy, precision, and uncertainty. The credibility factor sufficiency thresholds were assigned upon guidance from NASA EVA subject matter experts. The factor scores were ascertained through the evaluation of M&S credibility assessment procedures described in the published literature and FE model reports. The M&S credibility scores for the Elemance and THUMS FE models vary between 1 and 3 for data pedigree, input pedigree, and referent validation credibility factors, and between 0 and 1 for code verification, solution verification, conceptual validation, results uncertainty and results robustness. The results show that these FE models are credible for simulating fall from heights injury scenarios in specific kinematic and kinetic ranges that correlate with highway and contact sports-related external conditions. Multiple credibility elevation strategies are prescribed to improve the Elemance and THUMS models’ credibility for use in the EVA fall from height injury scenarios.

R A Perkins↗