Modern sensitivity analysis - 15651
Sensitivity analysis has been applied to complex models, such as performance assessment models, since at least the advent of computational tools to support such an approach to model evaluation. Initially, sensitivity analysis methods were applied to deterministic models and focused on changing one input variable at a time. Probabilistic modeling opened the door to evaluating changes in all input variables simultaneously. Standardized rank regression methods were primarily used in the last century to evaluate probabilistic models. These methods assume linearity in the rank response of the model to the inputs. At the very least this assumes monotonicity in the response, but for the most part the response needs to be near-linear. This is highly unlikely for performance assessment models that are based on differential equations, multiplication and division of variables that are usually not correlated, and conservatism that is not well understood. There are better statistical methods for evaluating highly non-linear and dynamic probabilistic models. Some of these methods were founded in the data mining industry. This century has seen a rapid development of the internet that led directly to the development of data mining techniques for massive databases. Although the goals were different, these techniques can be adapted to sensitivity analysis for complex probabilistic models, which, also, form large, or even massive, databases. These modern sensitivity analysis procedures assess the importance of every explanatory variable (input parameters) used in a probabilistic model. In practice, this means that for each endpoint of interest, every explanatory variable in the model is associated with a quantitative measure of importance. For a given explanatory variable, the importance value depends on the endpoint of interest. This global approach to sensitivity analysis allows all levels of interactions to be evaluated, so that the variation in the output can be measured at every level of possible interaction. The effects are collected together during sensitivity analysis processing, and then separated out to attribute the overall contribution of each input parameter to the model output. These global sensitivity analysis procedures have the ability to characterize non-linear and non-monotonic relationships between explanatory variables and endpoints of interest. This is critically important because of the need to characterize complex interactions among multiple explanatory variables in PA models that often have non-linear and non-monotonic effects. The options for sensitivity analysis are fully explored, including an evaluation of several methods when applying them to a well-known complex non-linear non-monotonic model. The results clearly demonstrate the relative benefit of using data mining methods over simple statistical approaches such as standardized rank regression. (authors)