Search NASA⌕ Search

SEARCH · Search NASA

Results for “data base for model validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Dark energy survey: Modeling strategy for multiprobe cluster cosmology and validation for the full six-year dataset

Here, we introduce an updated To&Krause2021 model for joint analyses of cluster abundances and large-scale two-point correlations of weak lensing and galaxy and cluster clustering (termed CL+3×2 pt analysis) and validate that this model meets the systematic accuracy requirements of analyses with the statistical precision of the final Dark Energy Survey (DES) Year 6 (Y6) dataset. The validation program consists of two distinct approaches, (i) identification of modeling and parametrization choices and impact studies using simulated analyses with each possible model misspecification and (ii) end-to-end validation using mock catalogs from customized Cardinal simulations that incorporate realistic galaxy populations and DES-Y6-specific galaxy and cluster selection and photometric redshift modeling, which are the key observational systematics. In combination, these validation tests indicate that the model presented here meets the accuracy requirements of DES-Y6 for CL+3×2 pt based on a large list of tests for known systematics. In addition, we also validate that the model is sufficient for several other data combinations: the CL+GC subset of this data vector (excluding galaxy–galaxy lensing and cosmic shear two-point statistics) and the CL+3×2 pt+BAO+SN (combination of CL+3×2 pt with the previously published Y6 DES baryonic acoustic oscillation and Y5 supernovae data).

79 ASTRONOMY AND ASTROPHYSICS↗

Final technical report for DE-FG02-06ER54871 Multi-Machine Validation of Fluid and Gyrokinetic Turbulence Models

This final report for the DE-FG02-06ER54871 “Multi-Machine Validation of Fluid and Gyrokinetic Turbulence Models” award (previously titled “Validation of Plasma Turbulence Models” during 2006-2010) documents accomplishments made over the full funding period of 6/1/2006 – 5/31/2018. The accomplishments can be broadly divided into two groups. First are integrated experimental and computational studies of fundamental drift-wave turbulence and shear flow dynamics, carried out using the Controlled Shear Decorrelation Experiment (CSDX) linear plasma experiment at UCSD and various fluid-based simulation models. The second set of accomplishments relates to validation studies of gyrokinetic and gyrofluid turbulence models of core tokamak turbulence, generally using data from the DIII-D tokamak operated by General Atomics. A number of results bridge these areas, such as investigations into bispectral energy transfer and shear flow generation at L-H transitions, carried out on a number of different experimental platforms. Some of the key accomplishments supported by this work include (1) the first experimental observations of nonlinear shear flow generation by drift-wave turbulence in a magnetized plasma, (2) detailed documentation of nonlinear interactions and energy transfer between drift-waves and zonal flows in both linear devices and tokamaks, (3) experimental and numerical investigations of parallel boundary condition effects on resistive drift-wave turbulence, (4) pioneering development of synthetic diagnostics for core tokamak turbulence validation studies, along with a system of validation metrics, and (5) extensive investigation of the fidelity of gyrokinetic and gyrofluid turbulent transport models in tokamaks which have significantly advanced our understanding of where these models can (and can not) reproduce experimental observations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Dark Energy Survey: Modeling strategy for multiprobe cluster cosmology and validation for the Full Six-year Dataset

We introduce an updated To&Krause2021 model for joint analyses of cluster abundances and large-scale two-point correlations of weak lensing and galaxy and cluster clustering (termed CL+3x2pt analysis) and validate that this model meets the systematic accuracy requirements of analyses with the statistical precision of the final Dark Energy Survey (DES) Year 6 (Y6) dataset. The validation program consists of two distinct approaches, (1) identification of modeling and parameterization choices and impact studies using simulated analyses with each possible model misspecification (2) end-to-end validation using mock catalogs from customized Cardinal simulations that incorporate realistic galaxy populations and DES-Y6-specific galaxy and cluster selection and photometric redshift modeling, which are the key observational systematics. In combination, these validation tests indicate that the model presented here meets the accuracy requirements of DES-Y6 for CL+3x2pt based on a large list of tests for known systematics. In addition, we also validate that the model is sufficient for several other data combinations: the CL+GC subset of this data vector (excluding galaxy--galaxy lensing and cosmic shear two-point statistics) and the CL+3x2pt+BAO+SN (combination of CL+3x2pt with the previously published Y6 DES baryonic acoustic oscillation and Y5 supernovae data).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Evaluation of regional isoprene emission factors and modeled fluxes in California

Accurately modeled biogenic volatile organic compound (BVOC) emissions are a vital input to atmospheric chemistry simulations of ozone and particle formation. BVOC emission models rely on basal emission factor (BEF) distribution maps based on emission measurements and vegetation land-cover data but these critical input components of the models as well as model simulations lack validation by regional scale measurements. We directly assess isoprene emission-factor distribution databases for BVOC emission models by deriving BEFs from direct airborne eddy covariance (AEC) fluxes (Misztal et al., 2014) scaled to the surface and normalized by the activity factor of the Guenther et al. (2006) algorithm. The available airborne BEF data from approx. 10 000 km of flight tracks over California were averaged spatially over 48 defined ecological zones called ecoregions. Consistently, BEFs used by three different emission models were averaged over the same ecoregions for quantitative evaluation. Ecoregion-averaged BEFs from the most current land cover used by the Model of Emissions of Gases and Aerosols from Nature (MEGAN) v.2.1 resulted in the best agreement among the tested land covers and agreed within 10 % with BEFs inferred from measurement. However, the correlation was sensitive to a few discrepancies (either overestimation or underestimation) in those ecoregions where land-cover BEFs are less accurate or less representative for the flight track. The two other land covers demonstrated similar agreement (within 30 % of measurements) for total average BEF across all tested ecoregions but there were a larger number of specific ecoregions that had poor agreement with the observations. Independently, we performed evaluation of the new California Air Resources Board (CARB) hybrid model by directly comparing its simulated isoprene area emissions averaged for the same flight times and flux footprints as actual measured area emissions. The model simulation and the observed surface area emissions agreed on average within 20 %. We demonstrate that the choice of model land-cover input data has the most critical influence on model-measurement agreement and the uncertainty in meteorology inputs has a lesser impact at scales relevant to regional air quality modeling.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Fusion of Multiple Models for Improving Gross Primary Production Estimation With Eddy Covariance Data Based on Machine Learning

Abstract Terrestrial gross primary production (GPP) represents the magnitude of CO 2 uptake through vegetation photosynthesis, and is a key variable for carbon cycles between the biosphere and atmosphere. Light use efficiency (LUE) models have been widely used to estimate GPP for its physiological mechanisms and availability of data acquisition and implementation, yet each individual GPP model has exhibited large uncertainties due to input errors and model structure, and further studies of systematic validation, comparison, and fusion of those models with eddy covariance (EC) site data across diverse ecosystem types are still needed in order to further improve GPP estimation. We here compared and fused five GPP models (VPM, EC‐LUE, GOL‐PEM, CHJ, and C‐Fix) across eight ecosystems based on FLUXNET2015 data set using the ensemble methods of Bayesian Model Averaging (BMA), Support Vector Machine (SVM), and Random Forest (RF) separately. Our results showed that for individual models, EC‐LUE gave a better performance to capture interannual variability of GPP than other models, followed by VPM and GLO‐PEM, while CHJ and C‐Fix were more limited in their estimation performance. We found RF and SVM were superior to BMA on merging individual models at various plant functional types (PFTs) and at the scale of individual sites. On the basis of individual models, the fusion methods of BMA, SVM, and RF were examined by a five‐fold cross validation for each ecosystem type, and each method successfully improved the average accuracy of estimation by 8%, 18%, and 19%, respectively.

Environmental Sciences & Ecology↗

An Approach for Validating Actinide and Fission Product Burnup Credit Criticality Safety Analyses-Isotopic Composition Predictions

The expanded use of burnup credit in the United States (U.S.) for storage and transport casks, particularly in the acceptance of credit for fission products, has been constrained by the availability of experimental fission product data to support code validation. The U.S. Nuclear Regulatory Commission (NRC) staff has noted that the rationale for restricting the Interim Staff Guidance on burnup credit for storage and transportation casks (ISG-8) to actinide-only is based largely on the lack of clear, definitive experiments that can be used to estimate the bias and uncertainty for computational analyses associated with using burnup credit. To address the issues of burnup credit criticality validation, the NRC initiated a project with the Oak Ridge National Laboratory to (1) develop and establish a technically sound validation approach for commercial spent nuclear fuel (SNF) criticality safety evaluations based on best-available data and methods and (2) apply the approach for representative SNF storage and transport configurations/conditions to demonstrate its usage and applicability, as well as to provide reference bias results. The purpose of this paper is to describe the isotopic composition (depletion) validation approach and resulting observations and recommendations. Validation of the criticality calculations is addressed in a companion paper at this conference. For isotopic composition validation, the approach is to determine burnup-dependent bias and uncertainty in the effective neutron multiplication factor (keff) due to bias and uncertainty in isotopic predictions, via comparisons of isotopic composition predictions (calculated) and measured isotopic compositions from destructive radiochemical assay utilizing as much assay data as is available, and a best-estimate Monte Carlo based method. This paper (1) provides a detailed description of the burnup credit isotopic validation approach and its technical bases, (2) describes the application of the approach for representative pressurized water reactor and boiling water reactor safety analysis models to demonstrate its usage and applicability, (3) provides reference bias and uncertainty results based on a quality-assurance-controlled prerelease version of the Scale 6.1 code package and the ENDF/B-VII nuclear cross section data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Laser Hardening Prediction Tool Based On a Solid State Transformations Numerical Model

This paper presents a tool to predict hardening layer in selective laser hardening processes where laser beam heats the part locally while the bulk acts as a heat sink.The tool to predict accurately the temperature field in the workpiece is a numerical model that combines a three dimensional transient numerical solution for heating where is possible to introduce different laser sources. The thermal field was modeled using a kinetic model based on Johnson-Mehl-Avrami equation. Considering this equation, an experimental adjustment of transformation parameters was carried out to get the heating transformation diagrams (CHT). With the temperature field and CHT diagrams the model predicts the percentage of base material converted into austenite. These two parameters are used as first step to estimate the depth of hardened layer in the part.The model has been adjusted and validated with experimental data for DIN 1.2379, cold work tool steel typically used in mold and die making industry. This steel presents solid state diffusive transformations at relative low temperature. These transformations must be considered in order to get good accuracy of temperature field prediction during heating phase. For model validation, surface temperature measured by pyrometry, thermal field as well as the hardened layer obtained from metallographic study, were compared with the model data showing a good adjustment.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Guide to Using the WIND Toolkit Validation Code

In response to the U.S. Department of Energy's goal of using 20% wind energy by 2030, the Wind Integration National Dataset (WIND) Toolkit was created to provide information on wind speed, wind direction, temperature, surface air pressure, and air density on more than 126,000 locations across the United States from 2007 to 2013. The numerical weather prediction model output, gridded at 2-km and at a 5-minute resolution, was further converted to detail the wind power production time series of existing and potential wind facility sites. For users of the dataset it is important that the information presented in the WIND Toolkit is accurate and that errors are known, as then corrective steps can be taken. Therefore, we provide validation code written in R that will be made public to provide users with tools to validate data of their own locations. Validation is based on statistical analyses of wind speed, using error metrics such as bias, root-mean-square error, centered root-mean-square error, mean absolute error, and percent error. Plots of diurnal cycles, annual cycles, wind roses, histograms of wind speed, and quantile-quantile plots are created to visualize how well observational data compares to model data. Ideally, validation will confirm beneficial locations to utilize wind energy and encourage regional wind integration studies using the WIND Toolkit.

17 WIND ENERGY↗

Short-Term Global Horizontal Irradiance Forecasting Based on Sky Imaging and Pattern Recognition

Accurate short-term forecasting is crucial for solar integration in the power grid. In this paper, a classification forecasting framework based on pattern recognition is developed for 1-hour-ahead global horizontal irradiance (GHI) forecasting. Three sets of models in the forecasting framework are trained by the data partitioned from the preprocessing analysis. The first two sets of models forecast GHI for the first four daylight hours of each day. Then the GHI values in the remaining hours are forecasted by an optimal machine learning model determined based on a weather pattern classification model in the third model set. The weather pattern is determined by a support vector machine (SVM) classifier. The developed framework is validated by the GHI and sky imaging data from the National Renewable Energy Laboratory (NREL). Results show that the developed short-term forecasting framework outperforms the persistence benchmark by 16% in terms of the normalized mean absolute error and 25% in terms of the normalized root mean square error.

14 SOLAR ENERGY↗

Development of QSAR models to predict blood-brain barrier permeability

Assessing drug permeability across the blood-brain barrier (BBB) is important when evaluating the abuse potential of new pharmaceuticals as well as developing novel therapeutics that target central nervous system disorders. One of the gold-standard in vivo methods for determining BBB permeability is rodent log BB; however, like most in vivo methods, it is time-consuming and expensive. In the present study, two statistical-based quantitative structure-activity relationship (QSAR) models were developed to predict BBB permeability of drugs based on their chemical structure. The in vivo BBB permeability data were harvested for 921 compounds from publicly available literature, non-proprietary drug approval packages, and University of Washington’s Drug Interaction Database. The cross-validation performance statistics for the BBB models ranged from 82 to 85% in sensitivity and 80–83% in negative predictivity. Additionally, the performance of newly developed models was assessed using an external validation set comprised of 83 chemicals. Overall, performance of individual models ranged from 70 to 75% in sensitivity, 70–72% in negative predictivity, and 78–86% in coverage. The predictive performance was further improved to 93% in coverage by combining predictions across the two software programs. These new models can be rapidly deployed to predict blood brain barrier permeability of pharmaceutical candidates and reduce the use of experimental animals.

59 BASIC BIOLOGICAL SCIENCES↗

A novel QSAR model of Salmonella mutagenicity and its application in the safety assessment of drug impurities

As indicated in ICH M7 draft guidance, in silico predictive tools including statistically-based QSARs and expert analysis may be used as a computational assessment for bacterial mutagenicity for the qualification of impurities in pharmaceuticals. To address this need, we developed and validated a QSAR model to predict Salmonella t. mutagenicity (Ames assay outcome) of pharmaceutical impurities using Prous Institute's Symmetry℠, a new in silico solution for drug discovery and toxicity screening, and the Mold2 molecular descriptor package (FDA/NCTR). Data was sourced from public benchmark databases with known Ames assay mutagenicity outcomes for 7300 chemicals (57% mutagens). Of these data, 90% was used to train the model and the remaining 10% was set aside as a holdout set for validation. The model's applicability to drug impurities was tested using a FDA/CDER database of 951 structures, of which 94% were found within the model's applicability domain. The predictive performance of the model is acceptable for supporting regulatory decision-making with 84 ± 1% sensitivity, 81 ± 1% specificity, 83 ± 1% concordance and 79 ± 1% negative predictivity based on internal cross-validation, while the holdout dataset yielded 83% sensitivity, 77% specificity, 80% concordance and 78% negative predictivity. Given the importance of having confidence in negative predictions, an additional external validation of the model was also carried out, using marketed drugs known to be Ames-negative, and obtained 98% coverage and 81% specificity. Additionally, Ames mutagenicity data from FDA/CFSAN was used to create another data set of 1535 chemicals for external validation of the model, yielding 98% coverage, 73% sensitivity, 86% specificity, 81% concordance and 84% negative predictivity. - Highlights: • A new in silico QSAR model to predict Ames mutagenicity is described. • The model is extensively validated with chemicals from the FDA and the public domain. • Validation tests show desirable high sensitivity and high negative predictivity. • The model predicted 14 reportedly difficult to predict drug impurities with accuracy. • The model is suitable to support risk evaluation of potentially mutagenic compounds.

60 APPLIED LIFE SCIENCES↗

Numerical solution of 2D wet steam flow with non-equilibrium condensation and real thermodynamics

An approach to modeling of wet steam flow with non-equilibrium condensation phenomenon is presented. The first part of our flow model is homogeneous Euler system of transport equations for mass, momentum and total energy of wet steam (mixture). The additional second part describes liquid phase via non-homogeneous system of transport equations for moments of droplets number distribution function and relies on corrected classical nucleation theory. Moment equations are closed by linearization of droplet growth rate model. All necessary relations for thermodynamic properties of steam are provided by IAPWS set of equations. However, properties of condensate are simply modeled by liquid saturation data. Two real equations of state are implemented. Recently developed CFD formulation for entropy (does not require iteration process) and so-called IAPWS special gas equation for Helmholtz energy (one iteration loop is necessary). Flow model is validated on converging-diverging supersonic nozzle with Barschdorff geometry. Simulations were performed by in-house CFD code based on finite volume method and stiff character of equations was solved by symmetrical time operator splitting. Achieved results satisfactorily agreed with experimental data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Measurement‐Model Fusion Approach for Improved Wet Deposition Maps and Trends

Abstract Air quality models provide spatial fields of wet deposition (WD) and dry deposition that explicitly account for the transport and transformation of emissions from thousands of sources. However, many sources of uncertainty in the air quality model including errors in emissions and meteorological inputs (particularly precipitation) and incomplete descriptions of the chemical and physical processes governing deposition can lead to bias and error in the simulation of WD. We present an approach to bias correct Community Multiscale Air Quality model output over the contiguous United States using observation‐based gridded precipitation data generated by the Parameter‐elevation Regressions on Independent Slopes Model and WD observations at the National Atmospheric Deposition Program National Trends Network sites. A cross‐validation analysis shows that the adjusted annual accumulated WD for NO 3 − , NH 4 + , and SO 4 2− from 2002 to 2012 has less bias and higher correlation with observed values than the base model output without adjustment. Temporal trends in observed WD are captured well by the adjusted model simulations across the entire contiguous United States. Consistent with previous trend analyses, WD NO 3 − and SO 4 2− are shown to decrease during this period in the eastern half of the United States, particularly in the Northeast, while remaining nearly constant in the West. Trends in WD of NH 4 + are more spatially and temporally heterogeneous, with some positive trends in the Great Plains and Central Valley of CA and slightly negative trends in the south.

54 ENVIRONMENTAL SCIENCES↗

Influence of sky radiance measurement errors on inversion-retrieved aerosol properties

Remote sensing of the atmospheric aerosol is a well-established technique that is currently used for routine monitoring of this atmospheric component, both from ground-based and satellite. The AERONET program, initiated in the 90's, is the most extended network and the data provided are currently used by a wide community of users for aerosol characterization, satellite and model validation and synergetic use with other instrumentation (lidar, in-situ, etc.). Aerosol properties are derived within the network from measurements made by ground-based Sun-sky scanning radiometers. Sky radiances are acquired in two geometries: almucantar and principal plane. Discrepancies in the products obtained following both geometries have been observed and the main aim of this work is to determine if they could be justified by measurement errors. Three systematic errors have been analyzed in order to quantify the effects on the inversion-derived aerosol properties: calibration, pointing accuracy and finite field of view. Simulations have shown that typical uncertainty in the analyzed quantities (5% in calibration, 0.2 Degree-Sign in pointing and 1.2 Degree-Sign field of view) yields to errors in the retrieved parameters that vary depending on the aerosol type and geometry. While calibration and pointing errors have relevant impact on the products, the finite field of view does not produce notable differences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

HDSense: An efficient method for ranking observable sensitivity

Identifying which observables most effectively constrain model parameters can be computationally prohibitive when considering full likelihoods of many correlated observables. This is especially important for, e.g., hadronization models, where high precision is required to interpret the results of collider experiments. We introduce the High-Dimensional Sensitivity (HDSense) score, a computationally efficient metric for ranking observable sets using only one-dimensional histograms. Derived by profiling over unknown correlations in the Fisher information framework, the score balances total information content against redundancy between observables. We apply HDSense to rank a set observables in terms of their constraining power with respect to five parameters of the Lund string model of hadronization implemented in Pythia using simulated leptonic collider events at the $Z$ pole. Validation against machine-learning--based full-likelihood approximations demonstrates that HDSense successfully identifies near-optimal observable subsets. The framework naturally handles data from multiple experiments with different acceptances and incorporates detector effects. While demonstrated on hadronization models, the methodology applies broadly to generic parameter estimation problems where correlations are unknown or difficult to model.

Assi, Benoît [Cincinnati U.] (ORCID:00000003092433↗

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗

SCC Aerosol Transport Model Summary Report

The purpose of the present work is to introduce a generic and reliable numerical model for prediction of aerosol transport, deposition, and plugging in leak paths similar to stress corrosion cracks while accounting for potential plugging formation. The model is dynamic (changing leak path geometry due to plugging) and it relies on the numerical solution of the aerosol transport equation in one dimension using finite differences. An extensive validation exercise (particle diameters: 0.01-10 μm and pressure difference up to 12 kPa) was conducted based on comparisons with experimental and theoretical data. The developed model is fairly general since it is based on a generic mechanistic description of the aerosol flow and particle deposition in the leak path, without requiring experimental fittings other than using the leak rate parameters commonly available in practical applications. Despite the approximate agreement of the model with the experimental data, the obtained predictions can be qualified as realistic. This is a major achievement of the model, which has been based on generic aerosol mechanics with no reference or adjustment to experimental data.

42 ENGINEERING↗