Search NASA⌕ Search

SEARCH · Search NASA

Results for “data base for model validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Fusion of Multiple Models for Improving Gross Primary Production Estimation With Eddy Covariance Data Based on Machine Learning

Abstract Terrestrial gross primary production (GPP) represents the magnitude of CO 2 uptake through vegetation photosynthesis, and is a key variable for carbon cycles between the biosphere and atmosphere. Light use efficiency (LUE) models have been widely used to estimate GPP for its physiological mechanisms and availability of data acquisition and implementation, yet each individual GPP model has exhibited large uncertainties due to input errors and model structure, and further studies of systematic validation, comparison, and fusion of those models with eddy covariance (EC) site data across diverse ecosystem types are still needed in order to further improve GPP estimation. We here compared and fused five GPP models (VPM, EC‐LUE, GOL‐PEM, CHJ, and C‐Fix) across eight ecosystems based on FLUXNET2015 data set using the ensemble methods of Bayesian Model Averaging (BMA), Support Vector Machine (SVM), and Random Forest (RF) separately. Our results showed that for individual models, EC‐LUE gave a better performance to capture interannual variability of GPP than other models, followed by VPM and GLO‐PEM, while CHJ and C‐Fix were more limited in their estimation performance. We found RF and SVM were superior to BMA on merging individual models at various plant functional types (PFTs) and at the scale of individual sites. On the basis of individual models, the fusion methods of BMA, SVM, and RF were examined by a five‐fold cross validation for each ecosystem type, and each method successfully improved the average accuracy of estimation by 8%, 18%, and 19%, respectively.

Environmental Sciences & Ecology↗

An Approach for Validating Actinide and Fission Product Burnup Credit Criticality Safety Analyses-Isotopic Composition Predictions

The expanded use of burnup credit in the United States (U.S.) for storage and transport casks, particularly in the acceptance of credit for fission products, has been constrained by the availability of experimental fission product data to support code validation. The U.S. Nuclear Regulatory Commission (NRC) staff has noted that the rationale for restricting the Interim Staff Guidance on burnup credit for storage and transportation casks (ISG-8) to actinide-only is based largely on the lack of clear, definitive experiments that can be used to estimate the bias and uncertainty for computational analyses associated with using burnup credit. To address the issues of burnup credit criticality validation, the NRC initiated a project with the Oak Ridge National Laboratory to (1) develop and establish a technically sound validation approach for commercial spent nuclear fuel (SNF) criticality safety evaluations based on best-available data and methods and (2) apply the approach for representative SNF storage and transport configurations/conditions to demonstrate its usage and applicability, as well as to provide reference bias results. The purpose of this paper is to describe the isotopic composition (depletion) validation approach and resulting observations and recommendations. Validation of the criticality calculations is addressed in a companion paper at this conference. For isotopic composition validation, the approach is to determine burnup-dependent bias and uncertainty in the effective neutron multiplication factor (keff) due to bias and uncertainty in isotopic predictions, via comparisons of isotopic composition predictions (calculated) and measured isotopic compositions from destructive radiochemical assay utilizing as much assay data as is available, and a best-estimate Monte Carlo based method. This paper (1) provides a detailed description of the burnup credit isotopic validation approach and its technical bases, (2) describes the application of the approach for representative pressurized water reactor and boiling water reactor safety analysis models to demonstrate its usage and applicability, (3) provides reference bias and uncertainty results based on a quality-assurance-controlled prerelease version of the Scale 6.1 code package and the ENDF/B-VII nuclear cross section data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Laser Hardening Prediction Tool Based On a Solid State Transformations Numerical Model

This paper presents a tool to predict hardening layer in selective laser hardening processes where laser beam heats the part locally while the bulk acts as a heat sink.The tool to predict accurately the temperature field in the workpiece is a numerical model that combines a three dimensional transient numerical solution for heating where is possible to introduce different laser sources. The thermal field was modeled using a kinetic model based on Johnson-Mehl-Avrami equation. Considering this equation, an experimental adjustment of transformation parameters was carried out to get the heating transformation diagrams (CHT). With the temperature field and CHT diagrams the model predicts the percentage of base material converted into austenite. These two parameters are used as first step to estimate the depth of hardened layer in the part.The model has been adjusted and validated with experimental data for DIN 1.2379, cold work tool steel typically used in mold and die making industry. This steel presents solid state diffusive transformations at relative low temperature. These transformations must be considered in order to get good accuracy of temperature field prediction during heating phase. For model validation, surface temperature measured by pyrometry, thermal field as well as the hardened layer obtained from metallographic study, were compared with the model data showing a good adjustment.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Guide to Using the WIND Toolkit Validation Code

In response to the U.S. Department of Energy's goal of using 20% wind energy by 2030, the Wind Integration National Dataset (WIND) Toolkit was created to provide information on wind speed, wind direction, temperature, surface air pressure, and air density on more than 126,000 locations across the United States from 2007 to 2013. The numerical weather prediction model output, gridded at 2-km and at a 5-minute resolution, was further converted to detail the wind power production time series of existing and potential wind facility sites. For users of the dataset it is important that the information presented in the WIND Toolkit is accurate and that errors are known, as then corrective steps can be taken. Therefore, we provide validation code written in R that will be made public to provide users with tools to validate data of their own locations. Validation is based on statistical analyses of wind speed, using error metrics such as bias, root-mean-square error, centered root-mean-square error, mean absolute error, and percent error. Plots of diurnal cycles, annual cycles, wind roses, histograms of wind speed, and quantile-quantile plots are created to visualize how well observational data compares to model data. Ideally, validation will confirm beneficial locations to utilize wind energy and encourage regional wind integration studies using the WIND Toolkit.

17 WIND ENERGY↗

Short-Term Global Horizontal Irradiance Forecasting Based on Sky Imaging and Pattern Recognition

Accurate short-term forecasting is crucial for solar integration in the power grid. In this paper, a classification forecasting framework based on pattern recognition is developed for 1-hour-ahead global horizontal irradiance (GHI) forecasting. Three sets of models in the forecasting framework are trained by the data partitioned from the preprocessing analysis. The first two sets of models forecast GHI for the first four daylight hours of each day. Then the GHI values in the remaining hours are forecasted by an optimal machine learning model determined based on a weather pattern classification model in the third model set. The weather pattern is determined by a support vector machine (SVM) classifier. The developed framework is validated by the GHI and sky imaging data from the National Renewable Energy Laboratory (NREL). Results show that the developed short-term forecasting framework outperforms the persistence benchmark by 16% in terms of the normalized mean absolute error and 25% in terms of the normalized root mean square error.

14 SOLAR ENERGY↗

Development of QSAR models to predict blood-brain barrier permeability

Assessing drug permeability across the blood-brain barrier (BBB) is important when evaluating the abuse potential of new pharmaceuticals as well as developing novel therapeutics that target central nervous system disorders. One of the gold-standard in vivo methods for determining BBB permeability is rodent log BB; however, like most in vivo methods, it is time-consuming and expensive. In the present study, two statistical-based quantitative structure-activity relationship (QSAR) models were developed to predict BBB permeability of drugs based on their chemical structure. The in vivo BBB permeability data were harvested for 921 compounds from publicly available literature, non-proprietary drug approval packages, and University of Washington’s Drug Interaction Database. The cross-validation performance statistics for the BBB models ranged from 82 to 85% in sensitivity and 80–83% in negative predictivity. Additionally, the performance of newly developed models was assessed using an external validation set comprised of 83 chemicals. Overall, performance of individual models ranged from 70 to 75% in sensitivity, 70–72% in negative predictivity, and 78–86% in coverage. The predictive performance was further improved to 93% in coverage by combining predictions across the two software programs. These new models can be rapidly deployed to predict blood brain barrier permeability of pharmaceutical candidates and reduce the use of experimental animals.

59 BASIC BIOLOGICAL SCIENCES↗

Integrated Medical Model Verification, Validation, and Credibility

The Integrated Medical Model (IMM) was designed to forecast relative changes for a specified set of crew health and mission success risk metrics by using a probabilistic (stochastic process) model based on historical data, cohort data, and subject matter expert opinion. A probabilistic approach is taken since exact (deterministic) results would not appropriately reflect the uncertainty in the IMM inputs. Once the IMM was conceptualized, a plan was needed to rigorously assess input information, framework and code, and output results of the IMM, and ensure that end user requests and requirements were considered during all stages of model development and implementation. METHODS: In 2008, the IMM team developed a comprehensive verification and validation (VV) plan, which specified internal and external review criteria encompassing 1) verification of data and IMM structure to ensure proper implementation of the IMM, 2) several validation techniques to confirm that the simulation capability of the IMM appropriately represents occurrences and consequences of medical conditions during space missions, and 3) credibility processes to develop user confidence in the information derived from the IMM. When the NASA-STD-7009 (7009) was published, the IMM team updated their verification, validation, and credibility (VVC) project plan to meet 7009 requirements and include 7009 tools in reporting VVC status of the IMM. RESULTS: IMM VVC updates are compiled recurrently and include 7009 Compliance and Credibility matrices, IMM VV Plan status, and a synopsis of any changes or updates to the IMM during the reporting period. Reporting tools have evolved over the lifetime of the IMM project to better communicate VVC status. This has included refining original 7009 methodology with augmentation from the NASA-STD-7009 Guidance Document. End user requests and requirements are being satisfied as evidenced by ISS Program acceptance of IMM risk forecasts, transition to an operational model and simulation tool, and completion of service requests from a broad end user consortium including Operations, Science and Technology Planning, and Exploration Planning. CONCLUSIONS: The VVC approach established by the IMM project of combining the IMM VV Plan with 7009 requirements is comprehensive and includes the involvement of end users at every stage in IMM evolution. Methods and techniques used to quantify the VVC status of the IMM have not only received approval from the local NASA community but have also garnered recognition by other federal agencies seeking to develop similar guidelines in the medical modeling community.

Validation↗

Testing the Paradigm that Ultra-Luminous X-Ray Sources as a Class Represent Accreting Intermediate

To test the idea that ultraluminous X-ray sources (ULXs) in external galaxies represent a class of accreting Intermediate-Mass Black Holes (IMBHs), we have undertaken a program to identify ULXs and a lower luminosity X-ray comparison sample with the highest quality data in the Chandra archive. We establish a general property of ULXs that the most X-ray luminous objects possess the fattest X-ray spectra (in the Chandra band pass). No prior sample studies have established the general hardening of ULX spectra with luminosity. This hardening occurs at the highest luminosities (absorbed luminosity > or equals 5x10(exp 39) ergs/s) and is in line with recent models arguing that ULXs are actually stellar-mass black holes. From spectral modeling, we show that the evidence originally taken to mean that ULXs are IMBHs - i.e., the "simple IMBH model" - is nowhere near as compelling when a large sample of ULXs is looked at properly. During the last couple of years, XMM-Newton spectroscopy of ULXs has to some large extent begun to negate the simple IMBH model based on fewer objects. We confirm and expand these results, which validates the XMM-Newton work in a broader sense with independent X-ray data. We find (1) that cool disk components are present with roughly equal probability and total flux fraction for any given ULX, regardless of luminosity, and (2) that cool disk components extend below the standard ULX luminosity cutoff of 10(exp 39) ergs/s, down to our sample limit of 10(exp 38:3) ergs/s. The fact that cool disk components are not correlated with luminosity damages the argument that cool disks indicate IMBHs in ULXs, for which a strong statistical support was never made.

Berghea, C. T.↗

Numerical solution of 2D wet steam flow with non-equilibrium condensation and real thermodynamics

An approach to modeling of wet steam flow with non-equilibrium condensation phenomenon is presented. The first part of our flow model is homogeneous Euler system of transport equations for mass, momentum and total energy of wet steam (mixture). The additional second part describes liquid phase via non-homogeneous system of transport equations for moments of droplets number distribution function and relies on corrected classical nucleation theory. Moment equations are closed by linearization of droplet growth rate model. All necessary relations for thermodynamic properties of steam are provided by IAPWS set of equations. However, properties of condensate are simply modeled by liquid saturation data. Two real equations of state are implemented. Recently developed CFD formulation for entropy (does not require iteration process) and so-called IAPWS special gas equation for Helmholtz energy (one iteration loop is necessary). Flow model is validated on converging-diverging supersonic nozzle with Barschdorff geometry. Simulations were performed by in-house CFD code based on finite volume method and stiff character of equations was solved by symmetrical time operator splitting. Achieved results satisfactorily agreed with experimental data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Quantifying Boreal Forest Structure and Composition Using UAV Structure from Motion

The vast extent and inaccessibility of boreal forest ecosystems are barriers to routine monitoring of forest structure and composition. In this research, we bridge the scale gap between intensive but sparse plot measurements and extensive remote sensing studies by collecting forest inventory variables at the plot scale using an unmanned aerial vehicle (UAV) and a structure from motion (SfM) approach. At 20 Forest Inventory and Analysis (FIA) subplots in interior Alaska, we acquired overlapping imagery and generated dense, 3D, RGB (red, green, blue) point clouds. We used these data to model forest type at the individual crown scale as well as subplot-scale tree density (TD), basal area (BA), and aboveground biomass (AGB). We achieved 85% cross-validation accuracy for five species at the crown level. Classification accuracy was maximized using three variables representing crown height, form, and color. Consistent with previous UAV-based studies, SfM point cloud data generated robust models of TD (r(sup 2) = 0.91), BA (r(sup 2) = 0.79), and AGB (r(sup 2) = 0.92), using a mix of plot- and crown-scale information. Precise estimation of TD required either segment counts or species information to differentiate black spruce from mixed white spruce plots. The accuracy of species-specific estimates of TD, BA, and AGB at the plot scale was somewhat variable, ranging from accurate estimates of black spruce TD (+/−1%) and aspen BA (−2%) to misallocation of aspen AGB (+118%) and white spruce AGB (−50%). These results convey the potential utility of SfM data for forest type discrimination in FIA plots and the remaining challenges to develop classification approaches for species-specific estimates at the plot scale that are more robust to segmentation error.

aboveground biomass↗

Bringing Back the Social Affordances of the Paper Memo to Aerospace Systems Engineering Work

Model-based systems engineering (MBSE) is a relatively new field that brings together the interdisciplinary study of technological components of a project (systems engineering) with a model-based ontology to express the hierarchical and behavioral relationships between the components (computational modeling). Despite the compelling promises of the benefits of MBSE, such as improved communication and productivity due to an underlying language and data model, we observed hesitation to its adoption at the NASA Jet Propulsion Laboratory. To investigate, we conducted a six-month ethnographic field investigation and needs validation with 19 systems engineers. This paper contributes our observations of a generational shift in one of JPL's core technologies. We report on a cultural misunderstanding between communities of practice that bolsters the existing technology drag. Given the high cost of failure, we springboard our observations into a design hypothesis - an intervention that blends the social affordances of the narrative-based work flow with the rich technological advantages of explicit data references and relationships of the model-based approach. We provide a design rationale, and the results of our evaluation.

design↗

A Measurement‐Model Fusion Approach for Improved Wet Deposition Maps and Trends

Abstract Air quality models provide spatial fields of wet deposition (WD) and dry deposition that explicitly account for the transport and transformation of emissions from thousands of sources. However, many sources of uncertainty in the air quality model including errors in emissions and meteorological inputs (particularly precipitation) and incomplete descriptions of the chemical and physical processes governing deposition can lead to bias and error in the simulation of WD. We present an approach to bias correct Community Multiscale Air Quality model output over the contiguous United States using observation‐based gridded precipitation data generated by the Parameter‐elevation Regressions on Independent Slopes Model and WD observations at the National Atmospheric Deposition Program National Trends Network sites. A cross‐validation analysis shows that the adjusted annual accumulated WD for NO 3 − , NH 4 + , and SO 4 2− from 2002 to 2012 has less bias and higher correlation with observed values than the base model output without adjustment. Temporal trends in observed WD are captured well by the adjusted model simulations across the entire contiguous United States. Consistent with previous trend analyses, WD NO 3 − and SO 4 2− are shown to decrease during this period in the eastern half of the United States, particularly in the Northeast, while remaining nearly constant in the West. Trends in WD of NH 4 + are more spatially and temporally heterogeneous, with some positive trends in the Great Plains and Central Valley of CA and slightly negative trends in the south.

54 ENVIRONMENTAL SCIENCES↗

Influence of sky radiance measurement errors on inversion-retrieved aerosol properties

Remote sensing of the atmospheric aerosol is a well-established technique that is currently used for routine monitoring of this atmospheric component, both from ground-based and satellite. The AERONET program, initiated in the 90's, is the most extended network and the data provided are currently used by a wide community of users for aerosol characterization, satellite and model validation and synergetic use with other instrumentation (lidar, in-situ, etc.). Aerosol properties are derived within the network from measurements made by ground-based Sun-sky scanning radiometers. Sky radiances are acquired in two geometries: almucantar and principal plane. Discrepancies in the products obtained following both geometries have been observed and the main aim of this work is to determine if they could be justified by measurement errors. Three systematic errors have been analyzed in order to quantify the effects on the inversion-derived aerosol properties: calibration, pointing accuracy and finite field of view. Simulations have shown that typical uncertainty in the analyzed quantities (5% in calibration, 0.2 Degree-Sign in pointing and 1.2 Degree-Sign field of view) yields to errors in the retrieved parameters that vary depending on the aerosol type and geometry. While calibration and pointing errors have relevant impact on the products, the finite field of view does not produce notable differences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

HDSense: An efficient method for ranking observable sensitivity

Identifying which observables most effectively constrain model parameters can be computationally prohibitive when considering full likelihoods of many correlated observables. This is especially important for, e.g., hadronization models, where high precision is required to interpret the results of collider experiments. We introduce the High-Dimensional Sensitivity (HDSense) score, a computationally efficient metric for ranking observable sets using only one-dimensional histograms. Derived by profiling over unknown correlations in the Fisher information framework, the score balances total information content against redundancy between observables. We apply HDSense to rank a set observables in terms of their constraining power with respect to five parameters of the Lund string model of hadronization implemented in Pythia using simulated leptonic collider events at the $Z$ pole. Validation against machine-learning--based full-likelihood approximations demonstrates that HDSense successfully identifies near-optimal observable subsets. The framework naturally handles data from multiple experiments with different acceptances and incorporates detector effects. While demonstrated on hadronization models, the methodology applies broadly to generic parameter estimation problems where correlations are unknown or difficult to model.

Assi, Benoît [Cincinnati U.] (ORCID:00000003092433↗

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗

SCC Aerosol Transport Model Summary Report

The purpose of the present work is to introduce a generic and reliable numerical model for prediction of aerosol transport, deposition, and plugging in leak paths similar to stress corrosion cracks while accounting for potential plugging formation. The model is dynamic (changing leak path geometry due to plugging) and it relies on the numerical solution of the aerosol transport equation in one dimension using finite differences. An extensive validation exercise (particle diameters: 0.01-10 μm and pressure difference up to 12 kPa) was conducted based on comparisons with experimental and theoretical data. The developed model is fairly general since it is based on a generic mechanistic description of the aerosol flow and particle deposition in the leak path, without requiring experimental fittings other than using the leak rate parameters commonly available in practical applications. Despite the approximate agreement of the model with the experimental data, the obtained predictions can be qualified as realistic. This is a major achievement of the model, which has been based on generic aerosol mechanics with no reference or adjustment to experimental data.

42 ENGINEERING↗

NASA-Langley Web-Based Operational Real-time Cloud Retrieval Products from Geostationary Satellites

At NASA Langley Research Center (LaRC), radiances from multiple satellites are analyzed in near real-time to produce cloud products over many regions on the globe. These data are valuable for many applications such as diagnosing aircraft icing conditions and model validation and assimilation. This paper presents an overview of the multiple products available, summarizes the content of the online database, and details web-based satellite browsers and tools to access satellite imagery and products.

Palikonda, Rabindra↗