Search NASA⌕ Search

SEARCH · Search NASA

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Surrogate model evaluation and building energy benchmarking for commercial buildings

Building energy consumption benchmarking involves challenges associated with various energy patterns for different building types; heating, ventilating, and air-conditioning (HVAC) system types; and climates. Given significant variation in energy use patterns, accurate prediction of long-term energy use using surrogate models remains challenging. Multiple linear regression (MLR) is commonly used for building energy benchmarking because of its simple structure; however, it lacks accuracy compared to other black-box models. Although many studies have compared surrogate models and offer guidance on model selection based on metrics, they do not provide detailed analysis on improving the surrogate model accuracy. In this paper, we implement a surrogate model using polynomial ridge regression (i.e., MLR with interaction terms combined with ridge regularization) for small office and retail strip mall buildings across six HVAC system types and all climate zones, for electricity and natural gas in baseline and proposed scenarios. A simulation workflow is developed using OpenStudio TM /EnergyPlus TM to generate simulation data using measures over a wide range of efficiency inputs. Enhancements based on statistical insights are used for improving the model accuracy using filters, input transformations, and change points. Surrogate models achieved average coefficient of variation of the root mean squared error (CVRMSE) values of 2.17, 1.06, 2.05, and 3.26 for proposed electricity, proposed natural gas, baseline electricity, and baseline natural gas, respectively, with enhancements reducing CVRMSE by an average of 14.9% across all combinations. We provide model interpretation via Shapley additive explanations to determine which input variables most influence energy consumption and provide supportive arguments for enhancements.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1993

Turbulence model evaluation for the prediction of flows over a supercritical airfoil with deflected aileron at high Reynolds number

Navier-Stokes solutions about a supercritical airfoil with aileron deflection have been computed using the CFL3D code coupled with the Baldwin-Lomax, Johnson-King, Baldwin-Barth, and Spalart-Allmaras turbulence models. Computations were made at a Mach number of 0.716 and chord Reynolds numbers of 5, 15, and 25 million. The airfoil was analyzed with both 0 deg and 2 deg (TED) aileron deflections. Comparisons over a range of angles-of-attack showed that solutions obtained using the Baldwin-Barth turbulence model presented the best agreement with experimental pressures and sectional lift coefficients. However, Reynolds number trends in sectional lift coefficient and in aileron effectiveness were not predicted consistently.

Londenberg, W. K.↗
DOE OSTI2018

Model evaluation guidelines for geomagnetic index predictions

Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near–Earth space into a single parameter. Most of the best–known indices are calculated from ground–based magnetometer data sets, such as Dst, SYM–H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root mean square error (RMSE) and mean absolute error (MAE) that are applied to a time–series comparison of model output and observations; and (2) event detection performance metrics such as Heidke Skill Score and probability of detection (POD) that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. Furthermore, a few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2014

Model Evaluation of Methods for Estimating Surface Emissions and Chemical Lifetimes from Satellite Data

Column densities from satellite retrievals can provide valuable information for estimating emissions and chemical lifetimes objectively across the globe. To better understand the uncertainties associated with these estimates, we test four methods using simulated column densities from a point source: a box model approach, a 2D Gaussian fit, an Inverse Radius fit and an Exponentially-Modified Gaussian fit. The model results were simulated using the WRF and CAMx models for the year 2005, for a single point source outside Atlanta in Georgia, USA with specified emissions and three chemical scenarios: no chemical reactions, 12 h chemical lifetime and 1 h chemical lifetime. No other sources were included in the simulations. We find that the box model provides reliable estimates irrespective of plume speed and plume direction, if the plume speed and the chemical lifetime are known accurately. The 2D Gaussian fit was found to be sensitive to plume speed and direction, and requires omnidirectional dispersion in order to have a decent fit. However, the 2D Gaussian fit is only an approximate fit to the data, and the discrepancies mean that the results are dependent on the geographical domain used for the optimization. An Inverse Radius fit is introduced to correct this issue, which is found to provide improved emissions and lifetime estimates. The Exponentially-Modified Gaussian fit also gave improved estimates. It is however dependent on accurate plume rotation such that reported chemical lifetimes with this method could be significantly underestimated.

Emission inventory↗
NASA NTRS2010

Hybrid FEA/SEA Assessment for an Orthogrid Cylindrical Panel Section and Periodic Subsystem Modeling Evaluation

In the lower frequency range, where particular boundary conditions can make a significant difference to panel response characteristics Statistical Energy Analysis (SEA) has never been the analytical tool of choice. In addition to boundary condition effects, SEA is not well suited in frequency bands where no modes or less than a few modes exist. The advent of the Hybrid Module has enabled integration of Finite Element Analysis to expand and enhance the capability for response calculations within VA One into the lower frequency range. Exploration of several additional modeling approaches was completed for the cylindrical orthogrid panel test article that was examined in Reference 1. Comparison of the new analytical response predictions with the measured response data from ground test and the pure SEA results from the reference will be presented. One approach that is considered promising is the periodic subsystem capability. Initially, a detailed FEM of just one region of the test article is defined. After evaluating this small region using symmetric boundary conditions, the FEM may be expanded to determine the properties of the entire system using similar connected regions that map over the entire test article. Another approach is the direct use of a very detailed finite element model of the entire panel, explicitly modeling pocket and rib details of the structure. A third approach is to approximate localized structure geometry details with a smeared property generalization using a PCOMP (NASTRAN card used to define layered composite structures) to define skin layer and ribbed layer for the orthogrid panel. The authors expect to demonstrate that the integrated Hybrid/FEM approach increases confidence in response prediction in the lower frequency range (for example from 20-300 Hz for the test article under consideration). In addition the strength and weakness of each additional approach will be highlighted and compared to those reported with those reported in an earlier paper

Smith, Andrew M.↗
NASA NTRS1980

Planetary boundary-layer wind model evaluation at a mid-Atlantic coastal site

Detailed measurements of the mean flow and turbulence were made with the use of a micrometeorological facility consisting of an instrumented 76-m tall tower located within a 100-m distance from the Atlantic Ocean at Wallops Island, Virginia. Under moderately strong wind conditions, the popular neutral boundary layer flow model fails to provide an adequate description of the actual flow. In addition to detailed flow information for all wind directions, averages of the important flow parameters used for design such as vertical distribution of mean velocity, turbulence intensities and turbulence integral scales were presented for wind direction sectors with near uniform upstream terrain. Power spectra of the three velocity components for the prevailing northwesterly and southerly winds are discussed.

Tieleman, H. W.↗
DOE OSTI2025

Fire and Thermal Experiments in Support of the Model Evaluation Protocol for LNG Facility Fires

The motivation for the experiments reported here pertains to the siting of Liquefied Natural Gas (LNG) facilities which requires assessing the potential adverse radiant thermal impacts of accidental fires on the public. The objective is to obtain data on jet fires, pool fires, fireballs, and concrete walls that could serve as thermal barriers for model validation. The fuels tested include ethane, ethylene, propane, and isopentane.

42 ENGINEERING↗
NASA NTRS1997

Model Evaluation and Sensitivity Studies for Determining Aircraft Effects on the Global Atmosphere

This project, started in July 1995 and ending in July 1996, related: to evaluation of the possible importance of soot and sulfur dioxide emissions from subsonic and supersonic aircraft; to research contributions and special responsibilities for NASA AEAP assessments of subsonic aircraft and High Speed Civil Transport aircraft; and to science team responsibilities supporting the development of the three-dimensional atmospheric chemistry model of the Global Modeling Initiative.

Wuebbles, Donald J.↗
DOE OSTI2017

Model Evaluation of New Techniques for Maintaining High-NO Conditions in Oxidation Flow Reactors for the Study of OH-Initiated Atmospheric Chemistry

Oxidation flow reactors (OFRs) efficiently produce OH radicals using low-pressure Hg-lamp emissions at λ=254 nm (OFR254) or both λ=185 and 254 nm (OFR185). OFRs under most conditions are limited to studying low-NO chemistry (where RO2+HO2 dominates RO2 fate), even though substantial amounts of initial NO may be injected. This is due to very fast NO oxidation by high concentrations of OH, HO2, and O3. Here, we model new techniques for maintaining high-NO conditions in OFRs, i.e., continuous NO addition along the length of the reactor in OFR185 (OFR185-cNO), recently proposed injection of N2O at the entrance of the reactor in OFR254 (OFR254-iN2O), and an extension of that idea to OFR185 (OFR185-iN2O). For these techniques, we evaluate (1) fraction of conditions dominated by RO2+NO while avoiding significant non-tropospheric photolysis and (2) fraction of conditions where reactions of precursors with OH dominate over unwanted reactions with NO3. OFR185-iN2O is the most practical for general high-NO experiments since it represents the best compromise between experimental complexity and performance upon proper usage. Short lamp distances are recommended for OFR185-iN2O to ensure a relatively uniform radiation field. OFR185-iN2O with low O2 or using Hg lamps with higher 185-nm-to-254-nm ratio can improve performance. OFR185-iN2O experiments should generally be conducted at higher relative humidity, higher UV, lower concentration of non-NOy external OH reactants, and percent-level N2O. OFR185-cNO and OFR185-iN2O at optimal NO precursor injection rate (~2 ppb/s) or concentration (~3%) would have satisfactory performance in typical field studies where ambient air is oxidized. Finally, we provide exposure estimation equations to aid experimental planning. This work enables improved high-NO OFR experimental design and interpretation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Aircraft Observations of Turbulence in Cloudy and Cloud-Free Boundary Layers Over the Western North Atlantic Ocean From ACTIVATE and Implications for the Earth System Model Evaluation and Development

This study examines boundary layer turbulence derived from high temporal resolution meteorological measurements from 40 research flights over the western North Atlantic Ocean during the 2020 deployments of ACTIVATE. Frequency distributions of various turbulent quantities reveal stronger turbulence during the winter deployment than in summer and for cloud‐topped than in cloud‐free boundary layers during the summer deployment. Maximum turbulence kinetic energy (TKE) is most often within cloud from observations in winter and summer, whereas it is mostly below cloud in both seasons by a global model turbulence parameterization. Further, bivariate frequency distributions are consistent with the bivariate Gaussian probability distribution functions assumed for the closure of higher‐order turbulence/shallow convection parameterizations used by some global models. Turbulence simulated by the Community Atmosphere Model version 6 and the Energy Exascale Earth System Model Atmosphere Model version 2 using such parameterizations is not as strong as observed, with more TKE going into vertical wind perturbations rather than into zonal wind perturbations as observed, suggesting that the treatment of turbulence in Earth system models still needs to be further improved.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Basement Fill Model Evaluation of Maximum Radionuclide Concentrations for Initial Suite of Radionuclides. Zion Station Restoration Project

ZionSolutions is in the process of decommissioning the Zion Nuclear Power Plant in order to establish a new water treatment plant. There is some residual radioactive particles from the plant which need to be brought down to levels so an individual who receives water from the new treatment plant does not receive a radioactive dose in excess of 25 mrem/y⁻¹ as specified in 10 CFR 20 Subpart E. The objectives of this report are: (a) To present a simplified conceptual model for release from the buildings with residual subsurface structures that can be used to provide an upper bound on radionuclide concentrations in the fill material and the water in the interstitial spaces of the fill. (b) Provide maximum water concentrations and the corresponding amount of mass sorbed to the solid fill material that could occur in each building for use by ZSRP in selecting ROCs for detailed dose assessment calculations.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2007

Evaluating Model Performance of an Ensemble-based Chemical Data Assimilation System During INTEX-B Field Mission

We present a global chemical data assimilation system using a global atmosphere model, the Community Atmosphere Model (CAM3) with simplified chemistry and the Data Assimilation Research Testbed (DART) assimilation package. DART is a community software facility for assimilation studies using the ensemble Kalman filter approach. Here, we apply the assimilation system to constrain global tropospheric carbon monoxide (CO) by assimilating meteorological observations of temperature and horizontal wind velocity and satellite CO retrievals from the Measurement of Pollution in the Troposphere (MOPITT) satellite instrument. We verify the system performance using independent CO observations taken on board the NSFINCAR C-130 and NASA DC-8 aircrafts during the April 2006 part of the Intercontinental Chemical Transport Experiment (INTEX-B). Our evaluations show that MOPITT data assimilation provides significant improvements in terms of capturing the observed CO variability relative to no MOPITT assimilation (i.e. the correlation improves from 0.62 to 0.71, significant at 99% confidence). The assimilation provides evidence of median CO loading of about 150 ppbv at 700 hPa over the NE Pacific during April 2006. This is marginally higher than the modeled CO with no MOPITT assimilation (-140 ppbv). Our ensemble-based estimates of model uncertainty also show model overprediction over the source region (i.e. China) and underprediction over the NE Pacific, suggesting model errors that cannot be readily explained by emissions alone. These results have important implications for improving regional chemical forecasts and for inverse modeling of CO sources and further demonstrate the utility of the assimilation system in comparing non-coincident measurements, e.g. comparing satellite retrievals of CO with in-situ aircraft measurements. The work described above also brought to light several short-comings of the data assimilation approach for CO profiles. Because of the limited vertical resolution of the measurement, the retrievals at different altitudes are correlated which can lead to problems with numerical error and overall efficiency. This has resulted in a manuscript that is about to be submitted to JGR:

Arellano, A. F., Jr.↗
DOE OSTI2020

Thermomechanical conversion in metals: dislocation plasticity model evaluation of the Taylor-Quinney coefficient

Using a partitioned-energy thermodynamic framework which assigns energy to that of atomic configurational stored energy of cold work and kinetic-vibrational, in this study we derive an important constraint on the Taylor-Quinney coefficient, which quantifies the fraction of plastic work that is converted into heat during plastic deformation. Associated with the two energy contributions are two separate temperatures – the ordinary temperature for the thermal energy and the effective temperature for the configurational energy. We show that the Taylor-Quinney coefficient is a function of the thermodynamically defined effective temperature that measures the atomic configurational disorder in the material. Finite-element analysis of recently published experiments on the aluminum alloy 6016-T4 [1], using the thermodynamic dislocation theory (TDT), shows good agreement between theory and experiment for both stress-strain behavior and temporal evolution of the temperature. The simulations include both conductive and convective thermal energy loss during the experiments, and significant thermal gradients exist within the simulation results. Computed values of the differential Taylor-Quinney coefficient are also presented and suggest a value which differs between materials and increases with increasing strain.

36 MATERIALS SCIENCE↗