Search NASA⌕ Search

SEARCH · Search NASA

Results for “simulation evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Evaluation of simulations of near-surface variables using the regional climate model CCLM for the MOSAiC winter period

The ship-based experiment MOSAiC 2019/2020 was carried out during a full year in the Arctic and yielded an excellent data set to test the parameterizations of ocean/sea-ice/atmosphere interaction processes in regional climate models (RCMs). In the present paper, near-surface data during MOSAiC are used for the verification of the RCM COnsortium for Small-scale MOdel–Climate Limited area Mode (COSMO-CLM or CCLM). CCLM is used in a forecast mode (nested in ERA5) for the whole Arctic with 15 km resolution and is run with different configurations of sea ice data. These include the standard sea ice concentration taken from passive microwave data with around 6 km resolution, sea ice concentration from Moderate Resolution Imaging Spectroradiometer (MODIS) thermal infrared data and MODIS sea ice lead fraction data for the winter period. CCLM simulations show a good agreement with the measurements. Relatively large negative biases for temperature occur for November and December, which are likely associated with a too large ice thickness used by CCLM. The consideration of sea ice leads in the sub-grid parameterization in CCLM yields improved results for the near-surface temperature. ERA5 data show a large warm bias of about 2.5°C and an underestimation of the temperature variability.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Evaluation of Simulated Marine Aerosol Production Using the WaveWatchIII Prognostic Wave Model Coupled to the Community Atmosphere Model within the Community Earth System Model

Primary marine aerosol (PMA) is emitted into the atmosphere via breaking wind waves on the ocean surface. Most parameterizations of PMA emissions use 10-meter wind speed as a proxy for wave action. This investigation coupled the 3 rd generation prognostic WAVEWATCH-III wind-wave model within a coupled Earth system model (ESM) to drive PMA production using wave energy dissipation rate – analogous to whitecapping – in place of 10-meter wind speed. The wind speed parameterization did not capture basin-scale variability in relations between wind and wave fields. Overall, the wave parameterization did not improve comparison between simulated versus measured AOD or Na + , thus highlighting large remaining uncertainties in model physics. Results confirm the efficacy of prognostic wind-wave models for air-sea exchange studies coupled with laboratory- and field-based characterizations of the primary physical drivers of PMA production. No discernible correlations were evident between simulated PMA fields and observed chlorophyll or sea surface temperature.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evaluation of WRF simulation of deep convection in the US Southern Great Plains

The Southern Great Plains (SGP) exhibits a relatively high frequency of periods with extremely high rainfall rates (RR) and hail. Here, seven months of 2017 are simulated using the Weather Research and Forecasting (WRF) model applied at convection permitting resolution with the Mibrandt-Yau microphysics scheme. Simulation fidelity is evaluated, particularly during intense convective events, using data from ASOS stations, dual-polarization RADAR, gridded data sets and observations at the DoE Atmospheric Radiation Measurement site. The spatial gradients and temporal variability of precipitation and the cumulative density functions for both RR and wind speeds exhibit fidelity. Odds ratios >1 indicate WRF is also skillful in simulating high composite reflectivity (cREF, used as a measure of widespread convection) and RR > 5 mmhr –1 over the domain. Detailed analyses of the ten days with highest spatial coverage of cREF >30 dBZ show spatially similar reflectivity fields and high RR in both RADAR data and WRF simulations. However, during periods of high reflectivity, WRF exhibits a positive bias in terms of very high RR (> 25 mmhr –1 ) and hail occurrence, and during the summer and transition months, maximum hail size is underestimated. For some renewable energy applications fidelity is required with respect to the joint probabilities of wind speed and RR and/or hail. While partial fidelity is achieved for the marginal probabilities, performance during events of critical importance to these energy applications is currently not sufficient. Further research into optimal WRF configurations in support of potential damage quantification for these applications is warranted.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Evaluating the streamflow simulation capability of PERSIANN-CDR daily rainfall products in two river basins on the Tibetan Plateau

On the Tibetan Plateau, the limited ground-based rainfall information owing to a harsh environment has brought great challenges to hydrological studies. Satellite-based rainfall products, which allow for a better coverage than both radar network and rain gauges on the Tibetan Plateau, can be suitable alternatives for studies on investigating the hydrological processes and climate change. In this study, a newly developed daily satellite-based precipitation product, termed Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks $-$ Climate Data Record (PERSIANN-CDR), is used as input for a hydrologic model to simulate streamflow in the upper Yellow and Yangtze River basins on the Tibetan Plateau. The results show that the simulated streamflows using PERSIANN-CDR precipitation and the Global Land Data Assimilation System (GLDAS) precipitation are closer to observation than that using limited gauge-based precipitation interpolation in the upper Yangtze River basin. The simulated streamflow using gauge-based precipitation are higher than the streamflow observation during the wet season. In the upper Yellow River basin, gauge-based precipitation, GLDAS precipitation, and PERSIANN-CDR precipitation have similar good performance in simulating streamflow. Finally, the evaluation of streamflow simulation capability in this study partly indicates that the PERSIANN-CDR rainfall product has good potential to be a reliable dataset and an alternative information source of a limited gauge network for conducting long-term hydrological and climate studies on the Tibetan Plateau.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Predicting ocean waves along the US east coast during energetic winter storms: sensitivity to whitecapping parameterizations

The performance of two methods for quantifying whitecapping dissipation incorporated in the Simulating Waves Nearshore (SWAN) wave model is evaluated for waves generated along and off the US east coast under energetic winter storms with a predominantly westerly wind. Parameterizing the whitecapping effect can be done using the Komen-type schemes, which are based on mean spectral parameters, or the saturation-based (SB) approach of van der Westhuysen (2007), which is based on local wave parameters and the saturation level concept of the wave spectrum (we use “Komen” and “Westhuysen” to denote these two approaches). Observations of wave parameters and frequency spectra at four National Data Buoy Center (NDBC) buoys are used to evaluate simulation results. Model–data comparisons show that when using the default parameters in SWAN, both Komen and Westhuysen methods underestimate wave height. Simulations of mean wave period using the Komen method agree with observations, but those using the Westhuysen method are substantially lower. Examination of source terms shows that the Westhuysen method underestimates the total energy transferred into the wave action equations, especially in the lower frequency bands that contain higher spectral energy. Several causes for this underestimation are identified. The primary reason is the difference between the wave growth conditions along the east coast during winter storms and the conditions used for the original whitecapping formula calibration. In addition, some deficiencies in simulation results are caused along the coast by the “slanting fetch” effect that adds low-frequency components to the 2-D wave spectra. These components cannot be simulated partly or entirely by available source terms (wind input, whitecapping, and quadruplet) in models and their interaction. Further, the effect of boundary layer instability that is not considered in the Komen and Westhuysen whitecapping wind input formulas may cause additional underestimation.

Allahdadi, Mohammad Nabi↗
DOE OSTI2016

Can nudging be used to quantify model sensitivities in precipitation and cloud forcing?

Efficient simulation strategies are crucial for the development and evaluation of high resolution climate models. This paper evaluates simulations with constrained meteorology for the quantification of parametric sensitivities in the Community Atmosphere Model version 5 (CAM5). Two parameters are perturbed as illustrating examples: the convection relaxation time scale (TAU), and the threshold relative humidity for the formation of low-level stratiform clouds (rhminl). Results suggest that the fidelity and computational efficiency of the constrained simulations depend strongly on 3 factors: the detailed implementation of nudging, the mechanism through which the perturbed parameter affects precipitation and cloud, and the magnitude of the parameter perturbation. In the case of a strong perturbation in convection, temperature and/or wind nudging with a 6-hour relaxation time scale leads to non-negligible side effects due to the distorted interactions between resolved dynamics and parameterized convection, while a 1-year free running simulation can satisfactorily capture the annual mean precipitation sensitivity in terms of both global average and geographical distribution. In the case of a relatively weak perturbation the large-scale condensation scheme, results from 1-year free-running simulations are strongly affected by noise associated with internal variability, while nudging winds effectively reduces the noise, and reasonably reproduces the response of precipitation and cloud forcing to parameter perturbation. These results indicate that caution is needed when using nudged simulations to assess precipitation and cloud forcing sensitivities to parameter changes in general circulation models. We also demonstrate that ensembles of short simulations are useful for understanding the evolution of model sensitivities.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles↗
DOE OSTI2020

Metaheuristic Optimization Tool

The metaheuristic optimization tool (MOT) is a generic python module for solving a real value, multiobjective, multidimensional optimization problem. Users design their optimization problem by defining search variables, simulations, evaluation metric functions, and constraints. Search variables and constraints are straightforwardly added via python object methods, while simulations and metric functions used for search variable set evaluation must be implemented as a python class. Final optimum result(s) are provided in a tabular text file, as well as search history if needed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

PCMDI Metrics Package

The Program for Climate Model Diagnosis & Intercomparison (PCMDI) Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. The PMP provides a diverse suite of analysis utilities each of which produce summary statistics that gauge the consistency between climate model simulations and available observations. The primary application of the PMP is to evaluate simulations from the Coupled Model Intercomparison Project (CMIP). It can also be used to provide objective performance summaries during the model development process as well as selected research purposes.

Ullrich, PaulA [Lawrence Livermore National Labora↗
DOE OSTI2026

Evaluation of Howard A. Hanson Dam Juvenile Fish Passage and Survival Study Live Fish Injury Assessment, Sensor Fish, and BioPA Modeling Tasks

The live fish injury assessment, Sensor Fish, and BioPA modeling study tasks were conducted by researchers from Pacific Northwest National Laboratory (PNNL). The four tasks were part of the larger Evaluation of Howard A. Hanson Dam (HAHD) Juvenile Fish Passage and Survival study, which had six total tasks. To achieve study objectives for each of the four tasks, field work occurred at Green Peter Dam (GPR) to evaluate the highest elevation steep slope bypass pipe, at HAHD to evaluate baseline conditions of the horseshoe tunnel, and at PNNL’s Aquatic Research Laboratory (ARL) to evaluate simulated dam passage conditions (i.e., shear forces and collision). Each of these evaluations utilized live fish injury assessment, Sensor Fish, and BioPA modeling. Live fish injury assessment and survival (tagged with and without balloon or passive integrated transponder [PIT] tags) was correlated with Sensor Fish to determine thresholds. The CFD analyses were then performed, and the computed values were compared to the corresponding measured values of Sensor Fish data. The results of the overall injury and survival of fish was also used in the validation of the CFD modeling method. Collectively, the results will aid in future modeling of fish passage at HAHD. Results from these tasks can be used by biologists, engineers, resource managers, and regional decision-makers to inform baseline conditions under current operations and the engineering design of the new FPF at HAHD. This draft report contains initial data and results from the four tasks. Table 8 1, Table 8 2, and Table 8 3, and Figure 8 1, Figure 8 2, and Figure 8 3 depict the CFD modeling findings for the GPR steep slope bypass, HAHD horseshoe tunnel, and laboratory testing. Table 8 4, Table 8 5, and Table 8 6 depict the Sensor Fish findings for the GPR steep slope bypass and HAHD horseshoe tunnel testing. The Mv values observed in the HAHD were significantly lower compared to the laboratory experiments conducted at PNNL. Currently, investigations are underway to understand the reasons for this disparity and to establish an appropriate threshold value for Mv. Survival predictions presented in the tables below should be considered preliminary and should not be used until further analyses and adjustments are completed. The next steps for modeling will include the flow regime, (i.e., density of flow regimes due to water and air mixing ) to continue to improve on the threshold value for Mv.

13 HYDRO ENERGY↗
DOE OSTI2021

Sequential Learning of Active Subspaces

This study shows that in recent years, active subspace methods (ASMs) have become a popular means of performing subspace sensitivity analysis on black-box functions. Naively applied, however, ASMs require gradient evaluations of the target function. In the event of noisy, expensive, or stochastic simulators, evaluating gradients via finite differencing may be infeasible. In such cases, often a surrogate model is employed, on which finite differencing is performed. When the surrogate model is a Gaussian process (GP), we show that the ASM estimator is available in closed form, rendering the finite-difference approximation unnecessary. We use our closed-form solution to develop acquisition functions focused on sequential learning tailored to sensitivity analysis on top of ASMs. We also show that the traditional ASM estimator may be viewed as a method of moments estimator for a certain class of GPs. We demonstrate how uncertainty on GP hyperparameters may be propagated to uncertainty on the sensitivity analysis, allowing model-based confidence intervals on the active subspace. Our methodological developments are illustrated on several examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Method of Test for Evaluating Building Performance Simulation Software

ANSI/ASHRAE Standard 140 Method of Test for Evaluating Building Performance Simulation Software specifies the method to test the core competency of building performance simulation (BPS) software, a broad class of software which includes building energy modeling (BEM) software. Standard 140 has been widely used by BEM software vendors to help diagnose and compare the results from the program with other modeling programs. The Standard is also referenced by codes, standards, government programs, and other incentive programs as a part of minimum requirements for qualifying BEM software. The explanatory information, summary tables and figures in this 140 User’s Manual (referred to as “this Manual”) are provided to help users in implementing various suites of tests specified in Standard 140-2023 (referred to in this manual as “Standard 140” or “the Standard”).

97 MATHEMATICS AND COMPUTING↗