Search NASA⌕ Search

SEARCH · Search NASA

Results for “simulation evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72

Evaluation of an air solar collector

Performance verification under simulated conditions tested by using Marshall Space Flight Center solar simulator is presented. Evaluation included thermal performance tests, time constant tests, and incident angle modifier tests.

Source record↗

A high-fidelity building performance simulation test bed for the development and evaluation of advanced controls

We present an open-source building performance simulation test bed, the Advanced Controls Test Bed (ACTB), that interfaces high-fidelity Spawn of EnergyPlus building models, with advanced controllers implemented in Python. Additionally, the ACTB leverages the Building Optimization Testing and Alfalfa platforms for managing simulations, providing an external clock, a representational state transfer (REST) application programming interface (API), and key performance indicators for evaluating the effectiveness of control strategies. The REST API allows the development of external controllers programmed in languages such as Python, which provides flexibility and a rich choice of scientific libraries for designing control sequences. We present three test cases based on the U.S. Department of Energy's Reference Small Office Building to demonstrate the ACTB's capabilities: (a) rule-based controls compliant with ASHRAE Guideline 36 control sequences; (b) an economic model predictive control implemented using do-mpc; and (c) a deep Q-network reinforcement learning agent implemented using OpenAI Gym.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Corroborative Evaluation of the Real-World Energy Saving Potentials of InfoRich Eco-Autonomous Driving (iREAD) System

There has been an increasing interest in exploring the potential to reduce energy consumption of future connected and automated vehicles. People have extensively studied various eco-driving implementations that leverage preview information provided by on-board sensors and connectivity, as well as the control authority enabled by automation. Quantitative real-world evaluation of eco-driving benefits is a challenging task. The standard regulatory driving cycles used for measuring exhaust emissions and fuel economy are not truly representative of real-world driving, nor for capturing how connectivity and automation might influence driving trajectories. To adequately consider real-world driving behavior and potential “off-cycle” impacts, this paper presents four collaborative evaluation methods: large-scale simulation, in-depth simulation, vehicle-in-the-loop testing, and vehicle road testing. These four approaches, spanning simulation and testing aspects, evaluate real-world fuel economy benefits with different ranges and resolutions. The large-scale simulations leverage an extensive real-world driving database to assess overall eco-driving benefits across a range of road network and driving scenarios. The real-world driving data are further leveraged to generate representative driving routes for deeper evaluation. Based on the representative routes, in-depth simulation relying on high-fidelity models investigates how different traffic scenarios can impact the eco-driving performance. The vehicle-in-the-loop setup reinforces the in-depth simulations by conducting tests with an actual vehicle operated on a chassis dynamometer; the measured energy savings were indeed found to agree with the in-depth simulation savings estimates. Finally, limited but representative road testing with the fully integrated vehicle will be conducted to demonstrate the eco-driving capability and conclude the overall evaluation regimen.

ARPA-E↗

Evaluating E3SM Global Storm‐Resolving Model Simulations of Deep Convection: Insights From DP‐SCREAM During TRACER

Global Storm-Resolving Models (GSRMs) are becoming increasingly vital for advancing climate modeling and improving the prediction of extreme weather events. Houston, a coastal region frequently affected by deep convective storms, offers an ideal setting to evaluate the ability of GSRMs to simulate deep convection. This study assesses the performance of the Doubly Periodic Simple Cloud-Resolving E3SM (Energy Exascale Earth System Model) Atmosphere Model (DP-SCREAM) using observations from the TRacking Aerosol Convection interactions ExpeRiment (TRACER) campaign. DP-SCREAM effectively reproduces the diurnal cycles of clouds and precipitation, demonstrating much greater skill than the E3SM single column model. The DP-SCREAM is demonstrated to be applicable to coastal regions, partially due to the forcing data sets already capturing the influence of breezes. DP-SCREAM also replicates biases persistent in the global version of SCREAM: the underrepresentation of boundary layer shallow clouds, a lack of mid-level congestus clouds, and the popcorn convection, characterized by small and disorganized convective cells generating the strongest precipitation. To investigate these issues, two sensitivity experiments were conducted: increasing the mixing length and scaling up the buoyancy flux within the Simplified Higher Order Closure scheme. Increasing the mixing length improved mid-level congestus representation and reduced unrealistic early morning fog occurrence. Enhancing buoyancy flux only marginally improved the bias of underproduced big convective cells. In conclusion, an additional resolution sensitivity test at 0.5 km grid spacing demonstrated that a refined horizontal resolution alone is insufficient to resolve these biases.

54 ENVIRONMENTAL SCIENCES↗

Emerging Computing Architectures: Simulation of Power Electronics in Power Grids

As the penetration of power electronics increases in power grids, new computing architectures needed to be evaluated for the simulation of high-fidelity models of power electronics in power grids in operations. In this paper, emerging computing architectures such as quantum processing units are evaluated for the simulation of power electronics in power grids. A hybrid algorithm based on classical computing and quantum computing is developed and tested for different use cases of electromagnetic transient (EMT) simulation of power electronics (PE)-based systems and simple circuits. The algorithms needed to simulate power electronics in emerging computing architectures are discussed and thereafter, simulation results are shown.

Debnath, Suman↗

Clinical evaluation of the iterative metal artifact reduction algorithm for CT simulation in radiotherapy

Purpose: To clinically evaluate an iterative metal artifact reduction (IMAR) algorithm prototype in the radiation oncology clinic setting by testing for accuracy in CT number retrieval, relative dosimetric changes in regions affected by artifacts, and improvements in anatomical and shape conspicuity of corrected images. Methods: A phantom with known material inserts was scanned in the presence/absence of metal with different configurations of placement and sizes. The relative change in CT numbers from the reference data (CT with no metal) was analyzed. The CT studies were also used for dosimetric tests where dose distributions from both photon and proton beams were calculated. Dose differences and gamma analysis were calculated to quantify the relative changes between doses calculated on the different CT studies. Data from eight patients (all different treatment sites) were also used to quantify the differences between dose distributions before and after correction with IMAR, with no reference standard. A ranking experiment was also conducted to analyze the relative confidence of physicians delineating anatomy in the near vicinity of the metal implants. Results: IMAR corrected images proved to accurately retrieve CT numbers in the phantom study, independent of metal insert configuration, size of the metal, and acquisition energy. For plastic water, the mean difference between corrected images and reference images was −1.3 HU across all scenarios (N = 37) with a 90% confidence interval of [−2.4, −0.2] HU. While deviations were relatively higher in images with more metal content, IMAR was able to effectively correct the CT numbers independent of the quantity of metal. Residual errors in the CT numbers as well as some induced by the correction algorithm were found in the IMAR corrected images. However, the dose distributions calculated on IMAR corrected images were closer to the reference data in phantom studies. Relative spatial difference in the dose distributions in the regions affected by the metal artifacts was also observed in patient data. However, in absence of a reference ground truth (CT set without metal inserts), these differences should not be interpreted as improvement/deterioration of the accuracy of calculated dose. With limited data presented, it was observed that proton dosimetry was affected more than photons as expected. Physicians were significantly more confident contouring anatomy in the regions affected by artifacts. While site specific preferences were detected, all indicated that they would consistently use IMAR corrected images. Conclusions: IMAR correction algorithm could be readily implemented in an existing clinical workflow upon commercial release. While residual errors still exist in IMAR corrected images, these images present with better overall conspicuity of the patient/phantom geometry and offer more accurate CT numbers for improved local dosimetry. The variety of different scenarios included herein attest to the utility of the evaluated IMAR for a wide range of radiotherapy clinical scenarios.

60 APPLIED LIFE SCIENCES↗

Performance Portability Evaluation of Fluid-Structure Interaction Simulations on Heterogeneous Platforms

The rapid proliferation of heterogeneous programming languages and multi-vendor hardware has underscored the critical need to evaluate the performance portability of scientific applications. In this work, we present the systematic porting and optimization of a massively parallel fluid-structure interaction code across multiple heterogeneous programming frameworks for deployment on leadership-class supercomputers from major vendors. Our analysis focuses on at-scale performance for simulations involving hundreds of millions of deformable cells, executed on a combination of CPUs and GPUs spanning thousands of nodes on exascale machines. We benchmark the performance of each implementation, highlighting the trade-offs inherent in adopting diverse programming models. Key insights regarding the portability of CUDA on multi-vendor platforms, the superior multi-core CPU performance from SYCL, and architectural considerations on performance optimization are distilled from our experience, offering guidance to other users of high performance computing based on our findings.

Martin, Aristotle [Duke University]↗

Incorporating the Acclimation of Photosynthesis and Leaf Respiration in the Noah‐MP Land Surface Model: Model Development and Evaluation

Abstract Realistic simulation of leaf photosynthetic and respiratory processes is needed for accurate prediction of the global carbon cycle. These two processes systematically acclimate to long‐term environmental changes by adjusting photosynthetic and respiratory traits (e.g., the maximum photosynthetic capacity at 25°C ( V cmax,25 ) and the leaf respiration rate at 25°C ( R 25 )) following increasingly well‐understood principles. While some land surface models (LSMs) now account for thermal acclimation, they do so by assigning empirical parameterizations for individual plant functional types (PFTs). Here, we have implemented an Eco‐Evolutionary Optimality (EEO)‐based scheme to represent the universal acclimation of photosynthesis and leaf respiration to multiple environmental effects, and that therefore requires no PFT‐specific parameterizations, in a standard version of the widely used LSM, Noah MP. We evaluated model performance with plant trait data from a 5‐year experiment and extensive global field measurements, and carbon flux measurements from FLUXNET2015. We show that observed R 25 and V cmax,25 vary substantially both temporally and spatially within the same PFT ( C.V. >20%). Our EEO‐based scheme captures 62% of the temporal and 70% of the spatial variations in V cmax,25 (73% and 54% of the variations in R 25 ). The standard scheme underestimates gross primary production by 10% versus 2% for the EEO‐based scheme and generates a larger spread in r (correlation coefficient) across flux sites (0.79 ± 0.16 vs. 0.84 ± 0.1, mean ± S.D.). The standard scheme greatly overestimates canopy respiration (bias: ∼200% vs. 8% for the EEO scheme), resulting in less CO 2 uptake by terrestrial ecosystems. Our approach thus simulates climate‐carbon coupling more realistically, with fewer parameters.

Ren, Yanghang [Department of Earth System Science ↗

Topologically Guided, Automated Construction of Metal–Organic Frameworks and Their Evaluation for Energy-Related Applications

Metal-organic frameworks (MOFs) are promising materials for a range of energy and environmental applications. Here we describe in detail a computational algorithm and code to generate MOFs based on edge-transitive topological nets for subsequent evaluation via molecular simulation. This algorithm has been previously used by us to construct and evaluate 13 512 MOFs of 41 different topologies for cryo-adsorbed hydrogen storage. Grand canonical Monte Carlo simulations are used here to evaluate the 13 512 structures for the storage of gaseous fuels such as hydrogen and methane and nondistillative separation of xenon/krypton mixtures at various operating conditions. MOF performance for both gaseous fuel storage and xenon/krypton separation is influenced by topology. In conclusion, simulation data suggest that gaseous fuel storage performance is topology-dependent due to MOF properties such as void fraction and surface area combining differently in different topologies, whereas xenon/krypton separation performance is topology-dependent due to how topology constrains the pore size distribution.

36 MATERIALS SCIENCE↗

Thermal Protection System Evaluation Using Arc-jet Flows: Flight Simulation or Research Tool?

The arc-jet has been used to evaluate thermal protection systems (TPS) and materials for the past forty years. Systems that have been studied in this environmerd include ablators, active, and passive TPS concepts designed for vehicles entering planetary and Earth atmospheres. The question of whether arc-jet flow can simulate a flight environment or is it a research tool that provides an aero-thermodynamic heating environment to obtain critical material properties will be addressed. Stagnation point tests in arc-jets are commonly used to obtain material properties such as mass loss rates, thermal chemical stability data, optical properties, and surface catalytic efficiency. These properties are required in computational fluid dynamic codes to accurately predict the performance of a TPS during flight. Special facilities have been developed at NASA Ames Research Center to approximate the flow environment over the mid-fuselage and body flap regions of proposed space-planes type vehicles. This paper compares flow environments generated in flight over a vehicle with those created over an arc-jet test articles in terms of scale, chemistry, and fluid dynamic properties. Flight experiments are essential in order to validate the material properties obtained from arc-jet tests and used to predict flight performance of any TPS being considered for use on a vehicle entering the Earth atmosphere at hypersonic speed.

Stewart, David A.↗

Evaluation of Mesoscale Convective Systems in Climate Simulations: Methodological Development and Results from MPAS-CAM over the U.S.

In this study, a process-oriented approach is developed to evaluate warm-season mesoscale convective system (MCS) precipitation and their favorable large-scale meteorological patterns (FLSMPs) over the U.S. This approach features a novel observation-driven MCS-tracking algorithm using infrared brightness temperature and precipitation feature at 12, 25 and 50 km resolution and metrics to evaluate the model large-scale environment favorable for MCS initiation. The tracking algorithm successfully reproduces the observed MCS statistics from a reference 4-km radar MCS database. To demonstrate the utility of the new methodologies in evaluating MCS in climate simulations with mesoscale resolution, the process-oriented approach is applied to two climate simulations produced by the Variable-Resolution Model for Prediction Across Scales coupled to the Community Atmosphere Model physics, with refined horizontal grid spacing at 50 km and 25 km over North America. With the tracking algorithm applied to simulations and observations at equivalent resolutions, the simulated number of MCS and associated precipitation amount, frequency and intensity are found to be consistently underestimated in the Central U.S., particularly from May to August. The simulated MCS precipitation shows little diurnal variation and lasts too long, while MCS precipitation area is too large and intensity is too weak. The model is able to simulate four types of observed FLSMP associated with frontal systems and low-level jets (LLJ) in spring, but the frequencies are underestimated because of low-level dry bias and weaker LLJ. Precipitation simulated under different FLSMPs peak during daytime, in contrast to the observed nocturnal peak. Implications of these findings for future model development and diagnostics are discussed.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of cloud-resolving model simulations of midlatitude cirrus with ARM and A-train observations

Cirrus clouds are ubiquitous in the upper troposphere and still constitute one of the largest uncertainties in climate predictions. Our paper evaluates cloud-resolving model (CRM) and cloud system-resolving model (CSRM) simulations of a midlatitude cirrus case with comprehensive observations collected under the auspices of the Atmospheric Radiation Measurements (ARM) program and with spaceborne observations from the National Aeronautics and Space Administration A-train satellites. The CRM simulations are driven with periodic boundary conditions and ARM forcing data, whereas the CSRM simulations are driven by the ERA-Interim product. Vertical profiles of temperature, relative humidity, and wind speeds are reasonably well simulated by the CSRM and CRM, but there are remaining biases in the temperature, wind speeds, and relative humidity, which can be mitigated through nudging the model simulations toward the observed radiosonde profiles. Simulated vertical velocities are underestimated in all simulations except in the CRM simulations with grid spacings of 500 m or finer, which suggests that turbulent vertical air motions in cirrus clouds need to be parameterized in general circulation models and in CSRM simulations with horizontal grid spacings on the order of 1 km. The simulated ice water content and ice number concentrations agree with the observations in the CSRM but are underestimated in the CRM simulations. The underestimation of ice number concentrations is consistent with the overestimation of radar reflectivity in the CRM simulations and suggests that the model produces too many large ice particles especially toward the cloud base. Simulated cloud profiles are rather insensitive to perturbations in the initial conditions or the dimensionality of the model domain, but the treatment of the forcing data has a considerable effect on the outcome of the model simulations. Despite considerable progress in observations and microphysical parameterizations, simulating the microphysical, macrophysical, and radiative properties of cirrus remains challenging. Comparing model simulations with observations from multiple instruments and observational platforms is important for revealing model deficiencies and for providing rigorous benchmarks. But, there still is considerable need for reducing observational uncertainties and providing better observations especially for relative humidity and for the size distribution and chemical composition of aerosols in the upper troposphere.

54 ENVIRONMENTAL SCIENCES↗

Effects of simulator motion and visual characteristics on rotorcraft handling qualities evaluations

The pilot's perceptions of aircraft handling qualities are influenced by a combination of the aircraft dynamics, the task, and the environment under which the evaluation is performed. When the evaluation is performed in a groundbased simulator, the characteristics of the simulation facility also come into play. Two studies were conducted on NASA Ames Research Center's Vertical Motion Simulator to determine the effects of simulator characteristics on perceived handling qualities. Most evaluations were conducted with a baseline set of rotorcraft dynamics, using a simple transfer-function model of an uncoupled helicopter, under different conditions of visual time delays and motion command washout filters. Differences in pilot opinion were found as the visual and motion parameters were changed, reflecting a change in the pilots' perceptions of handling qualities, rather than changes in the aircraft model itself. The results indicate a need for tailoring the motion washout dynamics to suit the task. Visual-delay data are inconclusive but suggest that it may be better to allow some time delay in the visual path to minimize the mismatch between visual and motion, rather than eliminate the visual delay entirely through lead compensation.

Mitchell, David G.↗

Evaluation of a Multi-Decadal Simulation of Stratospheric Ozone by Comparison with Total Ozone Mapping Spectrometer (TOMS) Observations

One key application of atmospheric chemistry and transport models is prediction of the response of ozone and other constituents to various natural and anthropogenic perturbations. These include changes in composition, such as the previous rise and recent decline in emission of man-made chlorofluorcarbons, changes in aerosol loading due to volcanic eruption, and changes in solar forcing. Comparisons of hindcast model results for the past few decades with observations are a key element of model evaluation and provide a sense of the reliability of model predictions. The 25 year data set from Total Ozone Mapping Spectrometers is a cornerstone of such model evaluation. Here we report evaluation of three-dimensional multi-decadal simulation of stratospheric composition. Meteorological fields for this off-line calculation are taken from a 50 year simulation of a general circulation model. Model fields are compared with observations from TOMS and also with observations from the Stratospheric Aerosol and Gas Experiment (SAGE), Microwave Limb Sounder (MLS), Cryogenic Limb Array Etalon Spectrometer (CLAES), and the Halogen Occultation Experiment (HALOE). This overall evaluation will emphasize the spatial, seasonal, and interannual variability of the simulation compared with observed atmospheric variability.

Douglass, Anne R.↗

Evaluation of multi-decadal UCLA-CFSv2 simulation and impact of interactive atmospheric-ocean feedback on global and regional variability

This paper evaluates multi-decadal simulations of the UCLA version of Climate Forecast System version 2 (CFSv2), in which the default Noah land surface model has been replaced with the Simplified Simple Biosphere Model version-2 (SSiB2). To examine the influence of the atmosphere-ocean (AO) interaction on the variability, two different simulations were conducted: one with interactive ocean component, and the other constrained by the prescribed sea surface temperature. We evaluate the mean seasonal climatology of precipitation and temperature, along with the model’s ability to reproduce atmospheric variability at different scales over the globe, including extratropical modes of atmospheric variability, and long-term trends of global and hemispheric temperature and regional precipitation. Here, we particularly selected two monsoon regions, East Asia and West Africa, where the simulation of multi-decadal variations which has heretofore been a challenging task, to examine decadal variation of monsoon precipitation. In general, temperature anomaly trends were better captured than those of precipitation in both simulations. Results suggest that the AO interaction, represented as latent heat flux, contributes to improve reproducibility of global-wide climatology, extratropical modes of atmospheric variability, and variability in the multi-decadal climate simulation, as well as for inter decadal variability of the East Asian summer monsoon.

54 ENVIRONMENTAL SCIENCES↗

Collaborative simulations and experiments for a novel yield model of coal devolatilization in oxy-coal combustion conditions

Oxy-coal combustion is an emerging low-cost “clean coal” technology for emissions reduction and Carbon Capture and Sequestration (CCS). The use of Computational Fluid Dynamics (CFD) tools is crucial for the development of cost-effective oxy-fuel technologies and the minimization of environmental concerns at industrial scale. The coupling of detailed chemistry models and CFD simulations is still challenging, especially for large-scale plants, because of the high computational efforts required. The development of scale-bridging models is therefore necessary, to find a good compromise between computational efforts and the physical-chemical modeling precision. This paper presents a procedure for scale-bridging modeling of coal devolatilization, in the presence of experimental error, that puts emphasis on the thermodynamic aspect of devolatilization, namely the final volatile yield of coal, rather than kinetics. The procedure consists of an engineering approach based on dataset consistency and Bayesian methodology including Gaussian-Process Regression (GPR). Experimental data from devolatilization tests carried out in an oxy-coal entrained flow reactor were considered and CFD simulations of the reactor were performed. Jointly evaluating experiments and simulations, a novel yield model was validated against the data via consistency analysis. In parallel, a Gaussian-Process Regression was performed, to improve the understanding of the uncertainty associated to the devolatilization, based on the experimental measurements. Potential model forms that could predict yield during devolatilization were obtained. The set of model forms obtained via GPR includes the yield model that was proven to be consistent with the data. Finally, the overall procedure has resulted in a novel yield model for coal devolatilization and in a valuable evaluation of uncertainty in the data, in the model form, and in the model parameters.

01 COAL, LIGNITE, AND PEAT↗

SU-F-P-51: Similarity Analysis of the Linear Accelerator Machines Based On Clinical Simulation

Purpose: To evaluate the clinical rationale for Truebeam and Trilogy Linac machines from Varian Medical System as exchangeable treatment modalities in the same radiation oncology department. Methods: Intensity Modulated Radiotherapy (IMRT) plans for different diseases were selected for this study. These disease sites included brain, head and neck, breast, lung, and prostate. The parameters selected for this study were the energy amount, Monitor Unit (MU); dose coverage of target reflected by prescription isodose volume(PIV); dose spillage described by the volume of 50% isodoseline of the prescription; and dose homogeneities represented by the maximum dose (MaxD) and the minimum dose (MinD) of target volume (TV) and critical structure (CS). Each percentage difference between the values of these parameters formed an element of a matrix, which was called Similarity Comparison Matrix(SCM). The elements of the SCM were then simplified by dimensional conversion algorithm, which was used to determine clinical similarity between two machines through a single value. Results: For the selected clinical cases in this study, the average percentage differences between Trilogy and Truebeam in MU was 0.28% with standard deviation(SD) 0.66%, PIV was 0.23% with SD 0.20%, Volume at 50% prescription dose was 0.31% with SD at 0.78%, MaxD at TV is 0.26% with SD 0.35%, MinD at TV is −0.04% with SD 0.51%, MaxD in CS is −0.53% with SD 0.92%, and MinD in CS 3.31%, with SD at 2.89%. The sum, product, geometric and harmonic mean for the matrix elements were 19.0%, 0.00%, 0.19%, and 0.00%. Conclusion: A method to compare two machines in clinical level was developed and some reference values were calculated for decision-making in clinical practice, and this strategy could be expanded to different clinical applications.

60 APPLIED LIFE SCIENCES↗