Search NASA⌕ Search

SEARCH · Search NASA

Results for “simulation evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Use of Criticality Eigenvalue Simulations for Subcritical Benchmark Evaluations

Subcritical multiplication experiments and simulations are important for a variety of applications including nonproliferation, safeguards, and criticality safety monitoring. Recently, subcritical experiments have been evaluated for inclusion in the International Criticality Safety Benchmark Evaluation Process (ICSBEP). In addition, new subcritical experiments have recently been designed. In these experiments, list-mode data (a list of each absorption in the neutron detector) is recorded and neutron noise analysis is used to determine the leakage multiplication (ML) of each configuration. One downside to the simulation of these types of experiments is that they are computationally expensive. This is because analog Monte Carlo with no variance reduction must be performed in order to maintain the time accuracy required in the measurement analysis. This work demonstrates that criticality eigenvalue simulations can be used to estimate sensitivities and uncertainties for three parameters: singles count rate (R{sub 1}), doubles count rate (R{sub 2}), and leakage multiplication (M{sub L}). This will provide a useful aid both in the design and evaluation of new subcritical measurements. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A simulation framework for evaluating electronic order workflows in integrated health records

Electronic health record (EHR) systems are critical to modern healthcare delivery, yet the dynamic workflows that govern electronic order processing remain underexplored. Inefficiencies in these digital pathways can cause delays in care, repetitive workloads, and even patient harm. This study presents a discrete-event simulation framework used to reconstruct and evaluate EHR-based order workflows in a large integrated healthcare system. Using real-world data extracted from the Veterans Health Administration’s Corporate Data Warehouse, the authors mapped order events to standardized state transitions and modeled their progression across different facilities of varying complexity levels. After being calibrated with empirical distributions of transition times and validated against observed time-in-system metrics, the simulation demonstrates close alignment with historical performance. Scenario analyses reveal that resource capacity constraints significantly amplify the impact of electronic order surges, which are reflected in the disproportionate growth in backlogs and processing delays. Adjustments in transition probabilities further increased recirculation and extended workflow paths. Network-based analysis identified Reserved, InProgress, and Completed as structurally critical states that function as hubs within the process network but the transitions in-between also act as major bottlenecks. These results showcased the effectiveness of simulation-based approaches in monitoring EHR order processing performance and evaluating consequences of workflow changes on healthcare network resources planning. The proposed simulation framework provides a scalable data-driven tool to support operational decision-making and improve the efficiency of electronic order management in complex healthcare environments.

Engineering↗

Simulation development and evaluation of an improved longitudinal velocity vector control wheel steering mode and electronic display format

Using simulation, an improved longitudinal velocity vector control wheel steering mode and an improved electronic display format for an advanced flight system were developed and tested. Guidelines for the development phase were provided by test pilot critique summaries of the previous system. The results include performances from computer generated step column inputs across the full airplane speed and configuration envelope, as well as piloted performance results taken from a reference line tracking task and an approach to landing task conducted under various environmental conditions. The analysis of the results for the reference line tracking and approach to landing tasks indicates clearly detectable improvement in pilot tracking accuracy with a reduction in physical workload. The original objectives of upgrading the longitudinal axis of the velocity vector control wheel steering mode were successfully met when measured against the test pilot critique summaries and the original purpose outlined for this type of augment control mode.

Steinmetz, G. G.↗

NAS-Wide Fast-Time Simulation Study for Evaluating Performance of UAS Detect-and-Avoid Alerting and Guidance Systems

This presentation contains the analysis results of NAS-wide fast-time simulations with UAS and VFR traffic for a single day for evaluating the performance of Detect-and-Avoid (DAA) alerting and guidance systems. This purpose of this study was to help refine and validate MOPS alerting and guidance requirements. In this study, we generated plots of all performance metrics that are specified by RTCA SC-228 Minimum Operational Performance Standards (MOPS): 1) to evaluate the sensitivity of alerting parameters on the performance metrics of each DAA alert type: Preventive, Corrective, and Warning alerts and 2) to evaluate the effect of sensor uncertainty on DAA alerting and guidance performance.

DAA Alerting Performance↗

Methodology development for evaluation of selective-fidelity rotorcraft simulation

This paper addressed the initial step toward the goal of establishing performance and handling qualities acceptance criteria for realtime rotorcraft simulators through a planned research effort to quantify the system capabilities of 'selective fidelity' simulators. Within this framework the simulator is then classified based on the required task. The simulator is evaluated by separating the various subsystems (visual, motion, etc.) and applying corresponding fidelity constants based on the specific task. This methodology not only provides an assessment technique, but also provides a technique to determine the required levels of subsystem fidelity for a specific task.

Lewis, William D.↗

Evaluation of global simulations of aerosol particle and cloud condensation nuclei number, with implications for cloud droplet formation

A total of 16 global chemistry transport models and general circulation models have participated in this study; 14 model shave been evaluated with regard to their ability to reproduce the near-surface observed number concentration of aerosol particles and cloud condensation nuclei (CCN), as well as derived cloud droplet number concentration (CDNC). Model results for the period 2011–2015 are compared with aerosol measurements(aerosol particle number, CCN and aerosol particle composition in the submicron fraction) from nine surface stations located in Europe and Japan.The evaluation focuses on the ability of models to simulate the average across time state in diverse environments and on the seasonal andshort-term variability in the aerosol properties. There is no single model that systematically performs best across all environments represented by the observations. Models tend to underestimate the observed aerosol particle and CCN number concentrations, with average normalized mean bias (NMB) of all models and for all stations, where dataare available, of –24% and –35% for particles with dry diameters <50 and <120 nm, as well as –36% and –34% for CCNat supersaturations of 0.2% and 1.0%, respectively. Yet, they seem to behave differently for particles activating at very low supersaturations (<0.1%) than at higher ones. A total of 15 models have been used to produce ensemble annual median distributions of relevant parameters. The model diversity (defined as the ratio of standard deviation to mean) is up to about 3 for simulated N 3 (number concentration of particles with dry diameters larger than 3 nm) and up to about 1 for simulated CCN in the extra-polar regions. A global mean reduction of a factor of about 2 is found in the model diversity for CCN at a supersaturation of 0.2% (CCN 0.2 ) compared to that for N 3 , maximizing over regions where new particle formation is important. An additional model has been used to investigate potential causes of model diversity in CCN and bias compared to the observations by performing a perturbed parameter ensemble (PPE) accounting for uncertainties in 26 aerosol-related model input parameters. This PPE suggests that biogenic secondary organic aerosol formation and the hygroscopic properties of the organic material are likely to be the major sources of CCN uncertainty in summer, with dry deposition and cloud processing being dominant in winter. Models capture the relative amplitude of the seasonal variability of the aerosol particle number concentration for all studied particle sizes with available observations (dry diameters larger than 50, 80 and 120 nm). The short-term persistence time (on the order of a few days) of CCN concentrations, which is a measure of aerosol dynamic behavior in the models, is underestimated o naverage by the models by 40% during winter and 20% in summer. In contrast to the large spread in simulated aerosol particle and CCN number concentrations, the CDNC derived from simulated CCN spectra is less diverse and in better agreement with CDNC estimates consistently derived from the observations (average NMB –13% and –22% for updraft velocities 0.3 and 0.6m s –1 , respectively). In addition, simulated CDNC is in slightly better agreement with observationally derived values at lower than at higher updraft velocities (index of agreement 0.64 vs. 0.65). The reduced spread of CDNC compared to that of CCN is attributed to the sublinear response of CDNC to aerosol particle number variations and the negative correlation between the sensitivities of CDNC to aerosol particle number concentration( ∂ N d / ∂ N a ) and to updraft velocity ( ∂ N d / ∂ w ). Overall, we find that while CCN is controlled by both aerosol particle number and composition, CDNC is sensitive to CCN at low and moderate CCN concentrations and to the updraft velocity when CCN levels are high. Discrepancies are found in sensitivities ∂ N d / ∂ N a and ∂ N d / ∂ w ; models may be predisposed to be too“aerosol sensitive” or “aerosol insensitive” in aerosol–cloud–climate interaction studies, even if they may capture average droplet numbers well.This is a subtle but profound finding that only the sensitivities can clearly reveal and may explain inter-model biases on the aerosol indirect effect.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Global Simulations of Aerosol Particle and Cloud Condensation Nuclei Number, with Implications for Cloud Droplet Formation

A total of 16 global chemistry transport models and general circulation models have participated in this study; 14 models have been evaluated with regard to their ability to reproduce the near-surface observed number concentration of aerosol particles and cloud condensation nuclei (CCN), as well as derived cloud droplet number concentration (CDNC). Model results for the period 2011-2015 are compared with aerosol measurements (aerosol particle number, CCN and aerosol particle composition in the submicron fraction) from nine surface stations located in Europe and Japan. The evaluation focuses on the ability of models to simulate the average across time state in diverse environments and on the seasonal and short-term variability in the aerosol properties. There is no single model that systematically performs best across all environments represented by the observations. Models tend to underestimate the observed aerosol particle and CCN number concentrations, with average normalized mean bias (NMB) of all models and for all stations, where data are available, of -24% and -35% for particles with dry diameters > 50 and > 120nm, as well as -36% and -34% for CCN at supersaturations of 0.2% and 1.0%, respectively. However, they seem to behave differently for particles activating at very low supersaturations (< 0.1%) than at higher ones. A total of 15 models have been used to produce ensemble annual median distributions of relevant parameters. The model diversity (defined as the ratio of standard deviation to mean) is up to about 3 for simulated N3 (number concentration of particles with dry diameters larger than 3 nm) and up to about 1 for simulated CCN in the extra-polar regions. A global mean reduction of a factor of about 2 is found in the model diversity for CCN at a supersaturation of 0.2% (CCN(0.2)) compared to that for N3, maximizing over regions where new particle formation is important. An additional model has been used to investigate potential causes of model diversity in CCN and bias compared to the observations by performing a perturbed parameter ensemble (PPE) accounting for uncertainties in 26 aerosol-related model input parameters. This PPE suggests that biogenic secondary organic aerosol formation and the hygroscopic properties of the organic material are likely to be the major sources of CCN uncertainty in summer, with dry deposition and cloud processing being dominant in winter. Models capture the relative amplitude of the seasonal variability of the aerosol particle number concentration for all studied particle sizes with available observations (dry diameters larger than 50, 80 and 120nm). The short-term persistence time (on the order of a few days) of CCN concentrations, which is a measure of aerosol dynamic behavior in the models, is underestimated on average by the models by 40% during winter and 20% in summer.

cloud droplets↗

Evaluating the GPSS simulation model for the Viking batch computer system

In anticipation of extremely heavy loading requirements by the Viking mission during the post-landing periods, a GPSS model has been developed for the purpose of simulating these requirements on the Viking batch computer system. This paper presents the effort pursued in evaluating such a model and results thereby obtained. The evaluation effort consists of selecting the evaluation approach, collecting actual test run data, making comparisons and deriving conclusions.

Lee, J.-J.↗

Evaluation of the Snow Simulations from the Community Land Model, Version 4 (CLM4)

This paper evaluates the simulation of snow by the Community Land Model, version 4 (CLM4), the land model component of the Community Earth System Model, version 1.0.4 (CESM1.0.4). CLM4 was run in an offline mode forced with the corrected land-only replay of the Modern-Era Retrospective Analysis for Research and Applications (MERRA-Land) and the output was evaluated for the period from January 2001 to January 2011 over the Northern Hemisphere poleward of 30 deg N. Simulated snow-cover fraction (SCF), snow depth, and snow water equivalent (SWE) were compared against a set of observations including the Moderate Resolution Imaging Spectroradiometer (MODIS) SCF, the Interactive Multisensor Snow and Ice Mapping System (IMS) snow cover, the Canadian Meteorological Centre (CMC) daily snow analysis products, snow depth from the National Weather Service Cooperative Observer (COOP) program, and Snowpack Telemetry (SNOTEL) SWE observations. CLM4 SCF was converted into snow-cover extent (SCE) to compare with MODIS SCE. It showed good agreement, with a correlation coefficient of 0.91 and an average bias of -1.54 x 10(exp 2) sq km. Overall, CLM4 agreed well with IMS snow cover, with the percentage of correctly modeled snow-no snow being 94%. CLM4 snow depth and SWE agreed reasonably well with the CMC product, with the average bias (RMSE) of snow depth and SWE being 0.044m (0.19 m) and -0.010m (0.04 m), respectively. CLM4 underestimated SNOTEL SWE and COOP snow depth. This study demonstrates the need to improve the CLM4 snow estimates and constitutes a benchmark against which improvement of the model through data assimilation can be measured.

northern hemisphere↗

Piloted evaluation of an integrated propulsion and flight control simulator

A piloted evaluation of the integrated flight and propulsion control simulator for advanced integrated propulsion and airframe control design is described. The evaluation will cover control effector gains and deadbands, control effectiveness and control authority, and heads up display functionality. For this evaluation the flight simulator is configured for transition flight using an advanced Short Take-Off and Vertical Landing fighter aircraft model, a simplified high-bypass turbofan engine model, fighter cockpit displays, and pilot effectors. The piloted tasks used for rating displays and control effector gains are described. Pilot comments and simulation results confirm that the display symbology and control gains are very adequate for the transition flight task. Additionally, it is demonstrated that this small-scale, fixed base flight simulator facility can adequately perform a real time, piloted control evaluation.

Bright, Michelle M.↗

Energy deposition evaluation for ultra-low energy electron beam irradiation systems using calibrated thin radiochromic film and Monte Carlo simulations

For evaluation of on-site dosimetry and process design in industrial use of ultra-low energy electron beam (ULEB) processes, we evaluate the energy deposition using a thin radiochromic film and a Monte Carlo simulation. The response of film dosimeter was calibrated using a high energy electron beam with an acceleration voltage of 2 MV and alanine dosimeters with uncertainty of 11% at coverage factor 2. Using this response function, the results of absorbed dose measurements for ULEB were evaluated from 10 kGy to 100 kGy as a relative dose. The deviation between the responses of deposit energy on the films and Monte Carlo simulations was within 15%. As far as this limitation, relative dose estimation using thin film dosimeters with response function obtained by high energy electron irradiation and simulation results is effective for ULEB irradiation processes management.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Real-time closed-loop simulation and upset evaluation of control systems in harsh electromagnetic environments

Digital control systems for applications such as aircraft avionics and multibody systems must maintain adequate control integrity in adverse as well as nominal operating conditions. For example, control systems for advanced aircraft, and especially those with relaxed static stability, will be critical to flight and will, therefore, have very high reliability specifications which must be met regardless of operating conditions. In addition, multibody systems such as robotic manipulators performing critical functions must have control systems capable of robust performance in any operating environment in order to complete the assigned task reliably. Severe operating conditions for electronic control systems can result from electromagnetic disturbances caused by lightning, high energy radio frequency (HERF) transmitters, and nuclear electromagnetic pulses (NEMP). For this reason, techniques must be developed to evaluate the integrity of the control system in adverse operating environments. The most difficult and illusive perturbations to computer-based control systems that can be caused by an electromagnetic environment (EME) are functional error modes that involve no component damage. These error modes are collectively known as upset, can occur simultaneously in all of the channels of a redundant control system, and are software dependent. Upset studies performed to date have not addressed the assessment of fault tolerant systems and do not involve the evaluation of a control system operating in a closed-loop with the plant. A methodology for performing a real-time simulation of the closed-loop dynamics of a fault tolerant control system with a simulated plant operating in an electromagnetically harsh environment is presented. In particular, considerations for performing upset tests on the controller are discussed. Some of these considerations are the generation and coupling of analog signals representative of electromagnetic disturbances to a control system under test, analog data acquisition, and digital data acquisition from fault tolerant systems. In addition, a case study of an upset test methodology for a fault tolerant electromagnetic aircraft engine control system is presented.

Belcastro, Celeste M.↗

Evaluation of 3D Simulation Capabilities within xRAGE using a Suite of Test Problems

This presentation describes a 3D verification and validation suite for xRAGE. Three verification test problems for hydrodynamics are presented: the Kidder ball problem, the Verney shell problem, and a 5-material compression problem. There is excellent agreement between 2D and 3D XRAGE simulation results for these problems, and between the xRAGE results and the benchmark solutions. In addition, two 3D ICF test problems are presented, based on an OMEGA direct drive capsule experiment, and on a NIF indirect drive capsule experiment. These two ICF test problems are used to evaluate two hydrodynamic methods within xRAGE. In addition to hydrodynamics the simulations also include three temperature plasma physics, thermal conduction and radiation diffusion. When simulating either capsule in 3D the newer unsplit hydrodynamic method in xRAGE produces more vorticity relative to the older default method. For the indirect drive capsule the 3D simulations are in reasonable agreement with the experimental values of ion temperature and neutron production. Unstable 3D vortex rings are present near the DT ice/carbon interface which enhance the confinement of the DT ice.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Simulation and flight evaluation of a head-up landing aid for general aviation

A head-up general aviation landing aid called a landing site indicator (LASI) was tested in a fixed-base, visual simulator and in an airplane to determine the effectiveness of the LASI. The display, which had a simplified format and method of implementation, presented to the pilot in his line of sight through the windshield a graphic representation of the airplane's velocity vector. In each testing model (simulation of flight), each of 4 pilots made 20 landing approaches with the LASI and 20 approaches without it. The standard deviations of approach and touchdown parameters were considered an indication of pilot consistency. Use of the LASI improved consistency and also reduced elevator, aileron, and rudder control activity. Pilots' comments indicated that the LASI reduced work load. An appendix is included with a discussion of the simulator effectiveness for visual flight tasks.

Harris, R. L., Sr.↗

Incorporation of aerosol into the COSPv2 satellite lidar simulator for climate model evaluation

Abstract. Atmospheric aerosol has substantial impacts on climate, air quality and biogeochemical cycles, and its concentrations are highly variable in space and time. A key variability to evaluate within models that simulate aerosol is the vertical distribution, which influences atmospheric heating profiles and aerosol–cloud interactions, to help constrain aerosol residence time and to better represent the magnitude of simulated impacts. To ensure a consistent comparison between modeled and observed vertical distribution of aerosol, we implemented an aerosol lidar simulator within the Cloud Feedback Model Intercomparison Project (CFMIP) Observation Simulator Package version 2 (COSPv2). We assessed the attenuated total backscattered (ATB) signal and the backscatter ratios (SRs) at 532 nm in the U.S. Department of Energy's Energy Exascale Earth System Model version 1 (E3SMv1). The simulator performs the computations at the same vertical resolution as the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), making use of aerosol optics from the E3SMv1 model as inputs and assuming that aerosol is uniformly distributed horizontally within each model grid box. The simulator applies a cloud masking and an aerosol detection threshold to obtain the ATB and SR profiles that would be observed above clouds by CALIOP with its aerosol detection capability. Our analysis shows that the aerosol distribution simulated at a seasonal timescale is generally in good agreement with observations. Over the Southern Ocean, however, the model does not produce the SR maximum as observed in the real world. Comparison between clear-sky and all-sky SRs shows little differences, indicating that the cloud screening by potentially incorrect model clouds does not affect the mean aerosol signal averaged over a season. This indicates that the differences between observed and simulated SR values are due not to sampling errors, but to deficiencies in the representation of aerosol in models. Finally, we highlight the need for future applications of lidar observations at multiple wavelengths to provide insights into aerosol properties and distribution and their representation in Earth system models.

54 ENVIRONMENTAL SCIENCES↗

An evaluation of thematic mapper simulator data for mapping forest cover

Computer-aided analysis techniques applied to Thematic Mapper Simulator (TMS) data were evaluated for the purpose of mapping forest cover types. Classification results obtained using a supervised set of training statistics and various combinations of three and four channel subsets of the seven available TMS channels are compared for the L2 (Minimum Euclidean Distance), GML (Gaussian Maximum Likelihood), and SECHO (Supervised Extraction and Classification of Homogeneous Objects) classification algorithms. SECHO performed significantly better than either of the two per-point classifiers for the untransformed data. Overall classification results of the Karhunen-Loeve transformation increased for the L2 algorithm, but decreased for both the GML and SECHO algorithms.

Dean, M. E.↗