Search NASA⌕ Search

SEARCH · Search NASA

Results for “simulation evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An open-source framework for simulation-based testing of buildings control strategies

Here this paper presents a simulation framework for evaluating building control strategies (BCSs), developed with VOLTTRON, an opensource platform that integrates data, devices, and systems for sensing and control applications, and is based on co-simulation interfaces. It realizes an integrated environment for both testing and deploying BCSs, thereby eliminating the need for having a dedicated implementation of BCS for testing. For the first time, this framework provides critical functionalities, including scalable communication management and time-drive simulation advance, i.e., advancing the simulation based on clock time. We applied this framework to evaluating two BCSs from ASHRAE Guideline 36-2008 and ASHRAE Standard 90.1-2019. The evaluation results show that those BCSs primarily benefit the heating operation by reducing gas usage but yield insignificant savings in electricity consumption. The results also emphasize the importance of tuning the parameters of the BCSs to achieve better performances.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2006

Evaluating Clouds in Long-Term Cloud-Resolving Model Simulations with Observational Data

Two 20-day, continental midlatitude cases are simulated with a three-dimensional (3D) cloud-resolving model (CRM) and compared to Atmospheric Radiation Measurement (ARM) data. This evaluation of long-term cloud-resolving model simulations focuses on the evaluation of clouds and surface fluxes. All numerical experiments, as compared to observations, simulate surface precipitation well but over-predict clouds, especially in the upper troposphere. The sensitivity of cloud properties to dimensionality and other factors is studied to isolate the origins of the over prediction of clouds. Due to the difference in buoyancy damping between 2D and 3D models, surface precipitation fluctuates rapidly with time, and spurious dehumidification occurs near the tropopause in the 2D CRM. Surface fluxes from a land data assimilation system are compared with ARM observations. They are used in place of the ARM surface fluxes to test the sensitivity of simulated clouds to surface fluxes. Summertime simulations show that surface fluxes from the assimilation system bring about a better simulation of diurnal cloud variation in the lower troposphere.

Zeng, Xiping↗
NASA NTRS2008

Evaluation of Recent Upgrades to the NESS (Nuclear Engine System Simulation) Code

The Nuclear Thermal Rocket (NTR) concept is being evaluated as a potential propulsion technology for exploratory expeditions to the moon, Mars, and beyond. The need for exceptional propulsion system performance in these missions has been documented in numerous studies, and was the primary focus of a considerable effort undertaken during the Rover/NERVA program from 1955 to 1973. The NASA Glenn Research Center is leveraging this past NTR investment in their vehicle concepts and mission analysis studies with the aid of the Nuclear Engine System Simulation (NESS) code. This paper presents the additional capabilities and upgrades made to this code in order to perform higher fidelity NTR propulsion system analysis and design, and a comparison of its results to the Small Nuclear Rocket Engine (SNRE) design.

Fittje, James E.↗
DOE OSTI2021

Quantitative Evaluation of Autonomous Driving in CARLA

There has been a great deal of recent advancements in end-to-end imitation and reinforcement learning for self-driving vehicles. Despite this, there is a severe lack of standardized metrics for evaluating the performance of autonomous self-driving agents. Existing metrics are generally lacking in their ability to capture a wide range of driving behaviors and compare the severity of different failure cases. In this work, we introduce the Quantitative Evaluation for Driving metric, or QED, which assigns a quantitative score from 0-100 that captures the quality of driving for any driving agent. Our QED metric assesses different aspects of driving behavior including the ability to stay in the center of the lane, avoid weaving and erratic behavior, follow the speed limit, and avoid collisions, and it can be used under a wide range of driving scenarios. To show the effectiveness of our QED metric, we compare the scores generated by QED against scores assigned by human evaluators on a total of 30 different drivers and 6 different towns in the CARLA driving simulator. In ``easy'' evaluation scenarios, where it is relatively straightforward to distinguish better drivers from worse drivers, QED attains an average Pearson correlation of 0.96 and average Spearman correlation of 0.97 when compared against human evaluators. In ``hard'' evaluation scenarios, where it is far more ambiguous how to rank/score different types of bad driving behavior, QED attains an average Pearson correlation of 0.82 and average Spearman correlation of 0.75 when compared against human evaluators, which are both slighter higher than when we compare human evaluators against each other. While QED may not capture every characteristic that defines good driving, we consider it an important foundation for reproducibility and standardization in the community.

Gao, Shang↗
NASA NTRS1987

A comparison of position and rate control for telemanipulations with consideration of manipulator system dynamics

Position and rate control are the two common manual control modes in teleoperations. Human operator performance using the two modes is evaluated and compared. Simulated three-axis pick-and-place operations are used as the primary task for evaluation. First, ideal position and rate control are compared by considering several factors, such as joystick gain, joystick type, display mode, task, and manipulator work space size. Then the effects of the manipulator system dynamics are investigated by varying the natural frequency and speed limit. Experimental results show that ideal position control is superior to ideal rate control, regardless of joystick type or display mode, when the manipulation work space is small or comparable to the human operator's control space. Results also show that when the manipulator system is slow, the superiority of position control disappears. Position control is recommended for small-work-space telemanipulation tasks, while rate control is recommended for slow wide-work-space telemanipulation tasks.

Kim, Won S.↗
NASA NTRS1965

A Simulator Study of Take-Off Characteristics of Proposed Supersonic Transports

Fixed-cockpit piloted simulator studies of delta-planform and variable-wing-sweep supersonic transport configurations are being conducted at the Ames Research Center to investigate the handling qualities and certification requirements related to the take-off maneuver. Validation of the simulation was achieved by duplicating the take-off certification program of a subsonic jet transport. Evaluation of the simulator was made by NASA pilots as well as company and FAA pilots involved in the actual certification flights of the airplane. The present paper is limited to a discussion of normal take-off, minimum control speed (ground), rotation characteristics, and initial climbout. Comparisons of the take-off characteristics are made between the supersonic transport and the current class of subsonic jet transports. Results indicate that minimum control speed (ground) characteristics are a function of thrust-weight ratio, the time provided for SST rotation should be at least as long as that for the subsonic jet transports, abused take-offs are more likely to result in tail scrapes, and climbout below the minimum drag speed requires that the pilot carefully monitor airspeed.

Jackson, Charles T., Jr.↗
NASA NTRSDate not supplied

Verification and Validation of Elastodynamic Simulation Software for Aerospace Research

Physics-based simulation of nondestructive evaluation (NDE) inspection can help to advance the inspectability and reliability of mechanical systems. However, NDE simulations applicable to non-idealized mechanical components often require large compute domains and long run times. This has prompted development of custom NDE simulation software tailored to high performance computing (HPC) hardware. Verification and validation (V&V) is an integral part of developing this software to ensure implementations are robust and applicable to inspection problems, producing tools and simulations suitable for computational NDE research. This presentation addresses factors common to V&V of several elastodynamic simulation codes applicable to ultrasonic NDE. Examples are drawn from in-house simulation software at NASA Langley Research Center, ranging from ensuring reliability in a 1D heterogeneous media wave equation solver to the V&V needs of 3D cluster-parallel elastodynamic software. Factors specific to a research environment are addressed, where individual simulation results can be as relevant as the software product itself. Distinct facets of V&V are discussed including testing to establish software reliability, employing systematic approaches for consistency with fundamental conservation laws, establishing the numerical stability of algorithms, and demonstrating concurrence with empirical data. This talk also addresses V&V practices for small groups of researchers. This includes establishing resources (e.g. time and personnel) for V&V during project planning to mitigate and control the risk of setbacks. Similarly, we identify ways for individual researchers to use V&V during simulation software development itself to both speed up the development process and reduce incurred technical debt.

NDE↗
NASA NTRS2008

Lunar Dust Characterization Activity at GRC

The fidelity of lunar simulants as compared to actual regolith is evaluated using Figures of Merit (FOM) which are based on four criteria: Particle Size, Particle Shape, Composition, and Density of the bulk material. In practice, equipment testing will require other information about both the physical properties (mainly of the dust fraction) and composition as a function of particle size. At Glenn Research Center (GRC) we are involved in evaluating a number of simulant properties of consequence to testing of lunar equipment in a relevant environment, in order to meet Technology Readiness Level (TRL) 6 criteria. Bulk regolith has been characterized for many decades, but surprisingly little work has been done on the dust fraction (particles less than 20 micrometers in diameter). GRC is currently addressing the information shortfall by characterizing the following physical properties: Particle Size Distribution, Adhesion, Abrasivity, Surface Energy, Magnetic Susceptibility, Tribocharging and Surface Chemistry/Reactivity. Since some of these properties are also dependent on the size of the particles we have undertaken the construction of a six stage axial cyclone particle separator to fractionate dust into discrete particle size distributions for subsequent evaluation of these properties. An introduction to this work and progress to date will be presented.

Street, Kenneth W.↗
NASA NTRS2006

Software for Simulating Air Traffic

Future Air Traffic Management Concepts Evaluation Tool (FACET) is a system of software for performing computational simulations for evaluating advanced concepts of advanced air-traffic management. FACET includes a program that generates a graphical user interface plus programs and databases that implement computational models of weather, airspace, airports, navigation aids, aircraft performance, and aircraft trajectories. Examples of concepts studied by use of FACET include aircraft self-separation for free flight; prediction of air-traffic-controller workload; decision support for direct routing; integration of spacecraft-launch operations into the U.S. national airspace system; and traffic- flow-management using rerouting, metering, and ground delays. Aircraft can be modeled as flying along either flight-plan routes or great-circle routes as they climb, cruise, and descend according to their individual performance models. The FACET software is modular and is written in the Java and C programming languages. The architecture of FACET strikes a balance between flexibility and fidelity; as a consequence, FACET can be used to model systemwide airspace operations over the contiguous U.S., involving as many as 10,000 aircraft, all on a single desktop or laptop computer running any of a variety of operating systems. Two notable applications of FACET include: (1) reroute conformance monitoring algorithms that have been implemented in one of the Federal Aviation Administration s nationally deployed, real-time, operational systems; and (2) the licensing and integration of FACET with the commercially available Flight Explorer, which is an Internet- based, real-time flight-tracking system.

Sridhar, Banavar↗
DOE OSTI2016

TU-FG-BRB-07: GPU-Based Prompt Gamma Ray Imaging From Boron Neutron Capture Therapy

Purpose: The purpose of this research is to perform the fast reconstruction of a prompt gamma ray image using a graphics processing unit (GPU) computation from boron neutron capture therapy (BNCT) simulations. Methods: To evaluate the accuracy of the reconstructed image, a phantom including four boron uptake regions (BURs) was used in the simulation. After the Monte Carlo simulation of the BNCT, the modified ordered subset expectation maximization reconstruction algorithm using the GPU computation was used to reconstruct the images with fewer projections. The computation times for image reconstruction were compared between the GPU and the central processing unit (CPU). Also, the accuracy of the reconstructed image was evaluated by a receiver operating characteristic (ROC) curve analysis. Results: The image reconstruction time using the GPU was 196 times faster than the conventional reconstruction time using the CPU. For the four BURs, the area under curve values from the ROC curve were 0.6726 (A-region), 0.6890 (B-region), 0.7384 (C-region), and 0.8009 (D-region). Conclusion: The tomographic image using the prompt gamma ray event from the BNCT simulation was acquired using the GPU computation in order to perform a fast reconstruction during treatment. The authors verified the feasibility of the prompt gamma ray reconstruction using the GPU computation for BNCT simulations.

60 APPLIED LIFE SCIENCES↗
NASA NTRSDate not supplied

Numerical Behaviour of a Smooth Local Correlation-based Transition Model in a Newton-Krylov Flow Solver

The numerical behaviour of transport-equation-based transition models, including both iterative and grid convergence, is influenced by the source terms. Transition models contain source terms that are large and highly nonlinear, and can be destabilizing in a strong implicit solver. Linearization strategies with varying levels of coupling are evaluated in conjunction with a source-term time step restriction to determine best-practices for solving the SA-sLM2015smooth local correlation-based transition model in an implicit Newton-Krylov flow solver. Achieving deep iterative convergence facilitates a detailed investigation of the grid convergence of these free-transition simulations, which are evaluated relative to fully-turbulent simulations performed using the Spalart-Allmaras turbulence model. Simulations of the NLF0416 general aviation airfoil, VA-2 supercritical airfoil, and NASA CRM-NLF wing-body geometry are performed over a range of grid levels. The results demonstrate that both a fully-coupled linearization strategy and a source-term time step restriction improve nonlinear convergence as the complexity of the free-transition simulations increases. In general, additional grid resolution is required for free-transition simulations relative to fully-turbulent simulations in order to achieve a similar level of accuracy, with the grid convergence of free-transition simulations sensitive to the streamwise grid spacings in the transition regions.

AATT↗
NASA NTRS2012

Simulation and Flight Test Capability for Testing Prototype Sense and Avoid System Elements

NASA Langley Research Center (LaRC) and The MITRE Corporation (MITRE) have developed, and successfully demonstrated, an integrated simulation-to-flight capability for evaluating sense and avoid (SAA) system elements. This integrated capability consists of a MITRE developed fast-time computer simulation for evaluating SAA algorithms, and a NASA LaRC surrogate unmanned aircraft system (UAS) equipped to support hardware and software in-the-loop evaluation of SAA system elements (e.g., algorithms, sensors, architecture, communications, autonomous systems), concepts, and procedures. The fast-time computer simulation subjects algorithms to simulated flight encounters/ conditions and generates a fitness report that records strengths, weaknesses, and overall performance. Reviewed algorithms (and their fitness report) are then transferred to NASA LaRC where additional (joint) airworthiness evaluations are performed on the candidate SAA system-element configurations, concepts, and/or procedures of interest; software and hardware components are integrated into the Surrogate UAS research systems; and flight safety and mission planning activities are completed. Onboard the Surrogate UAS, candidate SAA system element configurations, concepts, and/or procedures are subjected to flight evaluations and in-flight performance is monitored. The Surrogate UAS, which can be controlled remotely via generic Ground Station uplink or automatically via onboard systems, operates with a NASA Safety Pilot/Pilot in Command onboard to permit safe operations in mixed airspace with manned aircraft. An end-to-end demonstration of a typical application of the capability was performed in non-exclusionary airspace in October 2011; additional research, development, flight testing, and evaluation efforts using this integrated capability are planned throughout fiscal year 2012 and 2013.

Howell, Charles T.↗
NASA NTRS1972

Nonlinear estimation theory applied to the interplanetary orbit determination problem.

Martingale theory and appropriate smoothing properties of Loeve (1953) have been used to develop a modified Gaussian second-order filter. The performance of the filter is evaluated through numerical simulation of a Jupiter flyby mission. The observations used in the simulation are on-board measurements of the angle between Jupiter and a fixed star taken at discrete time intervals. In the numerical study, the influence of each of the second-order terms is evaluated. Five filter algorithms are used in the simulations. Four of the filters are the modified Gaussian second-order filter and three approximations derived by neglecting one or more of the second-order terms in the equations. The fifth filter is the extended Kalman-Bucy filter which is obtained by neglecting all of the second-order terms.

Tapley, B. D.↗
DOE OSTI2026

DetSuM: Detector Surrogate Model for LArTPC

In large neutrino experiments such as the Deep Underground Neutrino Experiment (DUNE), estimating detector response uncertainties typically requires simulation samples that consume substantial computing resources and time. To mitigate this challenge, we present DetSuM, an uncertainty-aware surrogate model designed to capture the detector response variations with reduced computing load compared to full simulations. This poster describes the construction and evaluation of DetSuM using simulation and reconstruction datasets in a rare-event search at DUNE. We assess DetSuM's ability to predict key detector-response variations and their associated uncertainties, discuss current limitations, and outline improvements to extend its validity in systematics studies of DUNE physics.

Li, Aobo [UC, San Diego]↗
NASA NTRS1972

The development of an inert simulant for HNS/teflon explosive

The report describes the development and evaluation of an inert simulant for the thermally stable, heat-resistant plastic-bonded explosive HNS/Teflon. The simulant is made by dry blending vinylidene fluoride, melamine and Teflon which when compared has a pressed density and thermal properties corresponding closely to the explosive. In addition, the machinability and handling characteristics of the simulant are similar to the explosive.

Elban, W. L.↗
NASA NTRS1999

A Further Look at X-33 Entry Guidance and Beyond

An entry guidance design developed at Iowa State University (ISU) for the X-33 advanced technology demonstrator is outlined and compared with the X-33 entry guidance algorithms developed at NASA Marshall. Both designs are based on the Space Shuttle entry guidance concept, but significant improvements have been made to enhance the performance and reduce the complexity. The ISU design was incorporated into MAVERIC, a high fidelity vehicle simulation software for the X-33, and evaluated in Monte Carlo simulations against random dispersions in propulsion system, wind and atmospheric properties, aerodynamic coefficients, interaction between propulsion and aerodynamics, and navigation data. The simulations clearly demonstrated the capability and precision of the ISU entry guidance design in successfully guiding the X-33 in some rather difficult flight scenarios. As the entry guidance development for the X-33 is completing, this report also offers some review of the strength and limitations of the current Shuttle-based entry guidance framework, and finally some potential candidates for next generation of more capable and cost-effective entry guidance designs are discussed.

Lu, Ping↗
NASA NTRS2002

Simulating Snow Over Sea Ice In Climate Models

We have evaluated two methods of simulating the seasonal cycle of snow over sea ice in and around the Arctic: The NCAR global climate model CCM3, with its standard snow hydrology, and the snow pack model SNTHERM, forced with hourly atmospheric output from CCM3. A new dataset providing dates for the onset of snow melt over Arctic sea ice provides a means for assessing basin-wide how well the models simulate melt onset, but contains no information on how long it then takes for all the snow to melt. Use of data from the SHEBA site provides very detailed information on the behavior of the snow before and during the melt season, but only for a very limited area. Russian drift data provide climatological data on the seasonal cycle of snow water equivalent and snow density, over multi-year sea ice in the central Arctic basin. These datasets are used to compare the two modeling methods, and to see if use of the more physically-realistic SNTHERM provides any significant improvements. Conclusions obtained so far include: 1. Both CCM3 and CCM3/SNTHERM do a good job overall of matching the onset of snow melt dataset; although CCM3/SNTHERM consistently trends to underestimate the date and CCM3 to overestimate it. 2. SHEBA and ice drift data for the Arctic show that CCM3/ SNTHERM does a better job than CCM3 at simulating the total melt period. 3. Ice drift snow density and accumulation data suggest that while providing superior results, CCM3/SNTHERM may still suffer from overly vigorous melting. 4. Both the large-scale atmospheric forcing and snow pack physical processes are important in proper simulation of the snow seasonal cycle. Ongoing work includes further diagnosis of CCM3/SNTHERM, use of more observational datasets, especially from marginal seas in the pan-Arctic, and full coupling of SNTHERM into CCM3 (work to date has all been off-line simulations).

Arnold, James E.↗
NASA NTRS2005

The Small Aircraft Transportation System Higher Volume Operations (SATS HVO) Flight Experiment

This paper provides a summary of conclusions from the Small Aircraft Transportation System (SATS) Higher Volume Operations (HVO) Flight Experiment which NASA conducted to determine pilot acceptability of the HVO concept for normal conditions. The SATS HVO concept improves efficiency at non-towered, non-radar airports in Instrument Meteorological Conditions (IMC) while achieving a level of safety equal to today s system. Reported are results from flight experiment data that indicate that the SATS HVO concept is viable. The success of the SATS HVO concept is based on acceptable pilot workload, performance, and subjective criteria when compared to the procedural control operations in use today at non-towered, non-radar controlled airfields in IMC. The HVO Flight Experiment, flown on NASA's Cirrus SR22, used a subset of the HVO Simulation Experiment scenarios and evaluation pilots in order to validate the simulation experiment results. HVO and Baseline (today s system) scenarios flown included: single aircraft arriving for a GPS non-precision approach; aircraft arriving for the approach with multiple traffic aircraft; and aircraft arriving for the approach with multiple traffic aircraft and then conducting a missed approach. Results reveal that all twelve low-time instrument-rated pilots preferred SATS HVO when compared to current procedural separation operations. These pilots also flew the HVO procedures safely and proficiently without additional workload in comparison to today s system (Baseline). Detailed results of pilot flight technical error, and their subjective assessments of workload and situation awareness are presented in this paper.

Williams, Daniel M.↗