Search NASA⌕ Search

SEARCH · Search NASA

Results for “tool evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Risk Evaluation in the Pre-Phase A Conceptual Design of Spacecraft

Typically, the most important decisions in the design of a spacecraft are made in the earliest stages of its conceptual design the Pre-Phase A stages. It is in these stages that the greatest number of design alternatives is considered, and the greatest number of alternatives is rejected. The focus of Pre-Phase A conceptual development is on the evaluation and comparison of whole concepts and the larger-scale systems comprising those concepts. This comparison typically uses general Figures of Merit (FOMs) to quantify the comparative benefits of designs and alternative design features. Along with mass, performance, and cost, risk should be one of the major FOMs in evaluating design decisions during the conceptual design phases. However, risk is often given inadequate consideration in conceptual design practice. The reasons frequently given for this lack of attention to risk include: inadequate mission definition, lack of rigorous design requirements in early concept phases, lack of fidelity in risk assessment methods, and under-evaluation of risk as a viable FOM for design evaluation. In this paper, the role of risk evaluation in early conceptual design is discussed. The various requirements of a viable risk evaluation tool at the Pre-Phase A level are considered in light of the needs of a typical spacecraft design study. A technique for risk identification and evaluation is presented. The application of the risk identification and evaluation approach to the conceptual design process is discussed. Finally, a computational tool for risk profiling is presented and applied to assess the risk for an existing Pre-Phase A proposal. The resulting profile is compared to the risks identified for the proposal by other means.

Fabisinski, Leo L., III↗

Portability Support for High Performance Computing

While a large number of tools have been developed to support application portability, high performance application developers often prefer to use vendor-provided, non-portable programming interfaces. This phenomena indicates the mismatch between user priorities and tool capabilities. This paper summarizes the results of a user survey and a developer survey. The user survey has revealed the user priorities and resulted in three criteria for evaluating tool support for portability. The developer survey has resulted in the evaluation of portability support and indicated the possibilities and difficulties of improvements.

Cheng, Doreen Y.↗

Using computer graphics to enhance astronaut and systems safety

Computer graphics is being employed at the NASA Johnson Space Center as a tool to perform rapid, efficient and economical analyses for man-machine integration, flight operations development and systems engineering. The Operator Station Design System (OSDS), a computer-based facility featuring a highly flexible and versatile interactive software package, PLAID, is described. This unique evaluation tool, with its expanding data base of Space Shuttle elements, various payloads, experiments, crew equipment and man models, supports a multitude of technical evaluations, including spacecraft and workstation layout, definition of astronaut visual access, flight techniques development, cargo integration and crew training. As OSDS is being applied to the Space Shuttle, Orbiter payloads (including the European Space Agency's Spacelab) and future space vehicles and stations, astronaut and systems safety are being enhanced. Typical OSDS examples are presented. By performing physical and operational evaluations during early conceptual phases. supporting systems verification for flight readiness, and applying its capabilities to real-time mission support, the OSDS provides the wherewithal to satisfy a growing need of the current and future space programs for efficient, economical analyses.

Computer Simulation↗

Evaluation of User Experience of Self-Scheduling Software for Astronauts: Defining a Satisfaction Baseline

As NASA turns its sights to deep-space exploration, a greater focus on sup-porting crew autonomy has led to the development of Playbook, a self-scheduling software tool. Evaluating the user satisfaction of Playbook is essential in ensuring its usability for critical spaceflight operations. Satisfaction of an interface is often quantified with attitude surveys, such as the User Experience Questionnaire (UEQ). This paper demonstrates an application of the UEQ in comparing the user experience of Playbook interface de-signs for displaying graphical data. We lay the foundation for future user experience comparisons by defining a satisfaction baseline, which is crucial as more features are integrated into Playbook’s interface. This work ex-tends a validated user experience framework into a spaceflight domain, allowing optimization of human-computer interaction as future operational tools are developed.

user experience↗

The smart highway project: Smart highways, smart vehicles, smart engineering

The Smart Highway project is a six mile, limited access roadway being built between Interstate 81 and Blacksburg, Virginia. The initial construction segment will be two miles long and is designed to serve as a test bed and test track for Intelligent Transportation Systems (ITS) research. The Center for Transportation Research (CTR) at Virginia Tech is developing three evaluation tools for its ITS research including DYNAVIMTS (a software framework), and the FLASH Lab (a 1/15th scale model highway and vehicle system). The Smart Highway rounds out the Center's evaluation methodology by allowing full scale operational tests, evaluations, and research under both experimental and conventional traffic conditions. Currently under development is a concept for a fully automated highway using a 'Cooperative Infrastructure Managed System' which involves ultra wide band communication beacons installed in the infrastructure with appropriate sensors, receivers and processors on board the vehicles. The project is part of the research program funded by the National Automated Highway System Consortium. The CTR hopes to develop the automated concept to prototype status by 1997. Other smart transportation and smart engineering concepts are proposed. This presentation will address the goals and objectives of the Smart Highway project, overview its status and importance to the region, and identify some of the transportation technology now under development and planned in the future.

Pethtel, Ray D.↗

Stage Separation CFD Tool Development and Evaluation

This viewgraph presentation evaluates CFD (Computational Fluid Dynamics) tools for solving stage separation problems. The demonstration and validation of the tools is for a second generation RLV (Reusable Launch Vehicle) stage separation. The flow solvers are: Cart3D; Overflow/Overflow-D; Unic.

Droege, Alan↗

Space station thermal control surfaces

Mission planning documents were used to analyze the radiator design and thermal control surface requirements for both space station and 25-kW power module, to analyze the missions, and to determine the thermal control technology needed to satisfy both sets of requirements. Parameters such as thermal control coating degradation, vehicle attitude, self eclipsing, variation in solar constant, albedo, and Earth emission are considered. Four computer programs were developed which provide a preliminary design and evaluation tool for active radiator systems in LEO and GEO. Two programs were developed as general programs for space station analysis. Both types of programs find the radiator-flow solution and evaluate external heat loads in the same way. Fortran listings are included.

Maag, C. R.↗

Green-Field Geothermal/Solar Designs (Cooperative Research and Development Final Report)

Geothermal power (GEO) and concentrating solar power (CSP) can be integrated to obtain the best aspects of both systems. The green-field design will utilize the full potential of the higher-temperature CSP sector by having a topping solar steam turbine and a bottoming geothermal cycle. In the proposed hybrid cycle, geothermal energy can be converted into electricity at a higher cycle efficiency because of the high-temperature topping cycle during the daytime. CSP benefits from lower-cost solar collectors, a less expensive topping turbine, and virtual elimination of the heat rejection system cost. The green-field case study will be focused on a geothermal resource site which will deliver the 160 - 220 °C pressurized brine at the inlet of power block. While the case study will look at a single case study, the tools to be developed as part of the project will be available to model annual performance of GEO/CSP hybrids for new greenfield projects (may also be applied to certain types of brown-field retrofits). The evaluation tools will also calculate important economic parameters such as the levelized cost of energy (LCOE) and the internal rate of return (IRR).

14 SOLAR ENERGY↗

Environmentally Friendly Rotorcraft Concepts: 2011-2014: US/France MoA Helicopter Aeromechanics - Task 5

Motivation: Greenhouse gas concentrations have increased well beyond their pre-industrial levels due to human activities; Aircraft emissions are becoming regulated in industrialized nations, so future rotorcraft will need to be designed for minimal environmental impact. Objectives: Determine suitable metrics to measure environmental impact of rotorcraft, focusing first on air pollution; Integrate performance metrics with rotorcraft design codes; Generate rotorcraft designs with minimal environmental impact. Outcomes: Obtain an understanding of how targeting reduced emissions affects the rotorcraft design process; Advance the capability of existing rotorcraft design tools to include environmental performance metrics. Tools: Evaluation and predesign - NDARC (NASA Design and Analysis of Rotorcraft) and CREATION (Concepts of Rotorcraft Enhanced Assessment Through Integrated Optimization Network); Flight simulation - CAMRAD II (Comprehensive Analytical Model of Rotorcraft Aerodynamics and Dynamics) and HOST (Helicopter Overall Simulation Tool).

Environmentally Friendly↗

Satellite Systems Design/Simulation Environment: A Systems Approach to Pre-Phase A Design

A toolset for the rapid development of small satellite systems has been created. The objective of this tool is to support the definition of spacecraft mission concepts to satisfy a given set of mission and instrument requirements. The objective of this report is to provide an introduction to understanding and using the SMALLSAT Model. SMALLSAT is a computer-aided Phase A design and technology evaluation tool for small satellites. SMALLSAT enables satellite designers, mission planners, and technology program managers to observe the likely consequences of their decisions in terms of satellite configuration, non-recurring and recurring cost, and mission life cycle costs and availability statistics. It was developed by Princeton Synergetic, Inc. and User Systems, Inc. as a revision of the previous TECHSAT Phase A design tool, which modeled medium-sized Earth observation satellites. Both TECHSAT and SMALLSAT were developed for NASA.

Ferebee, Melvin J., Jr.↗

Development and Application of Computational/In Vitro Toxicological Methods for Chemical Hazard Risk Reduction of New Materials for Advanced Weapon Systems

The development of quantitative structure-activity relationship (QSAR) is essential for reducing the chemical hazards of new weapon systems. The current collaboration between HEST (toxicology research and testing), MLPJ (computational chemistry) and PRS (computational chemistry, new propellant synthesis) is focusing R&D efforts on basic research goals that will rapidly transition to useful products for propellant development. Computational methods are being investigated that will assist in forecasting cellular toxicological end-points. Models developed from these chemical structure-toxicity relationships are useful for the prediction of the toxicological endpoints of new related compounds. Research is focusing on the evaluation tools to be used for the discovery of such relationships and the development of models of the mechanisms of action. Combinations of computational chemistry techniques, in vitro toxicity methods, and statistical correlations, will be employed to develop and explore potential predictive relationships; results for series of molecular systems that demonstrate the viability of this approach are reported. A number of hydrazine salts have been synthesized for evaluation. Computational chemistry methods are being used to elucidate the mechanism of action of these salts. Toxicity endpoints such as viability (LDH) and changes in enzyme activity (glutahoione peroxidase and catalase) are being experimentally measured as indicators of cellular damage. Extrapolation from computational/in vitro studies to human toxicity, is the ultimate goal. The product of this program will be a predictive tool to assist in the development of new, less toxic propellants.

Frazier, John M.↗

Development and validation of methods for man-made machine interface evaluation

The alternate methods of conducting a man-machine interface evaluation are classified as static and dynamic, and are evaluated. A dynamic evaluation tool is presented to provide for a determination of the effectiveness of the man-machine interface in terms of the sequence of operations (task and task sequences) and in terms of the physical characteristics of the interface. This dynamic checklist approach is recommended for shuttle and shuttle payload man-machine interface evaluations based on reduced preparation time, reduced data, and increased sensitivity of critical problems.

Malone, T. B.↗

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗

Retrofit Energy Analysis and Central Thermal modeling (REACT) v1.0

REACT is a website designed to simplify the analysis and decision-making process for retrofitting existing central plant heating and cooling systems with advanced heat pump technologies. The tool evaluates the technical and economic viability of replacing traditional boilers with various options including water-to-water or air-to-water heat pumps, which can provide efficient and lower-cost heating and cooling. It allows users to compare current central plant configurations with retrofit scenarios, assessing energy consumption, life-cycle costs, and environmental impact. Retrofitting traditional boiler and chiller systems with water-to-water or air-to-water heat pumps can significantly reduce energy consumption and lower lifecycle costs. The REACT provides: User-Friendly Tools: A user friendly web interface for quick, intuitive analysis accessible to non-experts. Advanced Modeling: A Python-powered engine for detailed parametric studies, optimization, and research applications. Comprehensive Analysis: Lifecycle cost evaluation, energy consumption modeling, and environmental impact assessment. Visual Insights: A variety of plots to visualize system performance and design trade-offs. The engine for the website (REACT) bases on several Python libraries, and the website will be hosted on an ETA server.

Kim, Donghun [Lawrence Berkeley National Laborator↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavordsky, Bradley↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavodsky, Bradley↗

Global Simulation of Aviation Operations

The simulation and analysis of global air traffic is limited due to a lack of simulation tools and the difficulty in accessing data sources. This paper provides a global simulation of aviation operations combining flight plans and real air traffic data with historical commercial city-pair aircraft type and schedule data and global atmospheric data. The resulting capability extends the simulation and optimization functions of NASA's Future Air Traffic Management Concept Evaluation Tool (FACET) to global scale. This new capability is used to present results on the evolution of global air traffic patterns from a concentration of traffic inside US, Europe and across the Atlantic Ocean to a more diverse traffic pattern across the globe with accelerated growth in Asia, Australia, Africa and South America. The simulation analyzes seasonal variation in the long-haul wind-optimal traffic patterns in six major regions of the world and provides potential time-savings of wind-optimal routes compared with either great circle routes or current flight-plans if available.

simulation↗

Evaluating Machine Learning-Based MRI Reconstruction Using Digital Image Quality Phantoms

Quantitative and objective evaluation tools are essential for assessing the performance of machine learning (ML)-based magnetic resonance imaging (MRI) reconstruction methods. However, the commonly used fidelity metrics, such as mean squared error (MSE), structural similarity (SSIM), and peak signal-to-noise ratio (PSNR), often fail to capture fundamental and clinically relevant MR image quality aspects. To address this, we propose evaluation of ML-based MRI reconstruction using digital image quality phantoms and automated evaluation methods. Our phantoms are based upon the American College of Radiology (ACR) large physical phantom but created in k-space to simulate their MR images, and they can vary in object size, signal-to-noise ratio, resolution, and image contrast. Our evaluation pipeline incorporates evaluation metrics of geometric accuracy, intensity uniformity, percentage ghosting, sharpness, signal-to-noise ratio, resolution, and low-contrast detectability. We demonstrate the utility of our proposed pipeline by assessing an example ML-based reconstruction model across various training and testing scenarios. The performance results indicate that training data acquired with a lower undersampling factor and coils of larger anatomical coverage yield a better performing model. The comprehensive and standardized pipeline introduced in this study can help to facilitate a better understanding of the performance and guide future development and advancement of ML-based reconstruction algorithms.

47 OTHER INSTRUMENTATION↗