SEARCH · Search NASA

Results for “User evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2020

How Do Visual Explanations Foster End Users' Appropriate Trust in Machine Learning?

We investigated the effects of different visual explanations on users' trust in machine learning classification. We proposed three forms of visual explanations of a classification based on identifying relevant training instances. We conducted a user study to evaluate these visual explanations as well as a no explanation condition. We measured users' trust of a classifier, quantified the effects of these three forms of explanations, and assessed the changes in users' trust. We found that participants trust a classifier appropriately when an explanation is available. The combination of human, classification algorithm and understandable explanation makes better decisions than the classifier and human alone.This work advances the state-of-the-art closer to building trust-able machine learning models and informs the design and appropriate use of automated systems.

Yang, Fumeng↗

NASA NTRS1994

Rule based design of conceptual models for formative evaluation

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool, (2) a low fidelity simulator development tool, (3) a dynamic, interactive interface between the HCI and the simulator, and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Moore, Loretta A.↗

NASA NTRS1994

Rule based design of conceptual models for formative evaluation

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool; (2) a low fidelity simulator development tool; (3) a dynamic, interactive interface between the HCI and the simulator; and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Moore, Loretta A.↗

NASA NTRS1992

Advanced communications technology satellite high burst rate link evaluation terminal experiment control and monitor software user's guide, version 1.0

The Experiment Control and Monitor (EC&M) software was developed at NASA Lewis Research Center to support the Advanced Communications Technology Satellite (ACTS) High Burst Rate Link Evaluation Terminal (HBR-LET). The HBR-LET is an experimenter's terminal to communicate with the ACTS for various investigations by government agencies, universities, and industry. The EC&M software is one segment of the Control and Performance Monitoring (C&PM) software system of the HBR-LET. The EC&M software allows users to initialize, control, and monitor the instrumentation within the HBR-LET using a predefined sequence of commands. Besides instrument control, the C&PM software system is also responsible for computer communication between the HBR-LET and the ACTS NASA Ground Station and for uplink power control of the HBR-LET to demonstrate power augmentation during rain fade events. The EC&M Software User's Guide, Version 1.0 (NASA-CR-189160) outlines the commands required to install and operate the EC&M software. Input and output file descriptions, operator commands, and error recovery procedures are discussed in the document.

Reinhart, Richard C.↗

NASA NTRS1991

Space Communications Artificial Intelligence for Link Evaluation Terminal (SCAILET)

A software application to assis end-users of the Link Evaluation Terminal (LET) for satellite communication is being developed. This software application incorporates artificial intelligence (AI) techniques and will be deployed as an interface to LET. The high burst rate (HBR) LET provides 30 GHz transmitting/20 GHz receiving, 220/110 Mbps capability for wideband communications technology experiments with the Advanced Communications Technology Satellite (ACTS). The HBR LET and ACTS are being developed at the NASA Lewis Research Center. The HBR LET can monitor and evaluate the integrity of the HBR communications uplink and downlink to the ACTS satellite. The uplink HBR transmission is performed by bursting the bit-pattern as a modulated signal to the satellite. By comparing the transmitted bit pattern with the received bit pattern, HBR LET can determine the bit error rate BER) under various atmospheric conditions. An algorithm for power augmentation is applied to enhance the system's BER performance at reduced signal strength caused by adverse conditions. Programming scripts, defined by the design engineer, set up the HBR LET terminal by programming subsystem devices through IEEE488 interfaces. However, the scripts are difficult to use, require a steep learning curve, are cryptic, and are hard to maintain. The combination of the learning curve and the complexities involved with editing the script files may discourage end-users from utilizing the full capabilities of the HBR LET system. An intelligent assistant component of SCAILET that addresses critical end-user needs in the programming of the HBR LET system as anticipated by its developers is described. A close look is taken at the various steps involved in writing ECM software for a C&P, computer and at how the intelligent assistant improves the HBR LET system and enhances the end-user's ability to perform the experiments.

Shahidi, Anoosh↗

NASA NTRS1999

Prototype and Evaluation of AutoHelp: A Case-based, Web-accessible Help Desk System for EOSDIS

AutoHelp is a case-based, Web-accessible help desk for users of the EOSDIS. Its uses a combination of advanced computer and Web technologies, knowledge-based systems tools, and cognitive engineering to offload the current, person-intensive, help desk facilities at the DAACs. As a case-based system, AutoHelp starts with an organized database of previous help requests (questions and answers) indexed by a hierarchical category structure that facilitates recognition by persons seeking assistance. As an initial proof-of-concept demonstration, a month of email help requests to the Goddard DAAC were analyzed and partially organized into help request cases. These cases were then categorized to create a preliminary case indexing system, or category structure. This category structure allows potential users to identify or recognize categories of questions, responses, and sample cases similar to their needs. Year one of this research project focused on the development of a technology demonstration. User assistance 'cases' are stored in an Oracle database in a combination of tables linking prototypical questions with responses and detailed examples from the email help requests analyzed to date. When a potential user accesses the AutoHelp system, a Web server provides a Java applet that displays the category structure of the help case base organized by the needs of previous users. When the user identifies or requests a particular type of assistance, the applet uses Java database connectivity (JDBC) software to access the database and extract the relevant cases. The demonstration will include an on-line presentation of how AutoHelp is currently structured. We will show how a user might request assistance via the Web interface and how the AutoHelp case base provides assistance. The presentation will describe the DAAC data collection, case definition, and organization to date, as well as the AutoHelp architecture. It will conclude with the year 2 proposal to more fully develop the case base, the user interface (including the category structure), interface with the current DAAC Help System, the development of tools to add new cases, and user testing and evaluation at (perhaps) the Goddard DAAC.

Mitchell, Christine M.↗

NASA NTRS2016

Considerations for the Use of Remote Gaze Tracking to Assess Behavior in Flight Simulators

Complex user interfaces (such as those found in an aircraft cockpit) may be designed from first principles, but inevitably must be evaluated with real users. User gaze data can provide valuable information that can help to interpret other actions that change the state of the system. However, care must be taken to ensure that any conclusions drawn from gaze data are well supported. Through a combination of empirical and simulated data, we identify several considerations and potential pitfalls when measuring gaze behavior in high-fidelity simulators. We show that physical layout, behavioral differences, and noise levels can all substantially alter the quality of fit for algorithms that segment gaze measurements into individual fixations. We provide guidelines to help investigators ensure that conclusions drawn from gaze tracking data are not artifactual consequences of data quality or analysis techniques.

eye tracking↗

NASA NTRS1991

An overview of Space Communication Artificial Intelligence for Link Evaluation Terminal (SCAILET) Project

A software application to assist end-users of the link evaluation terminal (LET) for satellite communications is being developed. This software application incorporates artificial intelligence (AI) techniques and will be deployed as an interface to LET. The high burst rate (HBR) LET provides 30 GHz transmitting/20 GHz receiving (220/110 Mbps) capability for wideband communications technology experiments with the Advanced Communications Technology Satellite (ACTS). The HBR LET can monitor and evaluate the integrity of the HBR communications uplink and downlink to the ACTS satellite. The uplink HBR transmission is performed by bursting the bit-pattern as a modulated signal to the satellite. The HBR LET can determine the bit error rate (BER) under various atmospheric conditions by comparing the transmitted bit pattern with the received bit pattern. An algorithm for power augmentation will be applied to enhance the system's BER performance at reduced signal strength caused by adverse conditions.

Shahidi, Anoosh K.↗

NASA NTRS2013

Aircraft Engine Gas Path Diagnostic Methods: Public Benchmarking Results

Recent technology reviews have identified the need for objective assessments of aircraft engine health management (EHM) technologies. To help address this issue, a gas path diagnostic benchmark problem has been created and made publicly available. This software tool, referred to as the Propulsion Diagnostic Method Evaluation Strategy (ProDiMES), has been constructed based on feedback provided by the aircraft EHM community. It provides a standard benchmark problem enabling users to develop, evaluate and compare diagnostic methods. This paper will present an overview of ProDiMES along with a description of four gas path diagnostic methods developed and applied to the problem. These methods, which include analytical and empirical diagnostic techniques, will be described and associated blind-test-case metric results will be presented and compared. Lessons learned along with recommendations for improving the public benchmarking processes will also be presented and discussed.

Simon, Donald L.↗

NASA NTRS2013

Elastic-Plastic J-Integral Solutions or Surface Cracks in Tension Using an Interpolation Methodology

No closed form solutions exist for the elastic-plastic J-integral for surface cracks due to the nonlinear, three-dimensional nature of the problem. Traditionally, each surface crack must be analyzed with a unique and time-consuming nonlinear finite element analysis. To overcome this shortcoming, the authors have developed and analyzed an array of 600 3D nonlinear finite element models for surface cracks in flat plates under tension loading. The solution space covers a wide range of crack shapes and depths (shape: 0.2 less than or equal to a/c less than or equal to 1, depth: 0.2 less than or equal to a/B less than or equal to 0.8) and material flow properties (elastic modulus-to-yield ratio: 100 less than or equal to E/ys less than or equal to 1,000, and hardening: 3 less than or equal to n less than or equal to 20). The authors have developed a methodology for interpolating between the goemetric and material property variables that allows the user to reliably evaluate the full elastic-plastic J-integral and force versus crack mouth opening displacement solution; thus, a solution can be obtained very rapidly by users without elastic-plastic fracture mechanics modeling experience. Complete solutions for the 600 models and 25 additional benchmark models are provided in tabular format.

Allen, P. A.↗

NASA NTRS2013

Elastic-Plastic J-Integral Solutions or Surface Cracks in Tension Using an Interpolation Methodology. Appendix C -- Finite Element Models Solution Database File, Appendix D -- Benchmark Finite Element Models Solution Database File

Allen, Phillip A.↗

DOE OSTI2021

REopt Lite User Manual

REopt Lite evaluates the economic viability of grid-connected solar photovoltaics, wind, combined heat and power, and electric and thermal storage at commercial and small industrial sites. It allows building owners to identify the system sizes and dispatch strategies that minimize the site’s life cycle cost of energy. REopt Lite also estimates the amount of time on-site generation and storage can sustain the site's critical load during a grid outage and allows the user the choice of optimizing for energy resilience. It is primarily used to inform project development decisions and to support research on the factors that drive project feasibility for market development and policy analysis. It is available through a web interface, application programing interface, and open-source code. This user manual provides an overview of the model, including its capabilities and typical applications; inputs and outputs; economic calculations; technology descriptions; and model parameters, variables, and equations. The model is highly flexible and is continually evolving to meet the needs of each analysis. Therefore, this report is not an exhaustive description of all capabilities, but rather a summary of the core components of the model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1987

User interface devices for mission control

The Mission Control Center (MCC) at Johnson Space Center (JSC) in Houston, Texas, is being upgraded with new technology engineering/scientific workstations. These workstations will replace the existing consoles and will emulate the present hardware input and display media. The workstations will be using new and different input devices for the flight controller to interact with the workstation and mainframes. This paper presents the results of the User Interface survey conducted by the Workstation Prototype Lab (WPL). The WPL offered the opportunity for users to do hands-on evaluations of a number of user interface options prototyped by lab personnel.

Boatman, Wayne↗

NASA NTRS1986

NASA PC software evaluation project

The USL NASA PC software evaluation project is intended to provide a structured framework for facilitating the development of quality NASA PC software products. The project will assist NASA PC development staff to understand the characteristics and functions of NASA PC software products. Based on the results of the project teams' evaluations and recommendations, users can judge the reliability, usability, acceptability, maintainability and customizability of all the PC software products. The objective here is to provide initial, high-level specifications and guidelines for NASA PC software evaluation. The primary tasks to be addressed in this project are as follows: to gain a strong understanding of what software evaluation entails and how to organize a structured software evaluation process; to define a structured methodology for conducting the software evaluation process; to develop a set of PC software evaluation criteria and evaluation rating scales; and to conduct PC software evaluations in accordance with the identified methodology. Communication Packages, Network System Software, Graphics Support Software, Environment Management Software, General Utilities. This report represents one of the 72 attachment reports to the University of Southwestern Louisiana's Final Report on NASA Grant NGT-19-010-900. Accordingly, appropriate care should be taken in using this report out of context of the full Final Report.

Dominick, Wayne D.↗

DOE OSTI2024

Evaluating downscaled products with expected hydroclimatic co-variances

Abstract. There has been widespread adoption of downscaled products amongst practitioners and stakeholders to ascertain risk from climate hazards at the local scale (e.g., ∼ 5 km resolution). Such products must nevertheless be consistent with physical laws to be credible and of value to users. Here we evaluate statistically and dynamically downscaled products by examining local co-evolution of downscaled temperature and precipitation during convective and frontal precipitation events (two mechanisms testable with just temperature and precipitation). We find that two widely used statistical downscaling techniques (Localized Constructed Analogs version 2, LOCA2, and Seasonal Trends and Analysis of Residuals Empirical Statistical Downscaling Model, STAR-ESDM) generally preserve expected co-variances during convective precipitation events over the historical and future projected intervals as compared to European Centre for Medium-Range Weather Forecasts Reanalysis v5 (ERA5) and two observation-based data products (Livneh and nClimGrid-Daily). However, both techniques dampen future intensification of frontal precipitation that is otherwise robustly captured in global climate models (i.e., prior to downscaling) and with process-based dynamical downscaling across five different regional climate models. In the case of LOCA2, this leads to appreciable underestimation of future frontal precipitation event intensity. This study is one of the first to quantify a likely ramification of the stationarity assumption underlying statistical downscaling methods and identify a phenomenon where projections of future change diverge depending on data production method employed. Finally, our work proposes expected co-variances during convective and frontal precipitation as useful evaluation diagnostics that can be universally applied to a wide range of statistically downscaled products.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1976

Central Data Processing System (CDPS) user's manual: Solar heating and cooling program

The software and data base management system required to assess the performance of solar heating and cooling systems installed at multiple sites is presented. The instrumentation data associated with these systems is collected, processed, and presented in a form which supported continuity of performance evaluation across all applications. The CDPS consisted of three major elements: communication interface computer, central data processing computer, and performance evaluation data base. Users of the performance data base were identified, and procedures for operation, and guidelines for software maintenance were outlined. The manual also defined the output capabilities of the CDPS in support of external users of the system.

Source record↗

NASA NTRS2013

Operational Evaluatioin of Dynamic Weather Routes at American Airlines

Dynamic Weather Routes (DWR) is a search engine that continuously and automatically analyzes inflight aircraft in en route airspace and proposes simple route amendments for more efficient routes around convective weather while considering sector congestion, traffic conflicts, and active Special Use Airspace. NASA and American Airlines (AA) are conducting an operational trial of DWR at the AA System Operations Center in Fort Worth, TX. The trial includes only AA flights in Fort Worth Center airspace. Over the period from July 31, 2012 through August 31, 2012, 45% of routes proposed by DWR and evaluated by AA users - air traffic control coordinators and flight dispatchers - were rated as acceptable as proposed or with some modifications. The wind-corrected potential flying time savings for these acceptable routes totals 470 flying min, and results suggest another 1,500 min of potential savings for flights not evaluated due to staffing limitations. A sector congestion analysis shows that in only two out of 83 DWR routes rated acceptable by AA staff were the flights predicted to fly through a congested sector inside of 30 min downstream of present position. This shows that users considered sector congestion data provided by DWR automation and in nearly all cases did not accept routes through over-capacity sectors. It is estimated that 12 AA flights were given reroute clearances as a direct result of DWR for a total savings of 67 flying min.

air traffic trajectory automation↗

DOE OSTI2024

Visual feedback and guided balance training in an immersive virtual reality environment for lower extremity rehabilitation

Balance training is essential for physical rehabilitation procedures, as it can improve functional mobility and enhance cognitive coordination. However, conventional balance training methods may have limitations in terms of motivation, real-time objective feedback, and personalization, which a virtual reality (VR) setup may provide a better alternative. In this work, we present an immersive VR training environment for lower extremity balance rehabilitation with real-time guidance and feedback. The VR training environment immerses the user in a 3D ice rink model where a virtual coach (agent) leads them through a series of balance poses, and the user controls a trainee avatar with their own movements. Here we developed two coaching styles: positive-reinforcement and autonomous-supportive, and two viewpoints of the trainee avatar: first-person and third-person. The proposed environment was evaluated in a user study with healthy, non-clinical participants (n = 16, 24.4 ± 5.7 years old, 9 females). Our results show that participants showed stronger performance in the positive-reinforcement style compared to the autonomous-supportive style. Additionally, in the third-person viewpoint, the participants exhibited more stability in the positive-reinforcement style compared to the autonomous-supportive style. For viewpoint, participants exhibited stronger performance in the first-person viewpoint compared to third-person in the autonomous-supportive style, while they were comparable in the positive-reinforcement style. We observed no significant effects on the foot height and number of mistakes. Furthermore, we report the analysis of user performance with balance training poses and subjective measures based on questionnaires to assess the user experience, usability, and task load. The proposed VR balance training could offer an interactive, adaptive, and engaging environment and open new potential research directions for lower extremity rehabilitation.

97 MATHEMATICS AND COMPUTING↗