Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1988

The development of a test methodology for the evaluation of EVA gloves

This paper describes the development of a standardized set of tests designed to assess EVA-gloved hand capabilities in six measurement domains: range of motion, strength, tactile perception, dexterity, fatigue, and comfort. Based upon an assessment of general human-hand functioning and EVA task requirements, several tests within each measurement domain were developed to provide a comprehensive evaluation. All tests were designed to be conducted in a glove box with the bare hand as a baseline and the EVA glove at operating pressure.

O'Hara, John M.↗
NASA NTRS1994

Beware of agents when flying aircraft: Basic principles behind a generic methodology for the evaluation and certification of advanced aviation systems

There is currently a growing interest in the aeronautical community to assess the effects of the increasing levels of automation on pilots' performance and overall safety. The first effect of automation is the change in the nature of the pilot's role on the flight deck. Pilots have become supervisors who monitor aircraft systems in usual situations and intervene only when unanticipated events occur. Instead of 'hand flying' the airplane, pilots contribute to the control of aircraft by acting as mediators, instructions given to the automation. By eliminating the need for manually controlling normal situations, such a role division has reduced the opportunities for the pilot to acquire experience and skills necessary to safely cope with abnormal events. Difficulties in assessing the state and behavior of automation arise mainly from four factors: (1) the complexity of current systems and consequence mode-related problems; (2) the intrinsic autonomy of automation which is able to fire mode transitions without explicit commands from the pilots; (3) the bad quality of feed-back from the control systems displays and interfaces to the pilots; and (4) the fact that the automation currently has no explicit representation of the current pilots' intentions and strategy. Assuming certification has among its major goals to guarantee the passengers' and pilots' safety and the airplane integrity under normal and abnormal operational conditions, the authors suggest it would be particularly fruitful to come up with a conceptual reference system providing the certification authorities both with a theoretical framework and a list of principles usable for assessing the quality of the equipment and designs under examination. This is precisely the scope of this paper. However, the authors recognize that the conceptual presented is still under development and would thus be best considered as a source of reflection for the design, evaluation and certification processes of advanced aviation technologies.

Javaux, Denis↗
NASA NTRS2017

An Evaluation of Structural Analysis Methodologies for Space Deployable Structures

Benchmarks are introduced for evaluating the performance of numerical simulations of space deployable structures. These benchmarks embody the key challenges of interest to future large space deployable structures, including large angle motion, contact between flexible bodies, and the presence of both soft and stiff mechanical components. The benchmarks were used in companion studies to evaluate the ADAMS multibody dynamics code, the LS-Dyna nonlinear finite element code, and the Sierra large-scale parallel nonlinear finite element code. In the past, only multibody codes would have been considered for this application. This study found that all three codes could be used for these benchmarks, a finding that may lead to larger scale, higher fidelity simulations in the future.

Mobrem, Mehran↗
NASA NTRS1972

Transportation Systems Evaluation

A methodology for the analysis of transportation systems consisting of five major interacting elements is reported. The analysis begins with the causes of travel demand: geographic, economic, and demographic characteristics as well as attitudes toward travel. Through the analysis, the interaction of these factors with the physical and economic characteristics of the transportation system is determined. The result is an evaluation of the system from the point of view of both passenger and operator. The methodology is applicable to the intraurban transit systems as well as major airlines. Applications of the technique to analysis of a PRT system and a study of intraurban air travel are given. In the discussion several unique models or techniques are mentioned: i.e., passenger preference modeling, an integrated intraurban transit model, and a series of models to perform airline analysis.

Fanning, M. L.↗
NASA NTRS1990

A framework for assessing the adequacy and effectiveness of software development methodologies

Tools, techniques, environments, and methodologies dominate the software engineering literature, but relatively little research in the evaluation of methodologies is evident. This work reports an initial attempt to develop a procedural approach to evaluating software development methodologies. Prominent in this approach are: (1) an explication of the role of a methodology in the software development process; (2) the development of a procedure based on linkages among objectives, principles, and attributes; and (3) the establishment of a basis for reduction of the subjective nature of the evaluation through the introduction of properties. An application of the evaluation procedure to two Navy methodologies has provided consistent results that demonstrate the utility and versatility of the evaluation procedure. Current research efforts focus on the continued refinement of the evaluation procedure through the identification and integration of product quality indicators reflective of attribute presence, and the validation of metrics supporting the measure of those indicators. The consequent refinement of the evaluation procedure offers promise of a flexible approach that admits to change as the field of knowledge matures. In conclusion, the procedural approach presented in this paper represents a promising path toward the end goal of objectively evaluating software engineering methodologies.

Arthur, James D.↗
NASA NTRS1994

Piloted Evaluation of an Integrated Methodology for Propulsion and Airframe Control Design

An integrated methodology for propulsion and airframe control has been developed and evaluated for a Short Take-Off Vertical Landing (STOVL) aircraft using a fixed base flight simulator at NASA Lewis Research Center. For this evaluation the flight simulator is configured for transition flight using a STOVL aircraft model, a full nonlinear turbofan engine model, simulated cockpit and displays, and pilot effectors. The paper provides a brief description of the simulation models, the flight simulation environment, the displays and symbology, the integrated control design, and the piloted tasks used for control design evaluation. In the simulation, the pilots successfully completed typical transition phase tasks such as combined constant deceleration with flight path tracking, and constant acceleration wave-off maneuvers. The pilot comments of the integrated system performance and the display symbology are discussed and analyzed to identify potential areas of improvement.

Bright, Michelle M.↗
NASA NTRS1984

A methodology for collecting valid software engineering data

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments (other NRL Reports). The application showed that the methodology was both feasible and useful.

Basili, V. R.↗
NASA NTRS1983

A methodology for collecting valid software engineering data

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. The data to be collected are based on the changes made to the software during development, and are obtained when the changes are made. To insure accuracy of the data, validation is performed concurrently with software development and data collection. Validation is based on interviews with those people supplying the data. Results from using the methodology show that data validation is a necessary part of change data collection. Without it, as much as 50% of the data may be erroneous. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments. The application showed that the methodology was both feasible and useful.

Basili, Victor R.↗
NASA NTRS2000

Development of a Methodology to Conduct Usability Evaluation for Hand Tools that May Reduce the Amount of Small Parts that are Dropped During Installation while Processing Space Flight Hardware

Foreign object debris (FOD) is an important concern while processing space flight hardware. FOD can be defined as "The debris that is left in or around flight hardware, where it could cause damage to that flight hardware," (United Space Alliance, 2000). Just one small screw left unintentionally in the wrong place could delay a launch schedule while it is retrieved, increase the cost of processing, or cause a potentially fatal accident. At this time, there is not a single solution to help reduce the number of dropped parts such as screws, bolts, nuts, and washers during installation. Most of the effort is currently focused on training employees and on capturing the parts once they are dropped. Advances in ergonomics and hand tool design suggest that a solution may be possible, in the form of specialty hand tools, which secure the small parts while they are being handled. To assist in the development of these new advances, a test methodology was developed to conduct a usability evaluation of hand tools, while performing tasks with risk of creating FOD. The methodology also includes hardware in the form of a testing board and the small parts that can be installed onto the board during a test. The usability of new hand tools was determined based on efficiency and the number of dropped parts. To validate the methodology, participants were tested while performing a task that is representative of the type of work that may be done when processing space flight hardware. Test participants installed small parts using their hands and two commercially available tools. The participants were from three groups: (1) students, (2) engineers / managers and (3) technicians. The test was conducted to evaluate the differences in performance when using the three installation methods, as well as the difference in performance of the three participant groups.

Miller, Darcy↗
NASA NTRS1999

An Evaluation of Aircraft Emissions Inventory Methodology by Comparisons with Reported Airline Data

This report provides results of work done to evaluate the calculation methodology used in generating aircraft emissions inventories. Results from the inventory calculation methodology are compared to actual fuel consumption data. Results are also presented that show the sensitivity of calculated emissions to aircraft payload factors. Comparisons of departures made, ground track miles flown and total fuel consumed by selected air carriers were made between U.S. Dept. of Transportation (DOT) Form 41 data reported for 1992 and results of simplified aircraft emissions inventory calculations. These comparisons provide an indication of the magnitude of error that may be present in aircraft emissions inventories. To determine some of the factors responsible for the errors quantified in the DOT Form 41 analysis, a comparative study of in-flight fuel flow data for a specific operator's 747-400 fleet was conducted. Fuel consumption differences between the studied aircraft and the inventory calculation results may be attributable to several factors. Among these are longer flight times, greater actual aircraft weight and performance deterioration effects for the in-service aircraft. Results of a parametric study on the variation in fuel use and NOx emissions as a function of aircraft payload for different aircraft types are also presented.

Daggett, D. L.↗
NASA NTRSDate not supplied

A Survey Protocol to Assess Meaningfulness and Usefulness of Automated Topic Finding in the NASA Aviation Safety Reporting System

Context: The NASA Aviation Safety Reporting System (ASRS) is a voluntary confidential aviation safety reporting system. The ASRS receives reports from pilots, air traffic controllers, flight attendants and other involved in aviation operations. The reports are de-identified and coded by ASRS expert safety analysts and a short descriptive synopsis is written to describe the safety issue. The de-identified reports are then disseminated to the aviation community in a number of ways including entry into an online database, Safety Alert Bulletins and For Your Information Notices, and the CALLBACK newsletter. Key to these publications are the timely processing (de-identification, coding and summarization) of new reports, which is currently done by ASRS expert safety analysts. Thus, we believe topic modelling could decrease effort in ASRS, if topics are comprehensible. Aim: We propose a methodology to evaluate whether automated topic finding using topic modelling provides meaningful and useful topics. Method: We extend the total error survey methodology to evaluate user topic comprehension of machine learning outputs. To accomplish this we performed a literature review to identify existing methods and define a construct for topic comprehension, utilizing existing ASRS synopsis writing practices to more precisely define meaningfulness and usefulness. Results: A survey protocol was created that addresses the limitations of other survey protocols found in the literature review, which we found lacking in rationale and clear protocol definition. Conclusion: The surveying of user understanding in machine learning outputs presents challenges due to the explosion of parameters to control for and the lack of systematic approach presented in the literature. More reproducible work and survey protocols are needed in the literature and our work is one step towards that direction.

topic finding↗
NASA NTRS1982

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation. Volume 1: Study results

A variety of artificial intelligence techniques which could be used with regard to NASA space applications and robotics were evaluated. The techniques studied were decision tree manipulators, problem solvers, rule based systems, logic programming languages, representation language languages, and expert systems. The overall structure of a robotic simulation tool was defined and a framework for that tool developed. Nonlinear and linearized dynamics equations were formulated for n link manipulator configurations. A framework for the robotic simulation was established which uses validated manipulator component models connected according to a user defined configuration.

Lowrie, J. W.↗
NASA NTRS1992

Summary: Experimental validation of real-time fault-tolerant systems

Testing and validation of real-time systems is always difficult to perform since neither the error generation process nor the fault propagation problem is easy to comprehend. There is no better substitute to results based on actual measurements and experimentation. Such results are essential for developing a rational basis for evaluation and validation of real-time systems. However, with physical experimentation, controllability and observability are limited to external instrumentation that can be hooked-up to the system under test. And this process is quite a difficult, if not impossible, task for a complex system. Also, to set up such experiments for measurements, physical hardware must exist. On the other hand, a simulation approach allows flexibility that is unequaled by any other existing method for system evaluation. A simulation methodology for system evaluation was successfully developed and implemented and the environment was demonstrated using existing real-time avionic systems. The research was oriented toward evaluating the impact of permanent and transient faults in aircraft control computers. Results were obtained for the Bendix BDX 930 system and Hamilton Standard EEC131 jet engine controller. The studies showed that simulated fault injection is valuable, in the design stage, to evaluate the susceptibility of computing sytems to different types of failures.

Iyer, R. K.↗
NASA NTRS1982

Evaluating software development by analysis of changes: The data from the software engineering laboratory

An effective data collection methodology for evaluating software development methodologies was applied to four different software development projects. Goals of the data collection included characterizing changes and errors, characterizing projects and programmers, identifying effective error detection and correction techniques, and investigating ripple effects. The data collected consisted of changes (including error corrections) made to the software after code was written and baselined, but before testing began. Data collection and validation were concurrent with software development. Changes reported were verified by interviews with programmers.

Source record↗
NASA NTRS1986

A methodology for the design and evaluation of user interfaces for interactive information systems

The definition of proposed research addressing the development and validation of a methodology for the design and evaluation of user interfaces for interactive information systems is given. The major objectives of this research are: the development of a comprehensive, objective, and generalizable methodology for the design and evaluation of user interfaces for information systems; the development of equations and/or analytical models to characterize user behavior and the performance of a designed interface; the design of a prototype system for the development and administration of user interfaces; and the design and use of controlled experiments to support the research and test/validate the proposed methodology. The proposed design methodology views the user interface as a virtual machine composed of three layers: an interactive layer, a dialogue manager layer, and an application interface layer. A command language model of user system interactions is presented because of its inherent simplicity and structured approach based on interaction events. All interaction events have a common structure based on common generic elements necessary for a successful dialogue. It is shown that, using this model, various types of interfaces could be designed and implemented to accommodate various categories of users. The implementation methodology is discussed in terms of how to store and organize the information.

Dominick, Wayne D.↗
NASA NTRS1984

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation

A generic computer simulation for manipulator systems (ROBSIM) was implemented and the specific technologies necessary to increase the role of automation in various missions were developed. The specific items developed are: (1) capability for definition of a manipulator system consisting of multiple arms, load objects, and an environment; (2) capability for kinematic analysis, requirements analysis, and response simulation of manipulator motion; (3) postprocessing options such as graphic replay of simulated motion and manipulator parameter plotting; (4) investigation and simulation of various control methods including manual force/torque and active compliances control; (5) evaluation and implementation of three obstacle avoidance methods; (6) video simulation and edge detection; and (7) software simulation validation.

Haley, D. C.↗