SEARCH · Search NASA

Results for “evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

NASA NTRS1992

Proposed evaluation framework for assessing operator performance with multisensor displays

Despite aggressive work on the development of sensor fusion algorithms and techniques, no formal evaluation procedures have been proposed. Based on existing integration models in the literature, an evaluation framework is developed to assess an operator's ability to use multisensor, or sensor fusion, displays. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The operator's performance with the sensor fusion display can be compared to the models' predictions based on the operator's performance when viewing the original sensor displays prior to fusion. This allows for the determination as to when a sensor fusion system leads to: 1) poorer performance than one of the original sensor displays (clearly an undesirable system in which the fused sensor system causes some distortion or interference); 2) better performance than with either single sensor system alone, but at a sub-optimal (compared to the model predictions) level; 3) optimal performance (compared to model predictions); or, 4) super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays. An experiment demonstrating the usefulness of the proposed evaluation framework is discussed.

Foyle, David C.↗

NASA NTRS1993

Stochastic performance modeling and evaluation of obstacle detectability with imaging range sensors

Statistical modeling and evaluation of the performance of obstacle detection systems for Unmanned Ground Vehicles (UGVs) is essential for the design, evaluation, and comparison of sensor systems. In this report, we address this issue for imaging range sensors by dividing the evaluation problem into two levels: quality of the range data itself and quality of the obstacle detection algorithms applied to the range data. We review existing models of the quality of range data from stereo vision and AM-CW LADAR, then use these to derive a new model for the quality of a simple obstacle detection algorithm. This model predicts the probability of detecting obstacles and the probability of false alarms, as a function of the size and distance of the obstacle, the resolution of the sensor, and the level of noise in the range data. We evaluate these models experimentally using range data from stereo image pairs of a gravel road with known obstacles at several distances. The results show that the approach is a promising tool for predicting and evaluating the performance of obstacle detection with imaging range sensors.

Matthies, Larry↗

NASA NTRS1994

Evaluating modeling tools for the EDOS

The Earth Observing System (EOS) Data and Operations System (EDOS) Project is developing a functional, system performance model to support the system implementation phase of the EDOS which is being designed and built by the Goddard Space Flight Center (GSFC). The EDOS Project will use modeling to meet two key objectives: (1) manage system design impacts introduced by unplanned changed in mission requirements; and (2) evaluate evolutionary technology insertions throughout the development of the EDOS. To select a suitable modeling tool, the EDOS modeling team developed an approach for evaluating modeling tools and languages by deriving evaluation criteria from both the EDOS modeling requirements and the development plan. Essential and optional features for an appropriate modeling tool were identified and compared with known capabilities of several modeling tools. Vendors were also provided the opportunity to model a representative EDOS processing function to demonstrate the applicability of their modeling tool to the EDOS modeling requirements. This paper emphasizes the importance of using a well defined approach for evaluating tools to model complex systems like the EDOS. The results of this evaluation study do not in any way signify the superiority of any one modeling tool since the results will vary with the specific modeling requirements of each project.

Knoble, Gordon↗

NASA NTRS1995

Flight test evaluation of the Stanford University/United Airlines differential GPS Category 3 automatic landing system

Test flights were conducted to evaluate the capability of Differential Global Positioning System (DGPS) to provide the accuracy and integrity required for International Civil Aviation Organization (ICAO) Category (CAT) 3 precision approach and landings. These test flights were part of a Federal Aviation Administration (FAA) program to evaluate the technical feasibility of using DGPS based technology for CAT 3 precision approach and landing applications. A United Airlines Boeing 737-300 (N304UA) was equipped with DGPS receiving equipment and additional computing capability provided by Stanford University. The test flights were conducted at NASA Ames Research Center's Crows Landing Flight Facility, Crows Landing, California. The flight test evaluation was based on completing 100 approaches and autolandings; 90 touch and go, and 10 terminating with a full stop. Two types of accuracy requirements were evaluated: 1) Total system error, based on the Required Navigation Performance (RNP), and 2) Navigation sensor error, based on ICAO requirements for the Microwave Landing System (MLS). All of the approaches and autolandings were evaluated against ground truth reference data provided by a laser tracker. Analysis of these approaches and autolandings shows that the Stanford University/United Airlines system met the requirements for a successful approach and autolanding 98 out of 100 approaches and autolandings, based on the total system error requirements as specified in the FAA CAT 3 Level 2 Flight Test Plan.

Kaufmann, David N.↗

NASA NTRS1994

Advanced Video Analysis Needs for Human Performance Evaluation

Evaluators of human task performance in space missions make use of video as a primary source of data. Extraction of relevant human performance information from video is often a labor-intensive process requiring a large amount of time on the part of the evaluator. Based on the experiences of several human performance evaluators, needs were defined for advanced tools which could aid in the analysis of video data from space missions. Such tools should increase the efficiency with which useful information is retrieved from large quantities of raw video. They should also provide the evaluator with new analytical functions which are not present in currently used methods. Video analysis tools based on the needs defined by this study would also have uses in U.S. industry and education. Evaluation of human performance from video data can be a valuable technique in many industrial and institutional settings where humans are involved in operational systems and processes.

Campbell, Paul D.↗

NASA NTRS1996

UNIX-based operating systems robustness evaluation

Robust operating systems are required for reliable computing. Techniques for robustness evaluation of operating systems not only enhance the understanding of the reliability of computer systems, but also provide valuable feed- back to system designers. This thesis presents results from robustness evaluation experiments on five UNIX-based operating systems, which include Digital Equipment's OSF/l, Hewlett Packard's HP-UX, Sun Microsystems' Solaris and SunOS, and Silicon Graphics' IRIX. Three sets of experiments were performed. The methodology for evaluation tested (1) the exception handling mechanism, (2) system resource management, and (3) system capacity under high workload stress. An exception generator was used to evaluate the exception handling mechanism of the operating systems. Results included exit status of the exception generator and the system state. Resource management techniques used by individual operating systems were tested using programs designed to usurp system resources such as physical memory and process slots. Finally, the workload stress testing evaluated the effect of the workload on system performance by running a synthetic workload and recording the response time of local and remote user requests. Moderate to severe performance degradations were observed on the systems under stress.

Chang, Yu-Ming↗

NASA NTRS1995

Solar Thermal Concept Evaluation

Concentrated solar thermal energy can be utilized in a variety of high temperature applications for both terrestrial and space environments. In each application, knowledge of the collector and absorber's heat exchange interaction is required. To understand this coupled mechanism, various concentrator types and geometries, as well as, their relationship to the physical absorber mechanics were investigated. To conduct experimental tests various parts of a 5,000 watt, thermal concentrator, facility were made and evaluated. This was in anticipation at a larger NASA facility proposed for construction. Although much of the work centered on solar thermal propulsion for an upper stage (less than one pound thrust range), the information generated and the facility's capabilities are applicable to material processing, power generation and similar uses. The numerical calculations used to design the laboratory mirror and the procedure for evaluating other solar collectors are presented here. The mirror design is based on a hexagonal faceted system, which uses a spherical approximation to the parabolic surface. The work began with a few two dimensional estimates and continued with a full, three dimensional, numerical algorithm written in FORTRAN code. This was compared to a full geometry, ray trace program, BEAM 4, which optimizes the curvatures, based on purely optical considerations. Founded on numerical results, the characteristics of a faceted concentrator were construed. The numerical methodologies themselves were evaluated and categorized. As a result, the three-dimensional FORTRAN code was the method chosen to construct the mirrors, due to its overall accuracy and superior results to the ray trace program. This information is being used to fabricate and subsequently, laser map the actual mirror surfaces. Evaluation of concentrator mirrors, thermal applications and scaling the results of the 10 foot diameter mirror to a much larger concentrator, were studied. Evaluations, recommendations and pit falls regarding the structure, materials and facility design are presented.

Hawk, Clark W.↗

NASA NTRS1998

Evaluation of Portable Multi-Gas Analyzers for use by Safety Personnel

During confined space entry operations as well as Shuttle-safing operations, United Space Alliance (USA)/National Aeronautics and Space Administration (NASA) safety personnel use a variety of portable instrumentation to monitor for hazardous levels of compounds such as nitrogen dioxide (N%), monomethylhydrazine (NMM), FREON 21, ammonia (NH3), oxygen (O2), and combustibles (as hydrogen (H2)). Except for O2 and H2, each compound is monitored using a single analyzer. In many cases these analyzers are 5 to 10 years old and require frequent maintenance. In addition, they are cumbersome to carry and tend to make the job of personnel monitoring physically taxing. As part of an effort to upgrade the sensor technology background information was requested from a total of 27 manufacturers of portable multi-gas instruments. A set of criteria was established to determine which vendors would be selected for laboratory evaluation. These criteria were based on requests made by USA/NASA Safety personnel in order to meet requirements within their respective areas for confined-space and Shuttle-safing operations. Each of the 27 manufacturers of multi-gas analyzers was sent a copy of the criteria and asked to fill in the appropriate information pertaining to their instrumentation. Based on the results of the sensor criteria worksheets, a total of 9 vendors out of 27 surveyed manufacturers were chosen for evaluation. Each vendor included in the final evaluation process was requested to configure each of two analyzers with NO2, NH3, O2, and combustible sensors. A set of lab tests was designed in order to determine which of the multi-gas instruments under evaluation was best suited for use in both shuttle and confined space operations. These tests included linearity/repeatability, zero/span drift response/recovery, humidity, interference, and maintenance. At the conclusion of lab testing three vendors were selected for additional field testing. Based on the results of both the lab and field evaluations a single vendor was recommended for use by NASA/IJSA Safety personnel. Vendor selection criteria, as well as the results from both laboratory and field testing of the multi-gas analyzers, are presented as part of this paper.

Lueck, D. E.↗

NASA NTRS1997

Nondestructive Evaluation of Adhesively Bonded Joints by Acousto-Ultrasonic Technique and Acoustic Emission

Reliable applications of adhesively bonded joints require an effective nondestructive evaluation technique for their bond strength prediction. To properly evaluate factors affecting bond strength, effects of defects such as voids and disbonds on stress distribution in the overlap region must be understood. At the same time, in order to use acousto-ultrasonic (AU) technique to evaluate bond quality, the effect of these defects on dynamic response of single lap joints must be clear. The stress distribution in a single lap joint with and without defects (void or disbond) is analyzed. A bar-Theta parameter which contains adherend and adhesive thickness and properties is introduced. It is shown for bonded joints with bar-Theta greater than 10, that a symmetric void or disbond in the middle of overlap up to the 70 percent of overlap length has negligible effect on bond strength. In contrast frequency response analyses by a finite element technique showed that the dynamic response is affected significantly by the presence of voids or disbonds. These results have direct implication in the interpretations of AU results. Through transmission attenuation and a number of AU parameters for various specimens with and without defects are evaluated. It is found that although void and disbond have similar effects on bond strength (stress distribution), they have completely different effects on wave propagation characteristics. For steel-adhesive-steel specimens with voids, the attenuation changes are related to the bond strength. However, the attenuation changes for specimens with disbond are fairly constant over a disbond range. In order to incorporate the location of defects in AU parameters, a weighting function is introduced. Using an immersion system with focused transducers, a number of AU parameters are evaluated. It is found that by incorporating weighting functions in these parameters better sensitivities (AU parameters vs. bond strength) are achieved. Acoustic emission (AE) activities of steel-adhesive-steel specimens with bar-Theta equal to 3.4 are monitored. Two different formats of energy vs. time have resulted, each corresponding to the perfect specimens or the specimens with void or disbond. The relative acoustic energy and the number of events at failure are found to be a means for predicting the bond strength.

Nayeb-Hashemi, Hamid↗

NASA NTRS1999

Gas Sensor Evaluations in Polymer Combustion Product Atmospheres

Toxic gases produced by the combustion or thermo-oxidative degradation of materials such as wire insulation, foam, plastics, or electronic circuit boards in space shuttle or space station crew cabins may pose a significant hazard to the flight crew. Toxic gas sensors are routinely evaluated in pure gas standard mixtures, but the possible interferences from polymer combustion products are not routinely evaluated. The NASA White Sands Test Facility (WSTF) has developed a test system that provides atmospheres containing predetermined quantities of target gases combined with the coincidental combustion products of common spacecraft materials. The target gases are quantitated in real time by infrared (IR) spectroscopy and verified by grab samples. The sensor responses are recorded in real time and are compared to the IR and validation analyses. Target gases such as carbon monoxide, hydrogen cyanide, hydrogen chloride, and hydrogen fluoride can be generated by the combustion of poly(vinyl chloride), polyimide-fluoropolymer wire insulation, polyurethane foam, or electronic circuit board materials. The kinetics and product identifications for the combustion of the various materials were determined by thermogravimetric-IR spectroscopic studies. These data were then scaled to provide the required levels of target gases in the sensor evaluation system. Multisensor toxic gas monitors from two manufacturers were evaluated using this system. In general, the sensor responses satisfactorily tracked the real-time concentrations of toxic gases in a dynamic mixture. Interferences from a number of organic combustion products including acetaldehyde and bisphenol-A were minimal. Hydrogen bromide in the products of circuit board combustion registered as hydrogen chloride. The use of actual polymer combustion atmospheres for the evaluation of sensors can provide additional confidence in the reliability of the sensor response.

Delgado, Rafael H.↗

NASA NTRS1999

Structural Evaluation of the RSRM Nozzle Replacement Adhesive

This paper describes the structural performance evaluation of a replacement adhesive for the Reusable Solid Rocket Motor (RSRM) nozzle utilizing finite element analysis. Due to material obsolescence and industrial safety issues, the two current structural adhesives, EA 913 and EA 946 are to be replaced with a new adhesive. TIGA 321. The structural evaluation in support of the adhesive replacement effort includes residual stress, transportation, and flight analyses. Factors of safety are calculated using the stress response from each analysis. The factors of safety are used as the limiting criteria to compare the replacement adhesive against the current adhesives. Included in this paper are the analytical approach, assumptions and modeling techniques as well as the results of the evaluation. An important factor to the evaluation is the similarity in constitutive material properties (elastic modulus and Poisson's ratio) between TIGA 321 and EA 913. This similarity leads to equivalent material response from the two adhesives. However, TIGA 321 surpasses EA 913's performance due to higher material capabilities. Conversely, the change in stress response from EA 946 to TIGA 321 is more apparent: this is primarily attributed to the difference in the modulii of the two adhesives, which differ by two orders of magnitude. The results of the bondline evaluation indicate that the replacement adhesive provides superior performance than the current adhesives with only minor exceptions. Furthermore, TIGA 321 causes only a minor chance in the response of the phenolic and metal components.

Batista-Rodriguez, A.↗

NASA NTRS2000

Evaluation of the Regional Atmospheric Modeling System in the Eastern Range Dispersion Assessment System

The Applied Meteorology Unit is conducting an evaluation of the Regional Atmospheric Modeling System (RAMS) contained within the Eastern Range Dispersion Assessment System (ERDAS). ERDAS provides emergency response guidance for operations at the Cape Canaveral Air Force Station and the Kennedy Space Center in the event of an accidental hazardous material release or aborted vehicle launch. The prognostic data from RAMS is available to ERDAS for display and is used to initialize the 45th Range Safety (45 SW/SE) dispersion model. Thus, the accuracy of the 45 SW/SE dispersion model is dependent upon the accuracy of RAMS forecasts. The RAMS evaluation task consists of an objective and subjective component for the Florida warm and cool seasons of 1999-2000. The objective evaluation includes gridded and point error statistics at surface and upper-level observational sites, a comparison of the model errors to a coarser grid configuration of RAMS, and a benchmark of RAMS against the widely accepted Eta model. The warm-season subjective evaluation involves a verification of the onset and movement of the Florida east coast sea breeze and RAMS forecast precipitation. This interim report provides a summary of the RAMS objective and subjective evaluation for the 1999 Florida warm season only.

Case, Jonathan↗

NASA NTRS2001

Phased Array Ultrasonic Evaluation of Space Shuttle Main Engine (SSME) Nozzle Weld

This viewgraph presentation gives an overview of the phased array ultrasonic evaluation of the Space Shuttle Main Engine (SSME) nozzle weld. Details are given on the nondestructive testing evaluation approach, conventional shear wave and phased array techniques, and an x-ray versus phased array risk analysis. The field set-up was duplicated to the greatest extent possible in the laboratory and the phased array ultrasonic technique was developed and validated prior to weld evaluation. Results are shown for the phased array ultrasonic evaluation and conventional ultrasonic evaluation results.

James, Steve↗

NASA NTRS2001

Electronic Nose: Evaluation of Kamina Prototype Unit

The Kamina, Sam and Cyranose electronic nose systems were evaluated and partially trained. Much work was performed on the Kamina as it has the ability to respond to low (less than 10 ppb) concentrations of hydrazine compounds. We were able to tell the difference between Hydrazine (Hz) and Monomethylhydrazine (MMH) in standard clean humid air. We were able to detect MMH in reduced pressure (1/3 atm) at about 250 ppb, however the training set was to far from the real situation to be useful now. Various engineering and usability aspects of both the noses was noted, especially the software. One serious physical engineering flaw was remedied in the Kamina system. A gas flow manifold was created for the Sam system. Different chips were evaluated for the Kamina system. It is still unclear if they can be exchanged without retraining the software.The Sam Detect commercial unit was evaluated for solvent detection and evaluation. It was able to successfully identify some solvents. The Cyranose, was observed and evaluated for two days. It has the ability to detect gasses in the 100 parts per million level but not the 10 parts per billion level. It is very sensitive to humidity changes; there is software to partially handle this.

Schattke, Nathan↗

NASA NTRS1999

AHS Technical Committee Highlights: Test and Evaluation

The AHS technical committee for Test and Evaluation is comprised of representatives from industry and government rotorcraft technology development centers. The committee considers many aspects of rotorcraft and VTOL testing, including the evaluation of components and subsystems. In addition, the committee seeks to evaluate the effectiveness of new procedures and operational solutions to problems that are encountered in the low-speed flight regime for both land and ship-based environments. This article presents the test and evaluation committee highlights for 98-99. Though by no means comprehensive, the article includes significant efforts related to test and evaluation contributed by the committee members for their respective organizations.

Jacklin, Stephen A.↗

NASA NTRS2003

A Gold Standards Approach to Training Instructors to Evaluate Crew Performance

The Advanced Qualification Program requires that airlines evaluate crew performance in Line Oriented Simulation. For this evaluation to be meaningful, instructors must observe relevant crew behaviors and evaluate those behaviors consistently and accurately against standards established by the airline. The airline industry has largely settled on an approach in which instructors evaluate crew performance on a series of event sets, using standardized grade sheets on which behaviors specific to event set are listed. Typically, new instructors are given a class in which they learn to use the grade sheets and practice evaluating crew performance observed on videotapes. These classes emphasize reliability, providing detailed instruction and practice in scoring so that all instructors within a given class will give similar scores to similar performance. This approach has value but also has important limitations; (1) ratings within one class of new instructors may differ from those of other classes; (2) ratings may not be driven primarily by the specific behaviors on which the company wanted the crews to be scored; and (3) ratings may not be calibrated to company standards for level of performance skill required. In this paper we provide a method to extend the existing method of training instructors to address these three limitations. We call this method the "gold standards" approach because it uses ratings from the company's most experienced instructors as the basis for training rater accuracy. This approach ties the training to the specific behaviors on which the experienced instructors based their ratings.

Baker, David P.↗

NASA NTRS2003

Novel approaches and application of contemporary sensory evaluation practices in iron fortification programs

Iron deficiency is the leading nutritional deficiency in the U.S. and the rest of the world, with its highest prevalences in the developing world. Iron fortification of food has been proposed as a strategy to reduce the high prevalence of iron deficiency. Poor consumer acceptance, unacceptable taste, and discoloration of the iron-fortified foods have been frequently listed as causes of unsuccessful iron fortification programs. An excellent prospect for improving consumer acceptance of iron-fortified foods is the incorporation of a thorough, organized, and unified approach to sensory evaluation practices into iron fortification programs for product optimization. The information gained from systematic sensory evaluation allows for the manipulation of the sensory attributes, and thus improvement of the sensory properties of the fortified food. However, iron fortification programs have not systematically measured the effect of fortification on the sensory quality of the food. Because sensory evaluation is an important criterion in successful iron fortification, an integrated approach is necessary. Therefore, nutritionists and sensory scientists should work closely with each other to select the most suitable sensory tests and methods. The objectives of this article are to: (1) critically review and discuss some traditional and contemporary approaches and applications of sensory evaluation practices in iron fortification programs, and (2) demonstrate the importance of incorporating a multidisciplinary, systematic sensory evaluation approach in iron fortification programs.

Review, Tutorial↗

NASA NTRS2004

An Evaluation Methodology for Traffic Awareness Displays

An evaluation methodology for traffic awareness displays for helicopters and other vertical/short takeoff aircraft was developed. Pilots of vertical/short takeoff aircraft wil1 require more traffic information than would pilots of conventional aircraft to avoid both other vertical/short takeoff traffic and conventional traffic. The BF Goodrich Skywatch traffic advisory display was used as a candidate display to develop a procedure for evaluating the usefulness of such displays. Four high-time helicopter pilots participated in a 16-hour flight evaluation. They flew a closed circuit in the San Francisco Bay Area. In one-half of the flights the evaluation pilot had the traffic advisory display as an aid in detecting and locating traffic. In the other half of the flights the traffic advisory display was not available to the evaluation pilot. Data examined include measures of traffic advisory display performance and pilot performance in detecting traffic, as well as subjective workload and situation awareness data. The traffic advisory system did not help the pilots to detect more traffic. The importance of detection to traffic awareness is discussed.

DeMaio, Joe↗