Search NASA⌕ Search

SEARCH · Search NASA

Results for “Evaluation Procedure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1976

An ad hoc map evaluation procedure

An ad hoc map evaluation procedure is proposed which is most suitable for evaluating low-resolution classification maps against high resolution ground truth maps, such as maps against interpreted aircraft photographs. Commonly practiced sampling and evaluation procedures are impracticable in this context because of difficulties in registration and in comparing the samples. This ad hoc procedure is designed to overcome these two major problems, and its practicability is discussed. Two widely accepted parameters are estimated by the new procedure; namely, the probability of correct classification and the proportion biases. Statistical qualifications are also provided.

Kan, E. P.↗
NASA NTRS1989

Expert systems should be more accurate than human experts - Evaluation procedures from human judgment and decisionmaking

Two procedures for the evaluation of the performance of expert systems are illustrated: one procedure evaluates predictive accuracy; the other procedure is complementary in that it uncovers the factors that contribute to predictive accuracy. Using these procedures, it is argued that expert systems should be more accurate than human experts in two senses. One sense is that expert systems must be more accurate to be cost-effective. Previous research is reviewed and original results are presented which show that simple statistical models typically perform better than human experts for the task of combining evidence from a given set of information sources. The results also suggest the second sense in which expert systems should be more accurate than human experts. They reveal that expert systems should share factors that contribute to human accuracy, but not factors that detract from human accuracy. Thus the thesis is that one should both require and expect systems to be more accurate than humans.

Levi, Keith↗
NASA NTRS1998

A Formal Approach for Designing and Evaluating Procedures

Operator interaction with modern control systems is a topic of great concern in high-risk industries such as nuclear power and commercial aviation. The issues associated with such systems focus on the ability of the operators (e.g., pilots) to achieve mission goals safely while containing failures. Operators must be able to interact safely and reliably with highly automatic and complex systems across the full spectrum of possible operating conditions, including normal, abnormal, and emergency situations. In environments such as commercial aviation, operator interaction with the machine is specified through a set of standard operating procedures (SOP). A procedure represents a collective agreement on the 'best' way to perform a given task. The intent of this paper is to suggest a formal methodology, for designing and evaluating procedures, that is both reliable and systematic. Our approach involves two major elements: a model of the machine and a list of the operator's task specifications (goals). We use formal modeling paradigms for describing the system and super-imposing on it the operator's tasks. Such paradigms, based on recent frameworks such as Statecharts and Hierarchical Hybrid Machines appear to be adequate methods for analyzing operator interaction with modern control systems. To illustrate this methodology, we model and analyze the sequence of actions for an emergency procedure. The procedure, Irregular Engine Start, for a medium-range aircraft, specifies the sequence of immediate actions that must be performed by the crew to avoid an uncontrolled rise in engine temperature during start-up. A model of engine behavior during a hot start is constructed. It also describes the various actions that can be taken by the crew and the resulting outcomes. The model is then opened up as a tree of all possible action sequences. This action tree allows us to trace the correct sequences necessary to achieve the desired end-goal (secure and shut down of the engine). In conclusion, we argue that the current process of designing and evaluating procedures can be improved. We discuss the implications of this approach for designing and evaluating this and other types of procedures. We conclude with insights about the benefits and limitation of this methodology, and offer suggestions for future research.

Degani, Asaf↗
DOE OSTI2025

CyberMESA: Evaluation Procedures and Metrics for Charging Infrastructure Cybersecurity

A report detailing the cybersecurity evaluation and assessment procedures for EV charging infrastructure. This report is authored by three national lab contributors. INL, PNNL, NRL. The grid integration of Electric Vehicle Charging Infrastructure (EVCI) is a large and complex system of systems (SoS) made up of components from many manufacturers and integrated by various companies responsible for their operation and maintenance. This complexity makes it challenging to implement security standards like IEC 62443 across the entire infrastructure. This document aims to aid in the vulnerability assessments of individual devices, such as Electric Vehicle Supply Equipment (EVSE), to support vendors in adopting and implementing security standards such as UL 2900, NIST SP 800-53 and Common Criteria (CC) (ISO/IEC 15408). This document serves as a guide to help evaluate the cybersecurity posture of EVCI assets. It aims to provide a consistent method for evaluating EVCI, enabling cybersecurity research teams to systematically test a wide range of electric vehicle (EV) assets. The target audiences include cybersecurity research teams, vendor development teams, automotive OEMs, and third-party evaluators.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2006

Medical Operations Console Procedure Evaluation: BME Response to Crew Call Down for an Emergency

International Space Station (ISS) Mission Operations are managed by multiple flight control disciplines located at the lead Mission Control Center (MCC) at NASA-Johnson Space Center (JSC). ISS Medical Operations are supported by the complementary roles of Flight Surgeons (Surgeon) and Biomedical Engineer (BME) flight controllers. The Surgeon, a board certified physician, oversees all medical concerns of the crew and the BME provides operational and engineering support for Medical Operations Crew Health Care System. ISS Medical Operations is currently addressing the coordinated response to a crew call down for an emergent medical event, in particular when the BME is the only Medical Operations representative in MCC. In this case, the console procedure BME Response to Crew Call Down for an Emergency will be used. The procedure instructs the BME to contact a Surgeon as soon as possible, coordinate with other flight disciplines to establish a Private Medical Conference (PMC) for the crew and Surgeon, gather information from the crew if time permits, and provide Surgeon with pertinent console resources. It is paramount that this procedure is clearly written and easily navigated to assist the BME to respond consistently and efficiently. A total of five BME flight controllers participated in the study. Each BME participant sat in a simulated MCC environment at a console configured with resources specific to the BME MCC console and was presented with two scripted emergency call downs from an ISS crew member. Each participant used the procedure while interacting with analog MCC disciplines to respond to the crew call down. Audio and video recordings of the simulations were analyzed and each BME participant's actions were compared to the procedure. Structured debriefs were conducted at the conclusion of both simulations. The procedure was evaluated for its ability to elicit consistent responses from each BME participant. Trials were examined for deviations in procedure task completion and/or navigation, in particular the execution of the Surgeon call sequence. Debrief comments were used to analyze unclear procedural steps and to discern any discrepancies between the procedure and generally accepted BME actions. The sequence followed by BME participants differed considerably from the sequence intended by the procedure. Common deviations included the call sequence used to contact Surgeon, the content of BME and crew interaction and the gathering of pertinent console resources. Differing perceptions of task priority and imprecise language seem to have caused multiple deviations from the procedure s intended sequence. The study generated 40 recommendations for the procedure, of which 34 are being implemented. These recommendations address improving the clarity of the instructions, identifying training considerations, expediting Surgeon contact, improving cues for anticipated flight control team communication and identifying missing console tools.

Johnson-Troop↗
NASA NTRS1972

Automation of reliability evaluation procedures through CARE - The computer-aided reliability estimation program.

Description of an on-line interactive computer program called CARE (Computer-Aided Reliability Estimation) which can model self-repair and fault-tolerant organizations and perform certain other functions. Essentially CARE consists of a repository of mathematical equations defining the various basic redundancy schemes. These equations, under program control, are then interrelated to generate the desired mathematical model to fit the architecture of the system under evaluation. The mathematical model is then supplied with ground instances of its variables and is then evaluated to generate values for the reliability-theoretic functions applied to the model.

Mathur, F. P.↗
DOE OSTI2025

The separatrix electron density in JET, ASDEX upgrade and alcator C-Mod H-mode plasmas: A common evaluation procedure and correlation with engineering parameters

The separatrix electron density is an important parameter for core-edge scenario integration in tokamak devices, as it influences plasma confinement, divertor detachment and disruption avoidance. This quantity has been measured in H-mode discharges on JET, ASDEX Upgrade and Alcator C-Mod by applying the same fitting function to Thomson scattering measurements, and by employing the same analysis technique based on scrape-off layer power balance. To estimate the power crossing the separatrix, the inter-ELM time derivative of the plasma energy d W /d t has been experimentally evaluated and found to be approximately a constant fraction of the absorbed heating power. Correlations between n e,sep and engineering parameters have been investigated, revealing that n e,sep scales with the divertor neutral pressure p 0,div in a similar manner across all devices. Additionally, when n e,sep is normalized to the obtained n e,sep dependency, no clear correlation with the plasma current is found. These observations are in agreement with the 2-point model, which suggests that the upstream separatrix density is mainly set by the recycling at the divertor target.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1976

Data reduction and evaluation procedures

The computational procedures that are involved in exhaust emissions data reduction and the use of these computational procedures for determining the quality of the data that is obtained from exhaust measurements were considered. Four problem areas were calculated: (1) the various methods for performing the carbon balance, (2) the method for calculating water correction factors, (3) the method for calculating the exhaust molecular weight, and (4) assessing the quality of the data.

Mirsky, W.↗
NASA NTRS2010

Battery Separator Characterization and Evaluation Procedures for NASA's Advanced Lithium-Ion Batteries

To address the future performance and safety requirements for the electrical energy storage technologies that will enhance and enable future NASA manned aerospace missions, advanced rechargeable, lithium-ion battery technology development is being pursued within the scope of the NASA Exploration Technology Development Program s (ETDP's) Energy Storage Project. A critical cell-level component of a lithium-ion battery which significantly impacts both overall electrochemical performance and safety is the porous separator that is sandwiched between the two active cell electrodes. To support the selection of the optimal cell separator material(s) for the advanced battery technology and chemistries under development, laboratory characterization and screening procedures were established to assess and compare separator material-level attributes and associated separator performance characteristics.

Baldwin, Richard S.↗
DOE OSTI2024

New procedure for evaluation of U(3) coupling and recoupling coefficients

A simple method to calculate Wigner coupling coefficients and Racah recoupling coefficients for U(3) in two group–subgroup chains is presented. While the canonical U(3) coupling and recoupling coefficients are applicable to any system that respects U(3) symmetry, the U(3) coupling coefficients are more specific to nuclear structure studies. This new procedure precludes the use of binomial coefficients and alternating sums which were used in the 1973 formulation of Draayer and Akiyama, and in so doing provides a faster and more accurate determination of any and all required results. The resolution of the outer multiplicity is based on the null space concept of the U(3) generators proposed by Alex et al., whereas the inner multiplicity in the angular momentum subgroup chain is obtained from the dimension of the null space of the SO(3) raising operator. It is anticipated that a C++ library will ultimately be available for determining generic coupling and recoupling coefficients associated with both the canonical and the physical group–subgroup chains of U(3).

Cross-Coupling Reaction↗