SEARCH · Search NASA

Results for “Expert Judgment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

33 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS1976

Cost estimation: An expert-opinion approach

A methodology is outlined which can be used to estimate the costs of research and development projects. The approach uses the Delphi technique a method developed by the Rand Corporation for systematically eliciting and evaluating group judgments in an objective manner. The use of the Delphi allows for the integration of expert opinion into the cost-estimating process in a consistent and rigorous fashion. This approach can also signal potential cost-problem areas. This result can be a useful tool in planning additional cost analysis or in estimating contingency funds. A Monte Carlo approach is also examined.

Buffalano, C.↗

NASA NTRS2005

Developing a Methodology for Eliciting Subjective Probability Estimates During Expert Evaluations of Safety Interventions: Application for Bayesian Belief Networks

The NASA Aviation Safety Program (AvSP) has defined several products that will potentially modify airline and/or ATC operations, enhance aircraft systems, and improve the identification of potential hazardous situations within the National Airspace System (NAS). Consequently, there is a need to develop methods for evaluating the potential safety benefit of each of these intervention products so that resources can be effectively invested to produce the judgments to develop Bayesian Belief Networks (BBN's) that model the potential impact that specific interventions may have. Specifically, the present report summarizes methodologies for improving the elicitation of probability estimates during expert evaluations of AvSP products for use in BBN's. The work involved joint efforts between Professor James Luxhoj from Rutgers University and researchers at the University of Illinois. The Rutgers' project to develop BBN's received funding by NASA entitled "Probabilistic Decision Support for Evaluating Technology Insertion and Assessing Aviation Safety System Risk." The proposed project was funded separately but supported the existing Rutgers' program.

Wiegmann, Douglas A.a↗

NASA NTRS2015

Effects of Acute Stress on Aircrew Performance: Literature Review and Analysis of Operational Aspects

Situational stress can adversely affect the cognition and skilled performance of pilots, as well as experts in other domains. Emergencies and other threatening situations require pilots to execute infrequently practiced procedures correctly and to use their skills and judgment to select an appropriate course of action, often under high workload, time pressure, and ambiguous indications, all of which can be stressful. Our current study, consited of three parts, starting with a critical review of the research literature on the effects of stress on skilled performance, going back to World War II and continuing to recent and more sophisticated studies of the cognitive effects of anxiety. In the second part we analyzed the specific ways stress may have impaired the performance of airline crews in twelve major accidents, selected for diversity of the situations the crews encountered. The third part examined the operational significance and practical implications of the findings from the first two parts, suggested specific ways to reduce the harmful effects of stress on flight crews, and identified aspects requiring further research. Even thought this study focused on flight crews, the findings apply to the effects of stress on the skilled performance of experts in almost any domain.

stress↗

NASA NTRSDate not supplied

LSKnowledge: Nexus for Transformative Scientific Discoveries and Enhanced Information Retrieval in NASA Life Sciences Portal

We stand at the brink of an extraordinary transformation in the field of AI, driven by the convergence of generative AI and semantic technologies (e.g., knowledge graphs). This fusion holds immense potential and could redefine the future of scientific exploration, particularly in the realm of life sciences research. In this context, we shed light on the pivotal roles that Large Language Models (LLMs) and semantic technologies will play in advancing research, unearthing and comprehending life sciences information through innovative approaches, and empowering researchers to extract insights from NASA's extensive Life Sciences Data Archive. Within the NASA Life Sciences Portal (NLSP), the integration of LLMs and semantic technologies unlocks several advanced capabilities. First and foremost, it equips scientists with sophisticated tools to manage the ever-expanding wealth of scientific literature and data. Furthermore, it facilitates the creation of knowledge graphs that visually represent intricate relationships among biological entities, enabling comprehensive systems-level analysis. Additionally, the fusion of generative AI (including LLMs) and semantic technology can significantly benefit NASA's life sciences research by enhancing information retrieval and hypothesis generation. These tools enhance natural language understanding, facilitating knowledge discovery within NLSP. The overarching vision is to establish a cohesive knowledge ecosystem within NLSP, harnessing the power of LLMs and semantic technologies to synthesize and cross-reference data from diverse missions, disciplines, and research domains. This holistic approach ultimately deepens our understanding of how space environments impact life sciences data. To advance this initiative, we have launched LSKnowledge, aimed at enhancing the information retrieval capabilities of NLSP. In the short term, our primary goal is to develop a robust semantic search system. This system will empower HRP (Human Research Program) researchers to navigate NLSP data repositories more efficiently and precisely, catalyzing the process of hypothesis formation and scientific breakthroughs. To achieve this, we have employed pre-trained LLMs as part of a semantic search tool that can rank and highlight the most relevant records for user queries. To assess the tool's performance, we have curated a set of approximately 200 queries from subject matter experts (SMEs) and manually ranked the top records retrieved by both the current search system and the new semantic search, using SME judgments as the gold standard for relevancy. Herein, we present the results of our comparative analysis and illustrate how these findings have informed the fine-tuning of the system for enhanced performance. In the long term, our objectives include 1) retrieving publicly available information and integrating it with NLSP data to provide more precise answers to user queries, and 2) incorporating non-textual information from the NLSP database into our approach. In conclusion, the fusion of LLMs and semantic technologies within NLSP represents a pioneering stride towards reshaping the landscape of scientific discovery. This synergy not only equips researchers with powerful tools to navigate the burgeoning sea of information but also facilitates a deeper understanding of complex biological relationships, all while accelerating hypothesis generation and knowledge discovery. Through our initiative, LSKnowledge, we are committed to continually refining and expanding these capabilities, with the aim of not only enhancing information retrieval but also integrating diverse data sources to provide more precise insights. In the grand vision, NLSP strives to become the cornerstone of a comprehensive knowledge ecosystem, unraveling the enigmatic intricacies of life sciences phenomena in the context of space environments.

Life Sciences↗

NASA NTRS1978

Satellite Power System (SPS): an Overview of Prospective Organizational Structures in the Solar Satellite Field

A literature survey, interviews with acknowledged experts in the fields of organizational entities, space, solar energy, and the SPS concept, and an analysis of these inputs to identify the organizational alternatives and make judgments as to their feasibility to serve as patterns for a future SPS entity are presented. Selection and evaluation criteria were determined to include timeliness, reliability, and adequacy to contribute meaningfully to the U.S. supply; political feasibility (both national and international) and cost-effectiveness (including environmental and other external costs). Based on these criteria, four organizational alternatives are discussed which offer reasonable promise as potential options for SPS. These included three domestic alternatives and one international alternative.

Edler, H. G.↗

NASA NTRS1992

An expert system for integrated structural analysis and design optimization for aerospace structures

The results of a research study on the development of an expert system for integrated structural analysis and design optimization is presented. An Object Representation Language (ORL) was developed first in conjunction with a rule-based system. This ORL/AI shell was then used to develop expert systems to provide assistance with a variety of structural analysis and design optimization tasks, in conjunction with procedural modules for finite element structural analysis and design optimization. The main goal of the research study was to provide expertise, judgment, and reasoning capabilities in the aerospace structural design process. This will allow engineers performing structural analysis and design, even without extensive experience in the field, to develop error-free, efficient and reliable structural designs very rapidly and cost-effectively. This would not only improve the productivity of design engineers and analysts, but also significantly reduce time to completion of structural design. An extensive literature survey in the field of structural analysis, design optimization, artificial intelligence, and database management systems and their application to the structural design process was first performed. A feasibility study was then performed, and the architecture and the conceptual design for the integrated 'intelligent' structural analysis and design optimization software was then developed. An Object Representation Language (ORL), in conjunction with a rule-based system, was then developed using C++. Such an approach would improve the expressiveness for knowledge representation (especially for structural analysis and design applications), provide ability to build very large and practical expert systems, and provide an efficient way for storing knowledge. Functional specifications for the expert systems were then developed. The ORL/AI shell was then used to develop a variety of modules of expert systems for a variety of modeling, finite element analysis, and design optimization tasks in the integrated aerospace structural design process. These expert systems were developed to work in conjunction with procedural finite element structural analysis and design optimization modules (developed in-house at SAT, Inc.). The complete software, AutoDesign, so developed, can be used for integrated 'intelligent' structural analysis and design optimization. The software was beta-tested at a variety of companies, used by a range of engineers with different levels of background and expertise. Based on the feedback obtained by such users, conclusions were developed and are provided.

Source record↗

NASA NTRS2008

Dawn: Cooperation, not Control

On September 27, 2007, a Delta II rocket carrying the Dawn spacecraft lifted off from Kennedy Space Center. Part of NASAs Discovery program, the $370 million Dawn mission began its three-billion-mile voyage to the asteroid belt to study the asteroid Vesta and Ceres, a dwarf planet. The spacecraft is scheduled to reach Vesta in 2011. After spending nine months measuring the composition, shape, and topography of that body, it will travel a billion miles to carry out a similar analysis of Ceres in 2015. The Important Lessons: The demands of Dawn and other challenging missions have taught some important lessons for successful program and project management. These are the main ones: a) Program management, particularly of uncoupled and loosely coupled projects, should be more about enabling than controlling. You're working with motivated, high-performing teams and institutions with a track record of quality and success. Emphasize commander's intent over rudder control; let them know where you want to go and when you want to be there, then let them figure out how to get there. b) Open and honest discussion of issues is essential. People fill the void of the unknown with their worst fears. Get folks around the table and have open, honest, and frank dialogue. I've seldom seen this fail to get to the root of issues. c) You have to earn your seat at the table, proving that you are competent, trustworthy, and dedicated to the success of the mission. d) Know when to fold 'em. Your pride can get rolled up in making a milestone or launch date, but you have to make a judgment based on the realities of the situation and not wear down the team trying to meet an increasingly impossible deadline. e) The NASA governance model that gives a voice to the concerns of engineers and safety experts works-trust it and use it.

May, Todd↗

NASA NTRS1989

Tracking and data relay satellite fault isolation and correction using PACES: Power and attitude control expert system

The Power and Attitude Control Expert System (PACES) is an object oriented and rule based expert system which provides spacecraft engineers with assistance in isolating and correcting problems within the Power and Attitude Control Subsystems of the Tracking and Data Relay Satellites (TDRS). PACES is designed to act in a consultant role. It will not interface to telemetry data, thus preserving full operator control over spacecraft operations. The spacecraft engineer will input requested information. This information will include telemetry data, action being performed, problem characteristics, spectral characteristics, and judgments of spacecraft functioning. Questions are answered either by clicking on appropriate responses (for text), or entering numeric values. A context sensitive help facility allows access to additional information when the user has difficulty understanding a question or deciding on an answer. The major functionality of PACES is to act as a knowledge rich system which includes block diagrams, text, and graphics, linked using hypermedia techniques. This allows easy movement among pieces of the knowledge. Considerable documentation of the spacecraft Power and Attitude Control Subsystems is embedded within PACES. The development phase of TDRSS expert system technology is intended to provide NASA with the necessary expertise and capability to define requirements, evaluate proposals, and monitor the development progress of a highly competent expert system for NASA's Tracking and Data Relay Satellite Program.

Erikson, Carol-Lee↗

NASA NTRS2009

Human Research Program Space Human Factors Engineering (SHFE) Standing Review Panel (SRP)

The Space Human Factors Engineering (SHFE) Standing Review Panel (SRP) evaluated 22 gaps and 39 tasks in the three risk areas assigned to the SHFE Project. The area where tasks were best designed to close the gaps and the fewest gaps were left out was the Risk of Reduced Safety and Efficiency dire to Inadequate Design of Vehicle, Environment, Tools or Equipment. The areas where there were more issues with gaps and tasks, including poor or inadequate fit of tasks to gaps and missing gaps, were Risk of Errors due to Poor Task Design and Risk of Error due to Inadequate Information. One risk, the Risk of Errors due to Inappropriate Levels of Trust in Automation, should be added. If astronauts trust automation too much in areas where it should not be trusted, but rather tempered with human judgment and decision making, they will incur errors. Conversely, if they do not trust automation when it should be trusted, as in cases where it can sense aspects of the environment such as radiation levels or distances in space, they will also incur errors. This will be a larger risk when astronauts are less able to rely on human mission control experts and are out of touch, far away, and on their own. The SRP also identified 11 new gaps and five new tasks. Although the SRP had an extremely large quantity of reading material prior to and during the meeting, we still did not feel we had an overview of the activities and tasks the astronauts would be performing in exploration missions. Without a detailed task analysis and taxonomy of activities the humans would be engaged in, we felt it was impossible to know whether the gaps and tasks were really sufficient to insure human safety, performance, and comfort in the exploration missions. The SRP had difficulty evaluating many of the gaps and tasks that were not as quantitative as those related to concrete physical danger such as excessive noise and vibration. Often the research tasks for cognitive risks that accompany poor task or information design addressed only part, but not all, of the gaps they were programmed to fill. In fact the tasks outlined will not close the gap but only scratch the surface in many cases. In other cases, the gap was written too broadly, and really should be restated in a more constrained way that can be addressed by a well-organized and complementary set of tasks. In many cases, the research results should be turned into guidelines for design. However, it was not clear whether the researchers or another group would construct and deliver these guidelines.

Wichansky, Anna↗

NASA NTRS1994

Shared Problem Models and Crew Decision Making

The importance of crew decision making to aviation safety has been well established through NTSB accident analyses: Crew judgment and decision making have been cited as causes or contributing factors in over half of all accidents in commercial air transport, general aviation, and military aviation. Yet the bulk of research on decision making has not proven helpful in improving the quality of decisions in the cockpit. One reason is that traditional analytic decision models are inappropriate to the dynamic complex nature of cockpit decision making and do not accurately describe what expert human decision makers do when they make decisions. A new model of dynamic naturalistic decision making is offered that may prove more useful for training or aiding cockpit decision making. Based on analyses of crew performance in full-mission simulation and National Transportation Safety Board accident reports, features that define effective decision strategies in abnormal or emergency situations have been identified. These include accurate situation assessment (including time and risk assessment), appreciation of the complexity of the problem, sensitivity to constraints on the decision, timeliness of the response, and use of adequate information. More effective crews also manage their workload to provide themselves with time and resources to make good decisions. In brief, good decisions are appropriate to the demands of the situation and reflect the crew's metacognitive skill. Effective crew decision making and overall performance are mediated by crew communication. Communication contributes to performance because it assures that all crew members have essential information, but it also regulates and coordinates crew actions and is the medium of collective thinking in response to a problem. This presentation will examine the relation between communication that serves to build performance. Implications of these findings for crew training will be discussed.

Orasanu, Judith↗

NASA NTRSDate not supplied

Methods for Developing Successful Systems Engineers

Systems Engineering (SE) is a complex and challenging field that incorporates the knowledge of systems engineering processes, the ability to synthesize a wide-range of engineering disciplines, and the ability to lead a team of people to successfully accomplish the goals of a project. It requires hard technical skills and soft-skill leadership savvy. As a result, three main development needs are identified: 1) Knowledge of SE processes, the benefits of these processes to a project and their tailored application 2) Knowledge of a wide-range of engineering disciplines, how they interrelate in a system, and the development of sound technical judgement 3) Team leadership to direct and motivate a team of subsystem and discipline experts This paper describes the establishment of a comprehensive training and development program for Systems Engineers at NASA Ames Research Center that addresses in part each of these three areas from the perspective of the implementing manager. A variety of methods have been utilized including the establishment of a SE Community of Practice, a unique and innovative web tool, on-line videos, classroom training in NASA’s 17 Common Technical Processes, guidance on the tailored application of these processes, monthly technical talks, mentoring in both technical judgment and team leadership, and NASA’s Leadership Development Programs. While much of professional SE development must come through project experience, the approaches listed above can accelerate development. The diversity of skills required of Systems Engineering demands a multi-faceted approach to successfully train and develop this critical skill.

development↗

NASA NTRSDate not supplied

Methods for Developing Successful Systems Engineers

Systems Engineering (SE) is a complex and challenging field that incorporates the knowledge of systems engineering processes, the ability to synthesize a wide-range of engineering disciplines, and the ability to lead a team of people to successfully accomplish the goals of a project. It requires hard technical skills and soft-skill leadership savvy. As a result, three main development needs are identified: 1. Knowledge of SE processes, the benefits of these processes to a project and their tailored application 2. Knowledge of a wide-range of engineering disciplines, how they interrelate in a system, and the development of sound technical judgement 3. Team leadership to direct and motivate a team of subsystem and discipline experts This paper describes the establishment of a comprehensive training and development program for Systems Engineers at NASA Ames Research Center that addresses in part each of these three areas from the perspective of the implementing manager. A variety of methods have been utilized including the establishment of a SE Community of Practice, a unique and innovative web tool, on-line videos, classroom training in NASA’s 17 Common Technical Processes, guidance on the tailored application of these processes, monthly technical talks, mentoring in both technical judgment and team leadership, and NASA’s Leadership Development Programs. While much of professional SE development must come through project experience, the approaches listed above can accelerate development. The diversity of skills required of Systems Engineering demands a multi-faceted approach to successfully train and develop this critical skill.

development↗

NASA NTRS1997

Alternative Perspectives on Risk: Individual Differences in Problem Structuring

Team decision making involves contributions of multiple players toward a common goal. While much has been written about the importance of developing shared mental models in order for teams to work together effectively, little has been done to determine the value of alternative perspectives on problem solving and decision making. Early studies of expertise contrasted experts with novices and noted that the two groups differ in the way they structure problems and in their selection of information as salient. Little attention has been given to differences among experts who differ in their specializations. A series of experiments was conducted to determine: (1) what dimensions of flight-related problem situations pilots judge to be most important when making flight-relevant decisions; and (2) whether pilots in different crew positions differ in the way they interpret problems relating to flight decisions. A sorting task was used to identify underlying dimensions judged as salient to individual pilots. Captains, first officers, and flight engineers from two major carriers participated in the study. Twenty-two flight scenarios were developed based on ASRS reports. Pilots were required to make judgments about how they would respond in each case and to sort the scenarios on the basis of similarity of decision factors. They were also asked to provide a verbal label that described each of their sorted categories. A second study required a different group of pilots (also captains, first officers and flight engineers) to sort on predetermined bases.

Orasanu, Judith↗

NASA NTRS1993

Designers' models of the human-computer interface

Understanding design models of the human-computer interface (HCI) may produce two types of benefits. First, interface development often requires input from two different types of experts: human factors specialists and software developers. Given the differences in their backgrounds and roles, human factors specialists and software developers may have different cognitive models of the HCI. Yet, they have to communicate about the interface as part of the design process. If they have different models, their interactions are likely to involve a certain amount of miscommunication. Second, the design process in general is likely to be guided by designers' cognitive models of the HCI, as well as by their knowledge of the user, tasks, and system. Designers do not start with a blank slate; rather they begin with a general model of the object they are designing. The author's approach to a design model of the HCI was to have three groups make judgments of categorical similarity about the components of an interface: human factors specialists with HCI design experience, software developers with HCI design experience, and a baseline group of computer users with no experience in HCI design. The components of the user interface included both display components such as windows, text, and graphics, and user interaction concepts, such as command language, editing, and help. The judgments of the three groups were analyzed using hierarchical cluster analysis and Pathfinder. These methods indicated, respectively, how the groups categorized the concepts, and network representations of the concepts for each group. The Pathfinder analysis provides greater information about local, pairwise relations among concepts, whereas the cluster analysis shows global, categorical relations to a greater extent.

Gillan, Douglas J.↗

NASA NTRS2019

Human Performance Contributions to Safety in Commercial Aviation

Every day in aviation, pilots, air traffic controllers, and other front-line personnel perform countless correct judgments and actions in a variety of operational environments. These judgments and actions are often the difference between an accident and a non-event. Ironically, data on these behaviors are rarely collected or analyzed. Data-driven decisions about safety management and design of safety-critical systems are limited by the available data, which influence how decision makers characterize problems and identify solutions. Large volumes of data are collected on the failures and errors that result in infrequent incidents and accidents, but in the absence of data on behaviors that result in routine successful outcomes, safety management and system design decisions are based on a small sample of nonrepresentative safety data. This assessment aimed to find and document “safety successes” made possible by human operators. With many Aeronautics Research Mission Directorate (ARMD) Programs and Projects focusing on increased automation and autonomy and decreased human involvement, failure to fully consider the human contributions to successful system performance in civil aviation represents a significant risk — a risk that has not been recognized to date. Without understanding how humans contribute to safety, any estimate of predicted safety of autonomous capabilities is incomplete and inherently suspect. Furthermore, understanding the ways in which humans contribute to safety can promote strategic interactions among safety technologies, functions, procedures and the people using them. Without this understanding, the full benefits of an integrated, optimized human/technology or autonomous system will not be realized. Historically, safety has been consistently defined in terms of the occurrence of accidents or recognized risks (i.e., in terms of things that go wrong). These adverse outcomes are explained by identifying their causes, and safety is restored by eliminating or mitigating these causes. An alternative to this approach is to focus on what goes right and identify how to replicate that process. Focusing on the rare cases of failures attributed to “human error” provides little information about why human performance routinely prevents adverse events. Hollnagel has proposed that things go right because people continuously adjust their work to match their operating conditions. These adjustments become increasingly important as systems continue to grow in complexity. Thus, the definition of safety should reflect not only “avoiding things that go wrong” but “ensuring that things go right.” The basis for safety management requires developing an understanding of everyday activities. However, few mechanisms to monitor everyday work exist in the aviation domain, which limits opportunities to learn how designs function in reality. This concept of safety thinking and safety management is reflected in the emerging field of resilience engineering. According to Hollnagel, a system is resilient if it can sustain required operations under expected and unexpected conditions by adjusting its functioning prior to, during, or following changes, disturbances, and opportunities. To explore “positive” behaviors that contribute to resilient performance in commercial aviation, the assessment team examined a range of existing sources of data about pilot and air traffic control (ATC) tower controller performance, including subjective interviews with domain experts and objective aircraft flight data records. These data were used to identify strategies that support resilient performance, methods for exploring and refining those strategies in existing data, and proposed methods for capturing and analyzing new data.

Null, Cynthia H.↗