SEARCH · Search NASA

Results for “Expert Judgement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2004

Development of an Expert Judgement Elicitation and Calibration Methodology for Risk Analysis in Conceptual Vehicle Design

A comprehensive expert-judgment elicitation methodology to quantify input parameter uncertainty and analysis tool uncertainty in a conceptual launch vehicle design analysis has been developed. The ten-phase methodology seeks to obtain expert judgment opinion for quantifying uncertainties as a probability distribution so that multidisciplinary risk analysis studies can be performed. The calibration and aggregation techniques presented as part of the methodology are aimed at improving individual expert estimates, and provide an approach to aggregate multiple expert judgments into a single probability distribution. The purpose of this report is to document the methodology development and its validation through application to a reference aerospace vehicle. A detailed summary of the application exercise, including calibration and aggregation results is presented. A discussion of possible future steps in this research area is given.

Unal, Resit↗

NASA NTRS2020

Burning Embers: Towards More Transparent and Robust Climate-change Risk Assessments

The Intergovernmental Panel on Climate Change (IPCC) reports provide policy-relevant insights about climate impacts, vulnerabilities and adaptation through a process of peer-reviewed literature assessments underpinned by expert judgement. An iconic output from these assessments is the burning embers diagram, first used in the Third Assessment Report to visualize reasons for concern, which aggregate climate-change-related impacts and risks to various systems and sectors. These burning embers use colour transitions to show changes in the assessed level of risk to humans and ecosystems as a function of global mean temperature. In this Review, we outline the history and evolution of the burning embers and associated reasons for concern framework, focusing on the methodological approaches and advances. While the assessment framework and figure design have been broadly retained over time, refinements in methodology have occurred, including the consideration of different risks, use of confidence statements, more formalized protocols and standardized metrics. Comparison across reports reveals that the risk level at a given temperature has generally increased with each assessment cycle, reflecting accumulating scientific evidence. For future assessments, an explicit, transparent and systematic process of expert elicitation is needed to enhance comparability, quality and credibility of burning embers.

burning embers diagram↗

NASA NTRS2002

A Life-Cycle Cost Estimating Methodology for NASA-Developed Air Traffic Control Decision Support Tools

This paper describes the development of a life-cycle cost (LCC) estimating methodology for air traffic control Decision Support Tools (DSTs) under development by the National Aeronautics and Space Administration (NASA), using a combination of parametric, analogy, and expert opinion methods. There is no one standard methodology and technique that is used by NASA or by the Federal Aviation Administration (FAA) for LCC estimation of prospective Decision Support Tools. Some of the frequently used methodologies include bottom-up, analogy, top-down, parametric, expert judgement, and Parkinson's Law. The developed LCC estimating methodology can be visualized as a three-dimensional matrix where the three axes represent coverage, estimation, and timing. This paper focuses on the three characteristics of this methodology that correspond to the three axes.

Wang, Jianzhong Jay↗

NASA NTRS2019

Tropospheric Ozone Assessment Report: Tropospheric Ozone From 1877 to 2016, Observed Levels, Trends and Uncertainties

From the earliest observations of ozone in the lower atmosphere in the 19th century, both measurement methods and the portion of the globe observed have evolved and changed. These methods have different uncertainties and biases, and the data records differ with respect to coverage (space and time), information content, and representativeness. In this study, various ozone measurement methods and ozone datasets are reviewed and selected for inclusion in the historical record of background ozone levels, based on relationship of the measurement technique to the modern UV absorption standard, absence of interfering pollutants, representativeness of the well-mixed boundary layer and expert judgement of their credibility. There are significant uncertainties with the 19th and early 20th-century measurements related to interference of other gases. Spectroscopic methods applied before 1960 have likely underestimated ozone by as much as 11% at the surface and by about 24% in the free troposphere, due to the use of differing ozone absorption coefficients. There is no unambiguous evidence in the measurement record back to 1896 that typical mid-latitude background surface ozone values were below about 20 nmol mol–1, but there is robust evidence for increases in the temperate and polar regions of the northern hemisphere of 30–70%, with large uncertainty, between the period of historic observations, 1896–1975, and the modern period (1990–2014). Independent historical observations from balloons and aircraft indicate similar changes in the free troposphere. Changes in the southern hemisphere are much less. Regional representativeness of the available observations remains a potential source of large errors, which are difficult to quantify. The great majority of validation and intercomparison studies of free tropospheric ozone measurement methods use ECC ozonesondes as reference. Compared to UV-absorption measurements they show a modest (~1–5% ±5%) high bias in the troposphere, but no evidence of a change with time. Umkehr, lidar, and FTIR methods all show modest low biases relative to ECCs, and so, using ECC sondes as a transfer standard, all appear to agree to within one standard deviation with the modern UV-absorption standard. Other sonde types show an increase of 5–20% in sensitivity to tropospheric ozone from 1970–1995. Biases and standard deviations of satellite retrieval comparisons are often 2–3 times larger than those of other free tropospheric measurements. The lack of information on temporal changes of bias for satellite measurements of tropospheric ozone is an area of concern for long-term trend studies.

UV absorption standard↗

NASA NTRSDate not supplied

A Modified Delphi Method to Accelerate Consensus Building in Expert Judgment Elicitation

The 2017 Earth Science Decadal Survey recommends the implementation of a novel Earth Observing mission to study Aerosols, Clouds, Convection, and Precipitation. The assessment of the candidate architectures under consideration requires the use of Expert Judgment Elicitation. Some of the assessment scores are obtained through consensus among the Science Leadership Team. A modified Delphi method was developed to accelerate the consensus building process and reduce the number of cycles required to converge. This paper discusses which elements of the traditional method were modified, how the method was applied, and the impact of the modifications on generating consensus.

Expert Judgement↗

NASA NTRS1990

General method of pattern classification using the two-domain theory

Human beings judge patterns (such as images) by complex mental processes, some of which may not be known, while computing machines extract features. By representing the human judgements with simple measurements and reducing them and the machine extracted features to a common metric space and fitting them by regression, the judgements of human experts rendered on a sample of patterns may be imposed on a pattern population to provide automatic classification.

Rorvig, Mark E.↗

NASA NTRS1993

General method of pattern classification using the two-domain theory

Rorvig, Mark E.↗

NASA NTRSDate not supplied

Application of Analytical Hierarchy Process for Narrowing Down Nep Candidate Reactor Designs

NASA is currently studying the possibility of human exploration of Mars in the late 2030s timeframe using a hybrid nuclear electric propulsion (NEP)/chemical propulsion system. Present analyses indicate such a mission could require a reactor power system in the range of 2 to 6 MWe operating for approximately 3 years. At present, the Technology Readiness Levels (TRL) of key NEP technologies are still low, implying that significant technology development activities must be undertaken prior to making informed design selections for an integrated propulsion system. NASA’s Space Nuclear Propulsion (SNP) project has recently undertaken an in-depth effort to identify NEP candidate technologies and determine the required developmental work to mature these technologies to the point where they could realistically support the design and assembly of a full NEP system. A version of the Analytical Hierarchy Process is being employed to narrow the technology candidates for SNP investment planning. This multi-attribute decision-making process relies on quantified technical inputs and the judgement of Subject Matter Experts (SMEs) to evaluate technology options against key technical and programmatic Figures-of-Merit (FOM) at the subsystem and optimized system level. The process and an example evaluation of nuclear fuel/moderator combinations in a sodium heat pipe honeycomb-geometry reactor concept are described. The evaluation and analysis methodology described in this paper can be employed for pairwise comparisons between reactor concepts and technology choices as new data become available.

Dasari V Rao↗

NASA NTRS1989

Neural networks in support of manned space

Many lobbyists in Washington have argued that artificial intelligence (AI) is an alternative to manned space activity. In actuality, this is the opposite of the truth, especially as regards artificial neural networks (ANNs), that form of AI which has the greatest hope of mimicking human abilities in learning, ability to interface with sensors and actuators, flexibility and balanced judgement. ANNs and their relation to expert systems (the more traditional form of AI), and the limitations of both technologies are briefly reviewed. A Few highlights of recent work on ANNs, including an NSF-sponsored workshop on ANNs for control applications are given. Current thinking on ANNs for use in certain key areas (the National Aerospace Plane, teleoperation, the control of large structures, fault diagnostics, and docking) which may be crucial to the long term future of man in space is discussed.

Werbos, Paul J.↗

NASA NTRS2020

Orbit Selection for the Proposed Lynx Observatory Mission

The Advanced Concepts Office design team performed several analyses and trades in support of orbit selection for the proposed Lynx mission, an x-ray observatory being submitted to the Astro2020 Decadal Survey. Though the descriptions in this Technical Memorandum (TM) focus on the Lynx mission, the approach and process for selecting the final orbit is applicable to a variety of proposed science and exploration missions. To select the best orbit for the Lynx science, mission designers assembled a team of subsystem and discipline experts, in addition to mission analysts, to evaluate several candidate orbits. These discipline experts included members of the science and instrument team, power and avionics, thermal, propulsion, and environments. The goal was to clearly show the benefits and weaknesses of each orbit in the trade space and provide sound justification for the final selection. Discipline experts conducted trades and evaluated the results using a variety of methods including engineering judgement, rough estimates, and detailed calculations, and rolled the results into a final grade using a weighted grading method. The orbit options could then be ranked. The principal investigator (PI) for the mission, along with the science team, was given the task of final orbit selection. The result of the trades indicated that a halo orbit about the second Sun-Earth Lagrange point (SE-L2), similar to the planned orbit for the James Webb Space Telescope (JWST), was the best choice for the Lynx mission. Details of how the team arrived at this selection are below.

Hopkins, R. C.↗

NASA NTRSDate not supplied

Methods for Developing Successful Systems Engineers

Systems Engineering (SE) is a complex and challenging field that incorporates the knowledge of systems engineering processes, the ability to synthesize a wide-range of engineering disciplines, and the ability to lead a team of people to successfully accomplish the goals of a project. It requires hard technical skills and soft-skill leadership savvy. As a result, three main development needs are identified: 1) Knowledge of SE processes, the benefits of these processes to a project and their tailored application 2) Knowledge of a wide-range of engineering disciplines, how they interrelate in a system, and the development of sound technical judgement 3) Team leadership to direct and motivate a team of subsystem and discipline experts This paper describes the establishment of a comprehensive training and development program for Systems Engineers at NASA Ames Research Center that addresses in part each of these three areas from the perspective of the implementing manager. A variety of methods have been utilized including the establishment of a SE Community of Practice, a unique and innovative web tool, on-line videos, classroom training in NASA’s 17 Common Technical Processes, guidance on the tailored application of these processes, monthly technical talks, mentoring in both technical judgment and team leadership, and NASA’s Leadership Development Programs. While much of professional SE development must come through project experience, the approaches listed above can accelerate development. The diversity of skills required of Systems Engineering demands a multi-faceted approach to successfully train and develop this critical skill.

development↗

NASA NTRSDate not supplied

Methods for Developing Successful Systems Engineers

Systems Engineering (SE) is a complex and challenging field that incorporates the knowledge of systems engineering processes, the ability to synthesize a wide-range of engineering disciplines, and the ability to lead a team of people to successfully accomplish the goals of a project. It requires hard technical skills and soft-skill leadership savvy. As a result, three main development needs are identified: 1. Knowledge of SE processes, the benefits of these processes to a project and their tailored application 2. Knowledge of a wide-range of engineering disciplines, how they interrelate in a system, and the development of sound technical judgement 3. Team leadership to direct and motivate a team of subsystem and discipline experts This paper describes the establishment of a comprehensive training and development program for Systems Engineers at NASA Ames Research Center that addresses in part each of these three areas from the perspective of the implementing manager. A variety of methods have been utilized including the establishment of a SE Community of Practice, a unique and innovative web tool, on-line videos, classroom training in NASA’s 17 Common Technical Processes, guidance on the tailored application of these processes, monthly technical talks, mentoring in both technical judgment and team leadership, and NASA’s Leadership Development Programs. While much of professional SE development must come through project experience, the approaches listed above can accelerate development. The diversity of skills required of Systems Engineering demands a multi-faceted approach to successfully train and develop this critical skill.

development↗

NASA NTRS1972

Roles of the ground and flight crew in Apollo operations.

The functions performed by ground personnel and flight crews of Apollo missions are reviewed, covering vehicle systems operation and management, Saturn launch vehicle, command-service and lunar modules, lunar rover, communication relay unit and ground-commanded TV assembly. Details of the flight path, scientific experiments, photographic equipment and crew health monitoring are also discussed in terms of these functions. The coordinated combination of large numbers of experts and data processing equipment on the ground with the crew's capabilities to operate the spacecraft and exercise on-the-spot judgements is viewed as the backbone of the successful fulfillment of Apollo missions.

Wolfer, B. M.↗

NASA NTRS1991

Neural nets for aligning optical components in harsh environments: Beam smoothing spatial filter as an example

The goal is to develop an approach to automating the alignment and adjustment of optical measurement, visualization, inspection, and control systems. Classical controls, expert systems, and neural networks are three approaches to automating the alignment of an optical system. Neural networks were chosen for this project and the judgements that led to this decision are presented. Neural networks were used to automate the alignment of the ubiquitous laser-beam-smoothing spatial filter. The results and future plans of the project are presented.

Decker, Arthur J.↗

NASA NTRS2000

Multimodality Instrument for Tissue Characterization

A system with multimodality instrument for tissue identification includes a computer-controlled motor driven heuristic probe with a multisensory tip is discussed. For neurosurgical applications, the instrument is mounted on a stereotactic frame for the probe to penetrate the brain in a precisely controlled fashion. The resistance of the brain tissue being penetrated is continually monitored by a miniaturized strain gauge attached to the probe tip. Other modality sensors may be mounted near the probe tip to provide real-time tissue characterizations and the ability to detect the proximity of blood vessels, thus eliminating errors normally associated with registration of pre-operative scans, tissue swelling, elastic tissue deformation, human judgement, etc., and rendering surgical procedures safer, more accurate, and efficient. A neural network, program adaptively learns the information on resistance and other characteristic features of normal brain tissue during the surgery and provides near real-time modeling. A fuzzy logic interface to the neural network program incorporates expert medical knowledge in the learning process. Identification of abnormal brain tissue is determined by the detection of change and comparison with previously learned models of abnormal brain tissues. The operation of the instrument is controlled through a user friendly graphical interface. Patient data is presented in a 3D stereographics display. Acoustic feedback of selected information may optionally be provided. Upon detection of the close proximity to blood vessels or abnormal brain tissue, the computer-controlled motor immediately stops probe penetration.

Mah, Robert W.↗

NASA NTRS2004

Multimodality instrument for tissue characterization

A system with multimodality instrument for tissue identification includes a computer-controlled motor driven heuristic probe with a multisensory tip. For neurosurgical applications, the instrument is mounted on a stereotactic frame for the probe to penetrate the brain in a precisely controlled fashion. The resistance of the brain tissue being penetrated is continually monitored by a miniaturized strain gauge attached to the probe tip. Other modality sensors may be mounted near the probe tip to provide real-time tissue characterizations and the ability to detect the proximity of blood vessels, thus eliminating errors normally associated with registration of pre-operative scans, tissue swelling, elastic tissue deformation, human judgement, etc., and rendering surgical procedures safer, more accurate, and efficient. A neural network program adaptively learns the information on resistance and other characteristic features of normal brain tissue during the surgery and provides near real-time modeling. A fuzzy logic interface to the neural network program incorporates expert medical knowledge in the learning process. Identification of abnormal brain tissue is determined by the detection of change and comparison with previously learned models of abnormal brain tissues. The operation of the instrument is controlled through a user friendly graphical interface. Patient data is presented in a 3D stereographics display. Acoustic feedback of selected information may optionally be provided. Upon detection of the close proximity to blood vessels or abnormal brain tissue, the computer-controlled motor immediately stops probe penetration. The use of this system will make surgical procedures safer, more accurate, and more efficient. Other applications of this system include the detection, prognosis and treatment of breast cancer, prostate cancer, spinal diseases, and use in general exploratory surgery.

Mah, Robert W.↗

NASA NTRS1994

Decision Making in Action: Applying Research to Practice

The importance of decision-making to safety in complex, dynamic environments like mission control centers, aviation, and offshore installations has been well established. NASA-ARC has a program of research dedicated to fostering safe and effective decision-making in the manned spaceflight environment: Because access to spaceflight is limited, environments with similar characteristics, including aviation and nuclear power plants, serve as analogs from which space-relevant data can be gathered and theories developed. Analyses of aviation accidents cite crew judgement and decision making as causes or contributing factors in over half of all accidents. Yet laboratory research on decision making has not proven especially helpful in improving the quality of decisions in these kinds of environments. One reason is that the traditional, analytic decision models are inappropriate to multi-dimensional, high-risk environments, and do not accurately describe what expert human decision makers do when they make decisions that have consequences. A new model of dynamic, naturalistic decision making is offered that may prove useful for improving decision making in complex, isolated, confined and high-risk environments. Based on analyses of crew performance in full-mission simulators and accident reports, features that define effective decision strategies in abnormal or emergency situations have been identified. These include accurate situation assessment (including time and risk assessment), appreciation of the complexity of the problem, sensitivity to constraints on the decision, timeliness of the response, and use of adequate information. More effective crews also manage their workload to provide themselves with time and resources to make good good decisions are appropriate to the demands of the situation. Effective crew decision making and overall performance are mediated by crew communication. Communication contributes to performance because it assures that all crew members have essential information, but it also regulates and coordinates crew actions and is the medium of collective thinking in response to a problem. This presentation will examine the relations between leadership, communication, decision making and overall crew performance. Implications of these findings for training will be discussed.

Orasanu, Judith↗

NASA NTRS1995

Decision Making in the Airplane

The Importance of decision-making to safety in complex, dynamic environments like mission control centers, aviation, and offshore installations has been well established. NASA-ARC has a program of research dedicated to fostering safe and effective decision-making in the manned spaceflight environment. Because access to spaceflight is limited, environments with similar characteristics, including aviation and nuclear power plants, serve as analogs from which space-relevant data can be gathered and theories developed. Analyses of aviation accidents cite crew judgement and decision making as causes or contributing factors in over half of all accidents. Yet laboratory research on decision making has not proven especially helpful In improving the quality of decisions in these kinds of environments. One reason is that the traditional, analytic decision models are inappropriate to multi-dimensional, high-risk environments, and do not accurately describe what expert human decision makers do when they make decisions that have consequences. A new model of dynamic, naturalistic decision making is offered that may prove useful for improving decision making in complex, isolated, confined and high-risk environments. Based on analyses of crew performance in full-mission simulators and accident reports, features that define effective decision strategies in abnormal or emergency situations have been identified. These include accurate situation assessment (including time and risk assessment), appreciation of the complexity of the problem, sensitivity to constraints on the decision, timeliness of the response, and use of adequate information. More effective crews also manage their workload to provide themselves with time and resources to make good decisions. In brief, good decisions are appropriate to the demands of the situation. Effective crew decision making and overall performance are mediated by crew communication. Communication contributes to performance because it assures that all crew members have essential information, but it also regulates and coordinates crew actions and is the medium of collective thinking In response to a problem, This presentation will examine the relations between leadership, communication, decision making and overall crew performance. Implications of these findings for training will be discussed.

Orasanu, Judith↗