Results for “Decision making under uncertainty” — Page 2

NASA NTRS2014

Uncertainty in Agricultural Impact Assessment

This chapter considers issues concerning uncertainty associated with modeling and its use within agricultural impact assessments. Information about uncertainty is important for those who develop assessment methods, since that information indicates the need for, and the possibility of, improvement of the methods and databases. Such information also allows one to compare alternative methods. Information about the sources of uncertainties is an aid in prioritizing further work on the impact assessment method. Uncertainty information is also necessary for those who apply assessment methods, e.g., for projecting climate change impacts on agricultural production and for stakeholders who want to use the results as part of a decision-making process (e.g., for adaptation planning). For them, uncertainty information indicates the degree of confidence they can place in the simulated results. Quantification of uncertainty also provides stakeholders with an important guideline for making decisions that are robust across the known uncertainties. Thus, uncertainty information is important for any decision based on impact assessment. Ultimately, we are interested in knowledge about uncertainty so that information can be used to achieve positive outcomes from agricultural modeling and impact assessment.

damage assessment↗

NASA NTRS1977

The Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making Under Dynamic Uncertainity

This note shows that the optimal control of dynamic systems with uncertain parameters has certain limitations. In particular, by means of a simple scalar linear-quadratic optimal control example, it is shown that the infinite horizon solution does not exist if the parameter uncertainty exceeds a certain quantifiable threshold; we call this the uncertainty threshold principle. The philosophical and design implications of this result are discussed.

Athans, M.↗

NASA NTRS2017

Uncertainty Estimation Cheat Sheet for Probabilistic Risk Assessment

Quantitative results for aerospace engineering problems are influenced by many sources of uncertainty. Uncertainty analysis aims to make a technical contribution to decision-making through the quantification of uncertainties in the relevant variables as well as through the propagation of these uncertainties up to the result. Uncertainty can be thought of as a measure of the 'goodness' of a result and is typically represented as statistical dispersion. This paper will explain common measures of centrality and dispersion; and-with examples-will provide guidelines for how they may be estimated to ensure effective technical contributions to decision-making.

Britton, Paul↗

NASA NTRS2017

Uncertainty Estimation Cheat Sheet for Probabilistic Risk Assessment

"Uncertainty analysis itself is uncertain, therefore, you cannot evaluate it exactly," Source Uncertain Quantitative results for aerospace engineering problems are influenced by many sources of uncertainty. Uncertainty analysis aims to make a technical contribution to decision-making through the quantification of uncertainties in the relevant variables as well as through the propagation of these uncertainties up to the result. Uncertainty can be thought of as a measure of the 'goodness' of a result and is typically represented as statistical dispersion. This paper will explain common measures of centrality and dispersion; and-with examples-will provide guidelines for how they may be estimated to ensure effective technical contributions to decision-making.

Britton, Paul T.↗

NASA NTRS2017

Lognormal Uncertainty Estimation for Failure Rates

"Uncertainty analysis itself is uncertain, therefore, you cannot evaluate it exactly," Source Uncertain. Quantitative results for aerospace engineering problems are influenced by many sources of uncertainty. Uncertainty analysis aims to make a technical contribution to decision-making through the quantification of uncertainties in the relevant variables as well as through the propagation of these uncertainties up to the result. Uncertainty can be thought of as a measure of the 'goodness' of a result and is typically represented as statistical dispersion. This presentation will explain common measures of centrality and dispersion; and-with examples-will provide guidelines for how they may be estimated to ensure effective technical contributions to decision-making.

Britton, Paul T.↗

NASA NTRS2000

Integration of Planetary Protection Activities

Research and activities under this grant have focused on a systematic examination and analysis of critical questions likely to impact planetary protection (PP) controls and implementation for Mars sample return missions (MSR). Four areas in the non-scientific and social realms were selected for special attention because of their importance to future mission planning and concern about critical timing or possible economic impacts on MSR mission implementation. These include: (1) questions of legal uncertainty and the decision making process, (2) public perception of risks associated with sample return, (3) risk communication and Education/Public Outreach , and (4) planetary protection implications of alternative mission architectures, for both robotic and human sample return missions. In its entirety, NAG 2-986 has encompassed three categories of activity: (1) research and analysis (Race), (2) subcontracted research (MacGregor/Decision Research), and (3) consulting services.

Race, Margaret S.↗

NASA NTRS2010

World, We Have Problems: Simulation for Large Complex, Risky Projects, and Events

Prior to a spacewalk during the NASA STS/129 mission in November 2009, Columbia Broadcasting System (CBS) correspondent William Harwood reported astronauts, "were awakened again", as they had been the day previously. Fearing something not properly connected was causing a leak, the crew, both on the ground and in space, stopped and checked everything. The alarm proved false. The crew did complete its work ahead of schedule, but the incident reminds us that correctly connecting hundreds and thousands of entities, subsystems and systems, finding leaks, loosening stuck valves, and adding replacements to very large complex systems over time does not occur magically. Everywhere major projects present similar pressures. Lives are at - risk. Responsibility is heavy. Large natural and human-created disasters introduce parallel difficulties as people work across boundaries their countries, disciplines, languages, and cultures with known immediate dangers as well as the unexpected. NASA has long accepted that when humans have to go where humans cannot go that simulation is the sole solution. The Agency uses simulation to achieve consensus, reduce ambiguity and uncertainty, understand problems, make decisions, support design, do planning and troubleshooting, as well as for operations, training, testing, and evaluation. Simulation is at the heart of all such complex systems, products, projects, programs, and events. Difficult, hazardous short and, especially, long-term activities have a persistent need for simulation from the first insight into a possibly workable idea or answer until the final report perhaps beyond our lifetime is put in the archive. With simulation we create a common mental model, try-out breakdowns of machinery or teamwork, and find opportunity for improvement. Lifecycle simulation proves to be increasingly important as risks and consequences intensify. Across the world, disasters are increasing. We anticipate more of them, as the results of global warming prove more and more ominous-glaciers melting in Bolivia, floods in Saudi Arabia, the Maldives sinking and salt rising along the Nile. Fear grows about potential asteroid crashes and nightly television images raise awareness of victims of floods, hurricanes, cyclones and typhoons, fire, tornado, tsunami, bombings, landslides, and cross-boundary criminality. The Red Cross says that disasters impact 250 million people each year. That means that 700,000 people are having a very bad day today. Modeling and simulation is and must be part of the solution.

Elfrey, Priscilla↗

NASA NTRS2020

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been evolving independently of each other, over the past decade. In this paper, we report our ongoing effort to integrate these two presently distinct areas into a single framework. To this end, we propose the two-player partially observable stochastic game formalism to capture both high-level autonomous mission planning under uncertainty and adversarial decision making subject to imperfect information. We show that synthesizing sub-optimal strategies for such games is possible under finite-memory assumptions for both the autonomous decision maker and the cyber-adversary. We then describe an experimental testbed to evaluate the efficacy of the proposed framework.

Ames, Aaron D.↗

NASA NTRS2011

Risk Assessment Overview

Risk assessment is used in many industries to identify and manage risks. Initially developed for use on aeronautical and nuclear systems, risk assessment has been applied to transportation, chemical, computer, financial, and security systems among others. It is used to gain an understanding of the weaknesses or vulnerabilities in a system so modification can be made to increase operability, efficiency, and safety and to reduce failure and down-time. Risk assessment results are primary inputs to risk-informed decision making; where risk information including uncertainty is used along with other pertinent information to assist management in the decision-making process. Therefore, to be useful, a risk assessment must be directed at specific objectives. As the world embraces the globalization of trade and manufacturing, understanding the associated risk become important to decision making. Applying risk assessment techniques to a global system of development, manufacturing, and transportation can provide insight into how the system can fail, the likelihood of system failure and the consequences of system failure. The risk assessment can identify those elements that contribute most to risk and identify measures to prevent and mitigate failures, disruptions, and damaging outcomes. In addition, risk associated with public and environment impact can be identified. The risk insights gained can be applied to making decisions concerning suitable development and manufacturing locations, supply chains, and transportation strategies. While risk assessment has been mostly applied to mechanical and electrical systems, the concepts and techniques can be applied across other systems and activities. This paper provides a basic overview of the development of a risk assessment.

Prassinos, Peter G.↗

NASA NTRS2017

Prospective Architectures for Onboard vs Cloud-Based Decision Making for Unmanned Aerial Systems

This paper investigates propsective architectures for decision-making in unmanned aerial systems. When these unmanned vehicles operate in urban environments, there are several sources of uncertainty that affect their behavior, and decision-making algorithms need to be robust to account for these different sources of uncertainty. It is important to account for several risk-factors that affect the flight of these unmanned systems, and facilitate decision-making by taking into consideration these various risk-factors. In addition, there are several technical challenges related to autonomous flight of unmanned aerial systems; these challenges include sensing, obstacle detection, path planning and navigation, trajectory generation and selection, etc. Many of these activities require significant computational power and in many situations, all of these activities need to be performed in real-time. In order to efficiently integrate these activities, it is important to develop a systematic architecture that can facilitate real-time decision-making. Four prospective architectures are discussed in this paper; on one end of the spectrum, the first architecture considers all activities/computations being performed onboard the vehicle whereas on the other end of the spectrum, the fourth and final architecture considers all activities/computations being performed in the cloud, using a new service known as Prognostics as a Service that is being developed at NASA Ames Research Center. The four different architectures are compared, their advantages and disadvantages are explained and conclusions are presented.

Software engineering↗

NASA NTRS2003

Exploration of Advanced Probabilistic and Stochastic Design Methods

The primary objective of the three year research effort was to explore advanced, non-deterministic aerospace system design methods that may have relevance to designers and analysts. The research pursued emerging areas in design methodology and leverage current fundamental research in the area of design decision-making, probabilistic modeling, and optimization. The specific focus of the three year investigation was oriented toward methods to identify and analyze emerging aircraft technologies in a consistent and complete manner, and to explore means to make optimal decisions based on this knowledge in a probabilistic environment. The research efforts were classified into two main areas. First, Task A of the grant has had the objective of conducting research into the relative merits of possible approaches that account for both multiple criteria and uncertainty in design decision-making. In particular, in the final year of research, the focus was on the comparison and contrasting between three methods researched. Specifically, these three are the Joint Probabilistic Decision-Making (JPDM) technique, Physical Programming, and Dempster-Shafer (D-S) theory. The next element of the research, as contained in Task B, was focused upon exploration of the Technology Identification, Evaluation, and Selection (TIES) methodology developed at ASDL, especially with regards to identification of research needs in the baseline method through implementation exercises. The end result of Task B was the documentation of the evolution of the method with time and a technology transfer to the sponsor regarding the method, such that an initial capability for execution could be obtained by the sponsor. Specifically, the results of year 3 efforts were the creation of a detailed tutorial for implementing the TIES method. Within the tutorial package, templates and detailed examples were created for learning and understanding the details of each step. For both research tasks, sample files and tutorials are attached in electronic form with the enclosed CD.

Marvis, Dimitri N.↗

NASA NTRS2003

Exploration of Advanced Probabilistic and Stochastic Design Methods

The primary objective of the three year research effort was to explore advanced, non-deterministic aerospace system design methods that may have relevance to designers and analysts. The research pursued emerging areas in design methodology and leverage current fundamental research in the area of design decision-making, probabilistic modeling, and optimization. The specific focus of the three year investigation was oriented toward methods to identify and analyze emerging aircraft technologies in a consistent and complete manner, and to explore means to make optimal decisions based on this knowledge in a probabilistic environment. The research efforts were classified into two main areas. First, Task A of the grant has had the objective of conducting research into the relative merits of possible approaches that account for both multiple criteria and uncertainty in design decision-making. In particular, in the final year of research, the focus was on the comparison and contrasting between three methods researched. Specifically, these three are the Joint Probabilistic Decision-Making (JPDM) technique, Physical Programming, and Dempster-Shafer (D-S) theory. The next element of the research, as contained in Task B, was focused upon exploration of the Technology Identification, Evaluation, and Selection (TIES) methodology developed at ASDL, especially with regards to identification of research needs in the baseline method through implementation exercises. The end result of Task B was the documentation of the evolution of the method with time and a technology transfer to the sponsor regarding the method, such that an initial capability for execution could be obtained by the sponsor. Specifically, the results of year 3 efforts were the creation of a detailed tutorial for implementing the TIES method. Within the tutorial package, templates and detailed examples were created for learning and understanding the details of each step. For both research tasks, sample files and tutorials are attached in electronic form with the enclosed CD.

Mavris, Dimitri N.↗

NASA NTRSDate not supplied

ATTRACTOR: Toward Trustworthy and Trusted Autonomous Systems

The question of what it means and what it takes for an autonomous system to consider another autonomous system justifiably trustworthy must be addressed by all who seek to integrate intelligent machine agents into real-world operations. A satisfactory answer to this question is an essential component in accepting autonomous machine decision-making in safety-critical and time-critical environments, such as aviation. Historically, simulation platforms for test and evaluation of complex systems have proven to be effective in assessing performance and contributing to decisions on the fitness of systems to operate in current general and commercial aviation airspace. Moreover, simulations have informed the definition of safety-critical constraints. However, as machine systems progressively take on responsibilities for decision-making traditionally supplied by humans, simulations require enhancement. Mixed reality simulation that integrates real-world platforms and data or high-fidelity simulation data in a sim-to-flight paradigm provides insight into agent interaction and the rationale behind autonomous agent decision-making as well as the capacity for seamless integrated implementation, testing, and operation of systems. Strong simulation capabilities are especially important in the presence of algorithms that hold great promise in decision-making yet increase the uncertainty in the system. Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) is a subproject of NASA’s Convergent Aeronautics Solutions (CAS) Project. ATTRACTOR’s objective is to build a basis for understanding trust and trustworthiness in multi-agent autonomous teams, and thus to inform future certification of safety-critical and time-critical autonomous systems in aviation. Because the concepts of trust and trustworthiness must be addressed in a context, ATTRACTOR has chosen Search and Rescue (SAR) in dynamic and unstructured environments, with emphasis on search, as its design reference mission (DRM). During dynamic planning and execution of trajectory-based operations, autonomous agents determine their trajectories given an assigned mission or missions and call for assistance from an appropriate teammate when needed. This experience along with the attendant human-machine and machine-machine interactions, serve as a platform for developing approaches to identifying and measuring trustworthiness and increasing trust. In this paper, we give an overview of some of ATTRACTOR’s research and development activities, findings, and ongoing work.

ATTRACTOR↗

NASA NTRS2020

Ensemble Methodologies for Astronaut Cancer Risk Assessment in the face of Large Uncertainties

A new approach to NASA space radiation risk modeling has successfully extended the current NASA probabilistic cancer risk model to an ensemble framework able to consider sub-model parameter uncertainty (e.g. uncertainty in a radiation quality parameter) as well as model-form uncertainty associated with differing theoretical or empirical formalisms (e.g. combined dose-rate and radiation quality effects). Ensemble methodologies are already widely used in weather prediction, modeling of infectious disease outbreaks, and certain terrestrial radiation protection applications to better understand how uncertainty may influence risk decision-making. Applying ensemble methodologies to space radiation risk projections offers the potential to efficiently incorporate emerging research results, allow for the incorporation of future (including international) models, improve uncertainty quantification for underlying sub-models developed against sparse experimental data, and reduce the impact of subjective bias on risk projections. Moreover, risk forecasting across an ensemble of multiple predictive models can provide stakeholders additional information on risk acceptance if current health/medical standards cannot be met or the level of knowledge doesn’t permit a specific risk or exposure limit to be developed for future space exploration missions. In this work, ensemble risk projections implementing multiple sub-models of radiation quality, dose and dose-rate effectiveness factors, excess risk, and latency as ensemble members are presented. Initial consensus methods for ensemble model weights and correlations to account for individual model bias are discussed. In these analyses, the ensemble forecast compares well to results from NASA's current operational cancer risk projection model used to assess permissible exposure limits and permissible mission durations for astronauts. However, a large range of projected risk values are obtained at the upper 95th confidence level where models must extrapolate beyond available biological data sets; closer agreement is seen at the median + one sigma due to the inherent similarities in available models. Future work, including the addition of new models and methods for statistical correlation between predictive members are discussed to define alternate ways of thinking about risk and ‘acceptable’ uncertainty with respect to NASA’s current permissible exposure limits.

space radiation↗

NASA NTRSDate not supplied

Ensemble Cancer Risk Model for Astronaut Risk Assessment

A new approach to NASA space radiation risk modeling has successfully extended the current NASA probabilistic cancer risk model to an ensemble framework able to consider sub-model parameter uncertainty (e.g. uncertainty in a radiation quality parameter) as well as model-form uncertainty associated with differing theoretical or empirical formalisms (e.g. combined dose-rate and radiation quality effects). Ensemble methodologies are already widely used in weather prediction, modeling of infectious disease outbreaks, and certain terrestrial radiation protection applications to better understand how uncertainty may influence risk decision-making. Applying ensemble methodologies to space radiation risk projections offers the potential to efficiently incorporate emerging research results, allow for the incorporation of future (including international) models, improve uncertainty quantification for underlying sub-models developed against sparse experimental data, and reduce the impact of subjective bias on risk projections. Moreover, risk forecasting across an ensemble of multiple predictive models can provide stakeholders additional information on risk acceptance if current health/medical standards cannot be met or the level of knowledge doesn’t permit a specific risk or exposure limit to be developed for future space exploration missions. In this work, ensemble risk projections implementing multiple sub-models of radiation quality, dose and dose-rate effectiveness factors, excess risk, and latency as ensemble members are presented. Initial consensus methods for ensemble model weights and correlations to account for individual model bias are discussed. In these analyses, the ensemble forecast compares well to results from NASA's current operational cancer risk projection model used to assess permissible exposure limits and permissible mission durations for astronauts. However, a large range of projected risk values are obtained at the upper 95th confidence level where models must extrapolate beyond available biological data sets; closer agreement is seen at the median + one sigma due to the inherent similarities in available models. Future work, including the addition of new models and methods for statistical correlation between predictive members are discussed to define alternate ways of thinking about risk and ‘acceptable’ uncertainty with respect to NASA’s current permissible exposure limits.

Lisa C Simonsen↗

NASA NTRSDate not supplied

Runway Configuration Management with Offline Reinforcement Learning

Runway configuration management (RCM) is a challenging task, and it affects the efficiency of the National Airspace System (NAS) and airport surface operations significantly. Each airport, depending on the geometry, capacity, local climate patterns, etc. has multiple configurations for the runway usage for arriving and departing flights. Many factors such as the incoming/outgoing traffic load, wind direction and speed, convective weather, cloud ceiling and other environmental factors might affect the choice of a runway configuration at any point in time. However, other factors such as safety measures and regulations, noise abatement, capacity of each configuration, and preference of the air traffic controllers (ATCs) can also play a significant role in selecting the configuration. A sub-optimal selection of the runway configuration, or delay in making configuration changes might result in significant increase in taxi times for aircraft on the surface of the airport, fuel and energy use of the aircraft, and maintenance costs. It can also lead to safety concerns, such as an aircraft performing one or more go-arounds before being able to land. All these factors make RCM an extremely important and challenging decision-making process for the ATCs. The current state of practice sets the runway configuration by the ATCs based on relevant information available at the time including weather, traffic, noise abatement, safety bounds, etc. This makes the decision-making process subjective based on the accuracy of the available information and the bias in human decision making. Unfortunately, this approach yields poor results (e.g., significant delays) if the predicted outcomes are uncertain and their relative impact is not well understood. This is especially evident when the uncertainty increases the size of possible predicted outcomes (combinatorial explosion in possible scenarios) that cannot be handled by human reasoning. On the other hand, an automated approach based on machine intelligence can make use of historical data and search through all (or significant amount of) possible scenarios under uncertainty and make well-informed decisions.

Milad Memarzadeh↗

NASA NTRSDate not supplied

A Practical Approach to Uncertainty Quantification Using Probability Boxes

To date, while the use of CFD for aerospace vehicle design and development is prevalent, the documentation of uncertainties associated with the simulations are rare. Instead, the current state-of-the-art relies heavily on the experience of the CFD practitioner to estimate the uncertainty associated with their simulations through simple sensitivity studies or subject matter expertise. This practice will have to be replaced with a formal uncertainty quantification (UQ) process if CFD is to play an expanded role in the research and engineering design community, test and evaluation community, and ultimately certification for flight. Accounting for uncertainties in a formal manner is a tedious process. Moreover, the typical CFD practitioner is not likely to be familiar with formal UQ methods. These factors have prevented the adoption of UQ methods in the engineering design and development cycle. This presentation will outline a credible approach to UQ using Probability Boxes that is straightforward to apply, and can readily be automated using existing UQ tool sets such as the DAKOTA packaged developed at Sandia. The added expense incurred when moving away from a deterministic CFD process to a stochastic one that captures uncertainties to enable risk-informed decision making will be discussed, as well as effective ways to reduce the computational costs.

Uncertainty Quantification↗

NASA NTRS2016

Benefit Opportunities for Integrated Surface and Airspace Departure Scheduling: A Study of Operations at Charlotte-Douglas International Airport

NASA is collaborating with the FAA and aviation industry to develop and demonstrate new capabilities that integrate arrival, departure, and surface air-traffic operations. The concept relies on trajectory-based departure scheduling and collaborative decision making to reduce delays and uncertainties in taxi and climb operations. The paper describes the concept and benefit mechanisms aimed at improving flight efficiency and predictability while maintaining or improving operational throughput. The potential impact of the technology is studied and discussed through a quantitative analysis of relevant shortfalls at the site identified for initial deployment and demonstration in 2017: Charlotte-Douglas International Airport. Results from trajectory analysis indicate substantial opportunity to reduce taxi delays for both departures and arrivals by metering departures at the gate in a manner that maximizes throughput while adhering to takeoff restrictions due mostly to airspace constraints. Substantial taxi-out delay reduction is shown for flights subject to departure restrictions stemming from traffic flow management initiatives. Opportunities to improve the predictability of taxi, takeoff, and climb operations are examined and their potential impact on airline scheduling decisions and air-traffic forecasting is discussed. In addition, the potential to improve throughput with departure scheduling that maximizes use of available runway and airspace capacity is analyzed.

air traffic management↗