Results for “sequential decision-making”

NASA NTRS2012

A Risk-Constrained Multi-Stage Decision Making Approach to the Architectural Analysis of Mars Missions

This paper presents a novel risk-constrained multi-stage decision making approach to the architectural analysis of planetary rover missions. In particular, focusing on a 2018 Mars rover concept, which was considered as part of a potential Mars Sample Return campaign, we model the entry, descent, and landing (EDL) phase and the rover traverse phase as four sequential decision-making stages. The problem is to find a sequence of divert and driving maneuvers so that the rover drive is minimized and the probability of a mission failure (e.g., due to a failed landing) is below a user specified bound. By solving this problem for several different values of the model parameters (e.g., divert authority), this approach enables rigorous, accurate and systematic trade-offs for the EDL system vs. the mobility system, and, more in general, cross-domain trade-offs for the different phases of a space mission. The overall optimization problem can be seen as a chance-constrained dynamic programming problem, with the additional complexity that 1) in some stages the disturbances do not have any probabilistic characterization, and 2) the state space is extremely large (i.e, hundreds of millions of states for trade-offs with high-resolution Martian maps). To this purpose, we solve the problem by performing an unconventional combination of average and minimax cost analysis and by leveraging high efficient computation tools from the image processing community. Preliminary trade-off results are presented.

entry, descent, and landing (EDL)↗

NASA NTRSDate not supplied

Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems

To find failure events and their likelihoods in flight-critical systems, we investigate the use of an advanced black-box stress testing approach called adaptive stress testing. We analyze a trajectory predictor from a developmental commercial flight management system which takes as input a collection of lateral waypoints and en-route environmental conditions. Our aim is to search for failure events relating to inconsistencies in the predicted lateral trajectories. The intention of this work is to find likely failures and report them back to the developers so they can address and potentially resolve shortcomings of the system before deployment. To improve search performance, this work extends the adaptive stress testing formulation to be applied more generally to sequential decision-making problems with episodic reward by collecting the state transitions during the search and evaluating at the end of the simulated rollout. We use a modified Monte Carlo tree search algorithm with progressive widening as our adversarial reinforcement learner. The performance is compared to direct Monte Carlo simulations and to the cross-entropy method as an alternative importance sampling baseline. The goal is to find potential problems otherwise not found by traditional requirements-based testing. Results indicate that our adaptive stress testing approach finds more failures and finds failures with higher likelihood relative to the baseline approaches.

adaptive stress testing↗

NASA NTRSDate not supplied

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

NASA NTRSDate not supplied

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learning↗

NASA NTRS1972

Speed in Information Processing with a Computer Driven Visual Display in a Real-time Digital Simulation

Information transfer between the operator and computer-generated display systems is an area where the human factors engineer discovers little useful design data relating human performance to system effectiveness. This study utilized a computer-driven, cathode-ray-tube graphic display to quantify human response speed in a sequential information processing task. The performance criteria was response time to sixteen cell elements of a square matrix display. A stimulus signal instruction specified selected cell locations by both row and column identification. An equal probable number code, from one to four, was assigned at random to the sixteen cells of the matrix and correspondingly required one of four, matched keyed-response alternatives. The display format corresponded to a sequence of diagnostic system maintenance events, that enable the operator to verify prime system status, engage backup redundancy for failed subsystem components, and exercise alternate decision-making judgements. The experimental task bypassed the skilled decision-making element and computer processing time, in order to determine a lower bound on the basic response speed for given stimulus/response hardware arrangement.

Kyle, R. G.↗

NASA NTRS2020

Ellicott City Disasters II - Building a Real-Time Predictive Flood Model for Improving Early Warning Systems in Ellicott City, Maryland

As flood events in the United States grow in frequency and intensity, the uses of applied remote sensing analyses are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters III project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems in Ellicott City, Maryland. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, this term built on the predictive capability of the long-short term memory (LSTM) model created by the second term of this DEVELOP project to create a Sequentially Trained Real-time EstimAted Model (STREAM). Enhancements to the model included the integration of both real-time and predicted weather products from the National Weather Service to increase predictive capacity. These weather products were supplemented by stream gauge data from the OEM as well as real-time radar products. The resultant flood risk model was trained to evaluate input variables and predict stage height in Ellicott City in real time. The model, upgraded to predict stage height up to 8 hours in advance, was incorporated into an online dashboard in a user-friendly interface. The project demonstrated the potential for integration of open data and NASA Earth observations into a flood risk forecasting tool capable of informing real-time decision-making.

Erika Munshi↗

NASA NTRS2020

Ellicott City Disasters III: Building a Real-Time Statistical Flood Model for Improving Early Warning Systems in Ellicott City, Maryland

As flood events in the United States grow in frequency and intensity, the uses of applied remote sensing analyses are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters III project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems in Ellicott City, Maryland. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, this term built on the predictive capability of the long-short term memory (LSTM) model created by the second term of this DEVELOP project to create a Sequentially Trained Real-time EstimAted Model (STREAM). Enhancements to the model included the integration of both real-time and predicted weather products from the National Weather Service to increase predictive capacity. These weather products were supplemented by stream gauge data from the OEM as well as real-time radar products. The resultant flood risk model was trained to evaluate input variables and predict stage height in Ellicott City in real time. The model, upgraded to predict stage height up to 8 hours in advance, was incorporated into an online dashboard in a user-friendly interface. The project demonstrated the potential for integration of open data and NASA Earth observations into a flood risk forecasting tool capable of informing real-time decision-making.

Erika Munshi↗

NASA NTRS2005

D-Side: A Facility and Workforce Planning Group Multi-criteria Decision Support System for Johnson Space Center

"To understand and protect our home planet, to explore the universe and search for life, and to inspire the next generation of explorers" is NASA's mission. The Systems Management Office at Johnson Space Center (JSC) is searching for methods to effectively manage the Center's resources to meet NASA's mission. D-Side is a group multi-criteria decision support system (GMDSS) developed to support facility decisions at JSC. D-Side uses a series of sequential and structured processes to plot facilities in a three-dimensional (3-D) graph on the basis of each facility alignment with NASA's mission and goals, the extent to which other facilities are dependent on the facility, and the dollar value of capital investments that have been postponed at the facility relative to the facility replacement value. A similarity factor rank orders facilities based on their Euclidean distance from Ideal and Nadir points. These similarity factors are then used to allocate capital improvement resources across facilities. We also present a parallel model that can be used to support decisions concerning allocation of human resources investments across workforce units. Finally, we present results from a pilot study where 12 experienced facility managers from NASA used D-Side and the organization's current approach to rank order and allocate funds for capital improvement across 20 facilities. Users evaluated D-Side favorably in terms of ease of use, the quality of the decision-making process, decision quality, and overall value-added. Their evaluations of D-Side were significantly more favorable than their evaluations of the current approach. Keywords: NASA, Multi-Criteria Decision Making, Decision Support System, AHP, Euclidean Distance, 3-D Modeling, Facility Planning, Workforce Planning.

Tavana, Madjid↗

NASA NTRS2009

Multi-Scale Hydrometeorological Modeling, Land Data Assimilation and Parameter Estimation with the Land Information System

The Land Information System (LIS; http://lis.gsfc.nasa.gov; Kumar et al., 2006; Peters- Lidard et al.,2007) is a flexible land surface modeling framework that has been developed with the goal of integrating satellite- and ground-based observational data products and advanced land surface modeling techniques to produce optimal fields of land surface states and fluxes. As such, LIS represents a step towards the next generation land component of an integrated Earth system model. In recognition of LIS object-oriented software design, use and impact in the land surface and hydrometeorological modeling community, the LIS software was selected ase co-winner of NASA's 2005 Software of the Year award. LIS facilitates the integration of observations from Earth-observing systems and predictions and forecasts from Earth System and Earth science models into the decision-making processes of partnering agency and national organizations. Due to its flexible software design, LIS can serve both as a Problem Solving Environment (PSE) for hydrologic research to enable accurate global water and energy cycle predictions, and as a Decision Support System (DSS) to generate useful information for application areas including disaster management, water resources management, agricultural management, numerical weather prediction, air quality and military mobility assessment. LIS has evolved from two earlier efforts North American Land Data Assimilation System (NLDAS; Mitchell et al. 2004) and Global Land Data Assimilation System (GLDAS; Rodell al. 2004) that focused primarily on improving numerical weather prediction skills by improving the characterization of the land surface conditions. Both of GLDAS and NLDAS now use specific configurations of the LIS software in their current implementations. In addition, LIS was recently transitioned into operations at the US Air Force Weather Agency (AFWA) to ultimately replace their Agricultural Meteorology (AGRMET) system, and is also used routinely by NOAA's National Centers for Environmental Prediction (NCEP)/Environmental Modeling Center (EMC) for their land data assimilation systems to support weather and climate modeling. LIS not only consolidates the capabilities of these two systems, but also enables a much larger variety of configurations with respect to horizontal spatial resolution, input datasets and choice of land surface model through "plugins,". As described in Kumar et al., 2007, and demonstrated in Case et al., 2008, and Santanello et al., 2009, LIS has been coupled to the Weather Research and Forecasting (WRF) model to support studies of land-atmosphere coupling the enabling ensembles of land surface states to be tested against multiple representations of the atmospheric boundary layer. LIS has also been demonstrated for parameter estimation as described in Peters-Lidard et al. (2008) and Santanello et al. (2007), who showed that the use of sequential remotely sensed soil moisture products can be used to derive soil hydraulic and texture properties given a sufficient dynamic range in the soil moisture retrievals and accurate precipitation inputs. LIS has also recently been demonstrated for multi-model data assimilation (Kumar et al., 2008) using an Ensemble Kalman Filter for sequential assimilation of soil moisture, snow, and temperature. Ongoing work has demonstrated the value of bias correction as part of the filter, and also that of joint calibration and assimilation. Examples and case studies demonstrating the capabilities and impacts of LIS for hydrometeoroogical modeling, assimilation and parameter estimation will be presented as advancements towards the next generation of integrated observation and modeling systems.

Peters-Lidard, Christa D.↗

NASA NTRS2011

Multi-Scale Hydrometeorological Modeling, Land Data Assimilation and Parameter Estimation with the Land Information System

The Land Information System (LIS; http://lis.gsfc.nasa.gov) is a flexible land surface modeling framework that has been developed with the goal of integrating satellite-and ground-based observational data products and advanced land surface modeling techniques to produce optimal fields of land surface states and fluxes. As such, LIS represents a step towards the next generation land component of an integrated Earth system model. In recognition of LIS object-oriented software design, use and impact in the land surface and hydrometeorological modeling community, the LIS software was selected as a co-winner of NASA?s 2005 Software of the Year award.LIS facilitates the integration of observations from Earth-observing systems and predictions and forecasts from Earth System and Earth science models into the decision-making processes of partnering agency and national organizations. Due to its flexible software design, LIS can serve both as a Problem Solving Environment (PSE) for hydrologic research to enable accurate global water and energy cycle predictions, and as a Decision Support System (DSS) to generate useful information for application areas including disaster management, water resources management, agricultural management, numerical weather prediction, air quality and military mobility assessment. LIS has e volved from two earlier efforts -- North American Land Data Assimilation System (NLDAS) and Global Land Data Assimilation System (GLDAS) that focused primarily on improving numerical weather prediction skills by improving the characterization of the land surface conditions. Both of GLDAS and NLDAS now use specific configurations of the LIS software in their current implementations.In addition, LIS was recently transitioned into operations at the US Air Force Weather Agency (AFWA) to ultimately replace their Agricultural Meteorology (AGRMET) system, and is also used routinely by NOAA's National Centers for Environmental Prediction (NCEP)/Environmental Modeling Center (EMC) for their land data assimilation systems to support weather and climate modeling. LIS not only consolidates the capabilities of these two systems, but also enables a much larger variety of configurations with respect to horizontal spatial resolution, input datasets and choice of land surface model through "plugins". LIS has been coupled to the Weather Research and Forecasting (WRF) model to support studies of land-atmosphere coupling be enabling ensembles of land surface states to be tested against multiple representations of the atmospheric boundary layer. LIS has also been demonstrated for parameter estimation, who showed that the use of sequential remotely sensed soil moisture products can be used to derive soil hydraulic and texture properties given a sufficient dynamic range in the soil moisture retrievals and accurate precipitation inputs.LIS has also recently been demonstrated for multi-model data assimilation using an Ensemble Kalman Filter for sequential assimilation of soil moisture, snow, and temperature.Ongoing work has demonstrated the value of bias correction as part of the filter, and also that of joint calibration and assimilation.Examples and case studies demonstrating the capabilities and impacts of LIS for hydrometeorological modeling, assimilation and parameter estimation will be presented as advancements towards the next generation of integrated observation and modeling systems

Peters-Lidard, Christa D.↗