SEARCH · Search NASA

Results for “Automated Scenario Generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Optimizing Decision Preparedness by Adapting Scenario Complexity and Automating Scenario Generation

Klein's recognition-primed decision (RPD) framework proposes that experts make decisions by recognizing similarities between current decision situations and previous decision experiences. Unfortunately, military personnel arQ often presented with situations that they have not experienced before. Scenario-based training (S8T) can help mitigate this gap. However, SBT remains a challenging and inefficient training approach. To address these limitations, the authors present an innovative formulation of scenario complexity that contributes to the larger research goal of developing an automated scenario generation system. This system will enable trainees to effectively advance through a variety of increasingly complex decision situations and experiences. By adapting scenario complexities and automating generation, trainees will be provided with a greater variety of appropriately calibrated training events, thus broadening their repositories of experience. Preliminary results from empirical testing (N=24) of the proof-of-concept formula are presented, and future avenues of scenario complexity research are also discussed.

Dunne, Rob↗

NASA NTRS2018

Automated Scenario Generation for Human-in-the-Loop Simulations

Automated Multi-Aircraft Control System scenario generation for Human-in-the-Loop evaluations of air traffic management concepts is described. Methods for analyzing and comparing the seed-scenario generated using the automated process and the Human-in-the-Loop-scenario designed to meet the experiment objectives are discussed. The main findings are: (1) many of the characteristics of the seed-scenario used for constructing the Human-in-the-Loop-scenario are preserved in the Human-in-the-Loop Scenario, (2) landing characteristics of the traffic generated by the Multi-Aircraft Control System using the input scenario compare reasonably well with that intended in the input scenario, and (3) many of desired characteristics of the Human-in-the-Loop-scenario can be achieved by further automation.

SMART-NAS Testbed↗

NASA NTRS2018

Automated Scenario Generation for Human-in-the-Loop Simulations

Automated Multi-Aircraft Control System scenario generation for Human-in-the-Loop (HITL) evaluations of air traffic management concepts is described. The objective is to replace the difficult manual process with the automated process for creating an initial (seed) scenario that serves as a starting point for manual adjustments for creating the Human-in-the-Loop scenario. Methods for analyzing and comparing the seed-scenario generated using the automated process and the Human-in-the-Loop-scenario derived from it to meet the experiment objectives are discussed. Results of comparison of input Human-in-the-Loop-scenario with the Multi-Aircraft Control System output are also presented. The main findings are: (1) many of the characteristics of the seed-scenario used for constructing the Human-in-the-Loop-scenario are preserved in the Human-in-the-Loop-scenario, (2) landing rate profile of the traffic generated by the Multi-Aircraft Control System (MACS) using the input scenario compares reasonably well with that intended in the input scenario, and (3) many of the desired characteristics of the Human-in-the-Loop-scenario can be achieved by further automation.

Automated Scenario Generation↗

NASA NTRS2019

Automated Scenario Generation for Meeting Human-in-the-Loop Simulation Requirements

A two-step automated Multi-Aircraft Control System traffic scenario generation process for Human-in-the-Loop evaluations of air traffic management concepts is described. The first step of the two-step process employs the scenario generation capability currently available in NASA's Air Traffic Management Testbed. The second step refines the scenario by filtering flights from the traffic scenario based on route length, cruise speed, cruise altitude, entry time and the desired ratio of internal to external flights. A solution for achieving the desired ratio of internal to external flights, where internal flights are shorter flights and external flights are longer flights based on a distance threshold, is described. Finally, schedulers are described for shaping the hourly arrival traffic count as a function of time in response to airport capacity constraint or for increasing the traffic demand with respect to the available arrival capacity. Results generated for arrival traffic to the four major airports in the New York Metroplex on a busy day using the two-step procedure are discussed. These results show that traffic scenarios for Multi-Aircraft Control System that meet the Human-in-the-Loop and fast-time simulation requirements can be created automatically following the procedures described in the paper. The automated process will improve the accuracy and efficiency by eliminating the tedious manual process for scenario generation.

Automated Scenario Generation↗

NASA NTRS2019

Automated Scenario Generation for Meeting Human-In-The-Loop Simulation Requirements

Automated Scenario Generation↗

NASA NTRS2007

Development of a Prototype Automation Simulation Scenario Generator for Air Traffic Management Software Simulations

A technique for automated development of scenarios for use in the Multi-Center Traffic Management Advisor (McTMA) software simulations is described. The resulting software is designed and implemented to automate the generation of simulation scenarios with the intent of reducing the time it currently takes using an observational approach. The software program is effective in achieving this goal. The scenarios created for use in the McTMA simulations are based on data taken from data files from the McTMA system, and were manually edited before incorporation into the simulations to ensure accuracy. Despite the software s overall favorable performance, several key software issues are identified. Proposed solutions to these issues are discussed. Future enhancements to the scenario generator software may address the limitations identified in this paper.

Khambatta, Cyrus F.↗

NASA NTRS1992

Interactive specification acquisition via scenarios: A proposal

Some reactive systems are most naturally specified by giving large collections of behavior scenarios. These collections not only specify the behavior of the system, but also provide good test suites for validating the implemented system. Due to the complexity of the systems and the number of scenarios, however, it appears that automated assistance is necessary to make this software development process workable. Interactive Specification Acquisition Tool (ISAT) is a proposed interactive system for supporting the acquisition and maintenance of a formal system specification from scenarios, as well as automatic synthesis of control code and automated test generation. This paper discusses the background, motivation, proposed functions, and implementation status of ISAT.

Hall, Robert J.↗

NASA NTRS2016

A Cognitive Systems Engineering Approach to Developing HMI Requirements for New Technologies

This document examines the challenges inherent in designing and regulating to support human-automation interaction for new technologies that will deployed into complex systems. A key question for new technologies, is how work will be accomplished by the human and machine agents. This question has traditionally been framed as how functions should be allocated between humans and machines. Such framing misses the coordination and synchronization that is needed for the different human and machine roles in the system to accomplish their goals. Coordination and synchronization demands are driven by the underlying human-automation architecture of the new technology, which are typically not specified explicitly by the designers. The human machine interface (HMI) which is intended to facilitate human-machine interaction and cooperation, however, typically is defined explicitly and therefore serves as a proxy for human-automation cooperation requirements with respect to technical standards for technologies. Unfortunately, mismatches between the HMI and the coordination and synchronization demands of the underlying human-automation architecture, can lead to system breakdowns. A methodology is needed that both designers and regulators can utilize to evaluate the expected performance of a new technology given potential human-automation architectures. Three experiments were conducted to inform the minimum HMI requirements a detect and avoid system for unmanned aircraft systems (UAS). The results of the experiments provided empirical input to specific minimum operational performance standards that UAS manufacturers will have to meet in order to operate UAS in the National Airspace System (NAS). These studies represent a success story for how to objectively and systematically evaluate prototype technologies as part of the process for developing regulatory requirements. They also provide an opportunity to reflect on the lessons learned from a recent research effort in order to improve the methodology for defining technology requirements for regulators in the future. The biggest shortcoming of the presented research program was the absence of the explicit definition, generation and analysis of potential human-automation architectures. Failure to execute this step in the research process resulted in less efficient evaluation of the candidate prototypes technologies in addition to the complete absence of different approaches to human-automation cooperation. For example, all of the prototype technologies that were evaluated in the research program assumed a human-automation architecture that relied on serial processing from the automation to the human. While this type of human-automation architecture is typical across many different technologies and in many different domains, it ignores different architectures where humans and automation work in parallel. Defining potential human-automation architectures a priori also allows regulators to develop scenarios that will stress the performance boundaries of the technology during the evaluation phase. The importance of adding this step of generating and evaluating candidate human-automation architectures prior to formal empirical evaluation is discussed.

human systems integration↗

NASA NTRS2007

Generating Scenarios When Data Are Missing

The Hypothetical Scenario Generator (HSG) is being developed in conjunction with other components of artificial-intelligence systems for automated diagnosis and prognosis of faults in spacecraft, aircraft, and other complex engineering systems. The HSG accepts, as input, possibly incomplete data on the current state of a system (see figure). The HSG models a potential fault scenario as an ordered disjunctive tree of conjunctive consequences, wherein the ordering is based upon the likelihood that a particular conjunctive path will be taken for the given set of inputs. The computation of likelihood is based partly on a numerical ranking of the degree of completeness of data with respect to satisfaction of the antecedent conditions of prognostic rules. The results from the HSG are then used by a model-based artificial- intelligence subsystem to predict realistic scenarios and states.

Mackey, Ryan↗

NASA NTRSDate not supplied

Open-source Techniques for Automated Landslide Inventory Generation for Rapid Response

Manual mapping is the most used method for generating landslide inventories. For rapid response scenario this method becomes tedious and time consuming. The Landslide team at NASA Goddard Space Flight Center has been developing open-source landslide mapping systems for rapid generation of landslide inventories. We have developed a Python-based landslide mapping framework known as the Semi-Automatic Landslide Detection (SALaD) system that uses Object-based Image Analysis and machine learning. For production of event-based inventories, SALaD was modified to include a change detection module (SALaD-CD). Utilizing high-resolution imagery form from Planet and Maxar, we have generated multiple rapid response landslide inventories that have been used by emergency responders on the ground, the NASA Disasters program, and academia. Currently, we are exploiting deep learning frameworks for landslide mapping. We are interested to learn about efficient way to harmonize multi-sensor data for creating a long-term record of landslides, training strategies and ongoing deep learning-based efforts for natural hazard characterization within NASA and UMD.

Pukar Amatya↗

NASA NTRS2007

Hypothetical Scenario Generator for Fault-Tolerant Diagnosis

The Hypothetical Scenario Generator for Fault-tolerant Diagnostics (HSG) is an algorithm being developed in conjunction with other components of artificial- intelligence systems for automated diagnosis and prognosis of faults in spacecraft, aircraft, and other complex engineering systems. By incorporating prognostic capabilities along with advanced diagnostic capabilities, these developments hold promise to increase the safety and affordability of the affected engineering systems by making it possible to obtain timely and accurate information on the statuses of the systems and predicting impending failures well in advance. The HSG is a specific instance of a hypothetical- scenario generator that implements an innovative approach for performing diagnostic reasoning when data are missing. The special purpose served by the HSG is to (1) look for all possible ways in which the present state of the engineering system can be mapped with respect to a given model and (2) generate a prioritized set of future possible states and the scenarios of which they are parts.

James, Mark↗

NASA NTRS2008

Automated Generation and Assessment of Autonomous Systems Test Cases

This slide presentation reviews some of the issues concerning verification and validation testing of autonomous spacecraft routinely culminates in the exploration of anomalous or faulted mission-like scenarios using the work involved during the Dawn mission's tests as examples. Prioritizing which scenarios to develop usually comes down to focusing on the most vulnerable areas and ensuring the best return on investment of test time. Rules-of-thumb strategies often come into play, such as injecting applicable anomalies prior to, during, and after system state changes; or, creating cases that ensure good safety-net algorithm coverage. Although experience and judgment in test selection can lead to high levels of confidence about the majority of a system's autonomy, it's likely that important test cases are overlooked. One method to fill in potential test coverage gaps is to automatically generate and execute test cases using algorithms that ensure desirable properties about the coverage. For example, generate cases for all possible fault monitors, and across all state change boundaries. Of course, the scope of coverage is determined by the test environment capabilities, where a faster-than-real-time, high-fidelity, software-only simulation would allow the broadest coverage. Even real-time systems that can be replicated and run in parallel, and that have reliable set-up and operations features provide an excellent resource for automated testing. Making detailed predictions for the outcome of such tests can be difficult, and when algorithmic means are employed to produce hundreds or even thousands of cases, generating predicts individually is impractical, and generating predicts with tools requires executable models of the design and environment that themselves require a complete test program. Therefore, evaluating the results of large number of mission scenario tests poses special challenges. A good approach to address this problem is to automatically score the results based on a range of metrics. Although the specific means of scoring depends highly on the application, the use of formal scoring - metrics has high value in identifying and prioritizing anomalies, and in presenting an overall picture of the state of the test program. In this paper we present a case study based on automatic generation and assessment of faulted test runs for the Dawn mission, and discuss its role in optimizing the allocation of resources for completing the test program.

Testing challenges↗

NASA NTRS2013

Predicting Pilot Behavior in Medium Scale Scenarios Using Game Theory and Reinforcement Learning

Effective automation is critical in achieving the capacity and safety goals of the Next Generation Air Traffic System. Unfortunately creating integration and validation tools for such automation is difficult as the interactions between automation and their human counterparts is complex and unpredictable. This validation becomes even more difficult as we integrate wide-reaching technologies that affect the behavior of different decision makers in the system such as pilots, controllers and airlines. While overt short-term behavior changes can be explicitly modeled with traditional agent modeling systems, subtle behavior changes caused by the integration of new technologies may snowball into larger problems and be very hard to detect. To overcome these obstacles, we show how integration of new technologies can be validated by learning behavior models based on goals. In this framework, human participants are not modeled explicitly. Instead, their goals are modeled and through reinforcement learning their actions are predicted. The main advantage to this approach is that modeling is done within the context of the entire system allowing for accurate modeling of all participants as they interact as a whole. In addition such an approach allows for efficient trade studies and feasibility testing on a wide range of automation scenarios. The goal of this paper is to test that such an approach is feasible. To do this we implement this approach using a simple discrete-state learning system on a scenario where 50 aircraft need to self-navigate using Automatic Dependent Surveillance-Broadcast (ADS-B) information. In this scenario, we show how the approach can be used to predict the ability of pilots to adequately balance aircraft separation and fly efficient paths. We present results with several levels of complexity and airspace congestion.

Game Theory↗

NASA NTRS2023

Leveraging Automated Fiber Placement Computer Aided Process Planning Framework for Defect Validation and Dynamic Layup Strategies

Process planning represents an essential stage of the Automated Fiber Placement (AFP) workflow. It develops useful and efficient machine processes based upon the working material, composite design, and manufacturing resources. The current state of process planning requires a high degree of interaction from the process planner and could greatly benefit from increased automation. Therefore, a list of key steps and functions are created to identify the more difficult and time-consuming phases of process planning. Additionally, a set of metrics must exist by which to evaluate the effectiveness of the manufactured laminate from the machine code created during the Process Planning stage. Layup strategies, in addition to dog ears, stagger shifts, steering constraints, and starting points, represented the group of functions labeled as process optimization and ranked the highest in terms of priority for automation. The laminates resulting from the selected parameters are evaluated through the occurrences of principal defect metrics such as fiber gaps, overlaps, angle deviation and steering violations. This document presents an automated software solution to the layup strategy and starting point selection phase of process planning. A series of ply scenarios are generated with variations of these ply parameters and evaluated according to a set of metrics entered by the Process Planner. These metrics are generated through use of the Analytical Hierarchy Process (AHP), where relative importance between each of the fiber features are defined. The ply scenarios are selected which reduce the overall fiber feature scores based on the defects the Process Planner wishes to minimize.

HiCAM↗

NASA NTRS2019

Constraint-Based Off-Nominal Behavior Modeling for Europa Clipper

The risk analysis for the Europa Clipper mission evaluates the probability of mission failure based on the failure rates of individual components and dependencies among them. The probabilities are calculated by integrating over the intervals of time within which a fault occurs, accounting for an infinite number of cases. The response of the spacecraft to different faults can result in different schedules of activities, changing the intervals of integration. Europa currently uses models of spacecraft systems and components to simulate individual flight scenarios. The goal is to develop a framework for integrating, automating, and improving this modeling process. We describe an approach to generating the schedules for the different fault cases and determining the intervals for faults. It is not enough to just simulate individual cases because we are working with continuous variables that generate an infinite number of possible futures. Instead, we determine time windows within which certain faults can occur and use these time windows as bounds for integration. We found that determining these time windows is a constraint optimization problem. In order to represent these problems, we employ a language based on ontologies of behavior and scenarios. The language enables us to specify constraints in a simple, declarative syntax. A constraint-based analysis engine uses the declarative specification to identify bounds on system parameters and fill in details of behavior. For example, we created a detailed model of power generation, power use, and the corresponding effects on the battery in order to determine when an undervoltage fault can occur. An undervoltage during a trajectory correction maneuver requires that thrusting be interrupted for just enough time to recharge the battery such that the maneuver can be completed within battery limits. This behavior is generated based on the model to minimize the interruption time. For certain scenarios the constraint optimization problems were simple enough to be solved by hand, but the framework made the process substantially faster. It also produced solutions to other problems that we could not solve by hand or with existing tools and allowed us to generate and run many scenarios at once. The scenario language and engine greatly simplified the process of identifying time bounds and separating cases.

Everline, Chester J.↗

NASA NTRS2001

Reducing costs of the modified Antarctic Mapping mission through automated planning

This paper describes the automated mission planning system for MAMM, which dramatically reduced mission-planning costs to just a few workweeks, and enabled rapid generation of 'what-if' scenarios for evaluating mission-design trades. This latter capability informed several critical design decisions and was instrumental in accurately costing the mission.

reduce mission costs↗

NASA NTRS2013

Designing Scenarios for Controller-in-the-Loop Air Traffic Simulations

Within the Human Factors Division at NASA Ames Research Center the Airspace Operations Laboratory (AOL) is developing advanced automation concepts that help to transform the National Airspace System into NextGen, the Next Generation Air Transportation System. High-fidelity human-in-the-loop (HITL) simulations are used as a means to investigate and develop roles, responsibilities, support tools, and requirements for human operators and automation. This paper describes the traffic scenario design process and strategies as used by AOL researchers. Details are presented on building scenarios for specific simulation objectives using various design strategies. A focus is set on creating scenarios based on recorded real world traffic for terminal-area simulations.

scenario design↗

NASA NTRS2021

System Analysis and Generative Design for Icenode, a Buoyant Vehicle for Measuring Melt Rate Under Ice Shelves

Antarctic ice shelves buttress the Antarctic Ice Sheet from sliding into the ocean, and their collapse could trigger a meter or more of global sea level rise by the end of the century. Current state-of-the-art predictions for ice shelf behavior in a warming climate have large uncertainty, significantly hindered by a lack of in situ melt rate observations under ice shelves, especially near grounding zones. IceNode is a novel robotic vehicle under development at the NASA Jet Propulsion Laboratory to acquire such measurements, but presents a complex design problem owing to the large design space and conflicting performance requirements. Evaluating a preliminary design requires several types of tedious analysis which prevents rapid iteration and exploration of the design space. We present the implementation of a custom system analysis framework which was used to automate analysis such as resource budgeting, mission simulation, mass and buoyancy balancing, and static landing stability analysis of a given design configuration. Using this framework, we selected parameters which had the largest effects on design success and conducted a generative design study in which we programmatically varied a handful of parameters to generate 1540 different design candidates and test them across 160 different environmental scenarios. The custom built system analysis tool enabled rapid development of critical analysis and automated exploration of a large design space to guide preliminary design of the IceNode vehicle.

Gebara, Christine↗