Search NASA⌕ Search

SEARCH · Search NASA

Results for “integrated planning and learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Monte Carlo Tree Search for Integrated Planning, Learning, and Execution in Nondeterministic Python

We present a novel use of Monte Carlo Tree Search (MCTS),adapted to explore a search space produced by the choice points embedded in Python code. The choice points are non-deterministic assignment statements and subroutine calls. We present MCTS extensions required for doing tree search in this context which includes control constructs like hierarchical decomposition (subroutine calls), iterative while loops and conditional statements. We demonstrate how the system works in a simulated rideshare scenario in an urban setting, and present preliminary experiments as a proof of concept.

Automatic planning↗

Using neural networks and Dyna algorithm for integrated planning, reacting and learning in systems

The traditional AI answer to the decision making problem for a robot is planning. However, planning is usually CPU-time consuming, depending on the availability and accuracy of a world model. The Dyna system generally described in earlier work, uses trial and error to learn a world model which is simultaneously used to plan reactions resulting in optimal action sequences. It is an attempt to integrate planning, reactive, and learning systems. The architecture of Dyna is presented. The different blocks are described. There are three main components of the system. The first is the world model used by the robot for internal world representation. The input of the world model is the current state and the action taken in the current state. The output is the corresponding reward and resulting state. The second module in the system is the policy. The policy observes the current state and outputs the action to be executed by the robot. At the beginning of program execution, the policy is stochastic and through learning progressively becomes deterministic. The policy decides upon an action according to the output of an evaluation function, which is the third module of the system. The evaluation function takes the following as input: the current state of the system, the action taken in that state, the resulting state, and a reward generated by the world which is proportional to the current distance from the goal state. Originally, the work proposed was as follows: (1) to implement a simple 2-D world where a 'robot' is navigating around obstacles, to learn the path to a goal, by using lookup tables; (2) to substitute the world model and Q estimate function Q by neural networks; and (3) to apply the algorithm to a more complex world where the use of a neural network would be fully justified. In this paper, the system design and achieved results will be described. First we implement the world model with a neural network and leave Q implemented as a look up table. Next, we use a lookup table for the world model and implement the Q function with a neural net. Time limitations prevented the combination of these two approaches. The final section discusses the results and gives clues for future work.

Lima, Pedro↗

Integrating planning, execution, and learning

To achieve the goal of building an autonomous agent, the usually disjoint capabilities of planning, execution, and learning must be used together. An architecture, called MAX, within which cognitive capabilities can be purposefully and intelligently integrated is described. The architecture supports the codification of capabilities as explicit knowledge that can be reasoned about. In addition, specific problem solving, learning, and integration knowledge is developed.

Kuokka, Daniel R.↗

Automated Data Assimilation and Flight Planning for Multi-Platform Observation Missions

This is a progress report on an effort in which our goal is to demonstrate the effectiveness of automated data mining and planning for the daily management of Earth Science missions. Currently, data mining and machine learning technologies are being used by scientists at research labs for validating Earth science models. However, few if any of these advanced techniques are currently being integrated into daily mission operations. Consequently, there are significant gaps in the knowledge that can be derived from the models and data that are used each day for guiding mission activities. The result can be sub-optimal observation plans, lack of useful data, and wasteful use of resources. Recent advances in data mining, machine learning, and planning make it feasible to migrate these technologies into the daily mission planning cycle. We describe the design of a closed loop system for data acquisition, processing, and flight planning that integrates the results of machine learning into the flight planning process.

Oza, Nikunj↗

Learning to integrate reactivity and deliberation in uncertain planning and scheduling problems

This paper describes an approach to planning and scheduling in uncertain domains. In this approach, a system divides a task on a goal by goal basis into reactive and deliberative components. Initially, a task is handled entirely reactively. When failures occur, the system changes the reactive/deliverative goal division by moving goals into the deliberative component. Because our approach attempts to minimize the number of deliberative goals, we call our approach Minimal Deliberation (MD). Because MD allows goals to be treated reactively, it gains some of the advantages of reactive systems: computational efficiency, the ability to deal with noise and non-deterministic effects, and the ability to take advantage of unforseen opportunities. However, because MD can fall back upon deliberation, it can also provide some of the guarantees of classical planning, such as the ability to deal with complex goal interactions. This paper describes the Minimal Deliberation approach to integrating reactivity and deliberation and describe an ongoing application of the approach to an uncertain planning and scheduling domain.

Chien, Steve A.↗

Optimizing Air Traffic - Integrating Artificial Intelligence and Machine Learning in Flight Path Planning and 3D Airspace Visualization for Air Traffic Control

Air Traffic Control (ATC) systems are vital components of the National Airspace System (NAS). ATC, Airport Traffic Control Towers (ATCT), and Terminal Radar Approach Control (TRACON) are responsible for directing all flights departing from and arriving at airports, managing our nation’s airspace, preventing potential accidents, and ensuring that every flight is accounted for. However, these systems often face challenges in effectively monitoring the skies. Issues such as poor communication between operators, difficulty in performing operations, and the constant need for vigilance frequently burden ATC operators. Additionally, the projected increase in air traffic in the coming years will only exacerbate the stress associated with this role. To address these issues, we propose a system that assists ATC operators in situations such as handovers, emergencies, and routing aircraft to avoid weather hazards. Our solution includes an Artificial Intelligence (AI) and Machine Learning (ML)-based Flight Pathways Planning System (FPPS) designed to find the fastest and most optimal routes for aircraft, taking into account weather conditions, restricted terrain, and Extended-Range Twin-Engine Operational Performance Standards (ETOPS) ratings. The proposed Predictive Weather Planning Model, included in FPPS, adjusts routes based on real-time and forecasted weather conditions. Additionally, our NVIDIA Omniverse 3D Visualization System offers a highly interactive environment for better visualization and a clear view of the airspace. By incorporating these systems, the roles of ATC, ATCT, and TRACON operators will become more manageable and less stressful, equipping them to efficiently handle the growing density of airspace.

Regina Ayoubi↗

Preparation and Integration of ALHAT Precision Landing Technology for Morpheus Flight Testing

The Autonomous precision Landing and Hazard Avoidance Technology (ALHAT) project has developed a suite of prototype sensors for enabling autonomous and safe precision land- ing of robotic or crewed vehicles on solid solar bodies under varying terrain lighting condi- tions. The sensors include a Lidar-based Hazard Detection System (HDS), a multipurpose Navigation Doppler Lidar (NDL), and a long-range Laser Altimeter (LAlt). Preparation for terrestrial ight testing of ALHAT onboard the Morpheus free- ying, rocket-propelled ight test vehicle has been in progress since 2012, with ight tests over a lunar-like ter- rain eld occurring in Spring 2014. Signi cant work e orts within both the ALHAT and Morpheus projects has been required in the preparation of the sensors, vehicle, and test facilities for interfacing, integrating and verifying overall system performance to ensure readiness for ight testing. The ALHAT sensors have undergone numerous stand-alone sensor tests, simulations, and calibrations, along with integrated-system tests in special- ized gantries, trucks, helicopters and xed-wing aircraft. A lunar-like terrain environment was constructed for ALHAT system testing during Morpheus ights, and vibration and thermal testing of the ALHAT sensors was performed based on Morpheus ights prior to ALHAT integration. High- delity simulations were implemented to gain insight into integrated ALHAT sensors and Morpheus GN&C system performance, and command and telemetry interfacing and functional testing was conducted once the ALHAT sensors and electronics were integrated onto Morpheus. This paper captures some of the details and lessons learned in the planning, preparation and integration of the individual ALHAT sen- sors, the vehicle, and the test environment that led up to the joint ight tests.

Carson, John M., III↗

Next Generation Launch Technology Program Lessons Learned

In November 2002, NASA revised its Integrated Space Transportation Plan (ISTP) to evolve the Space Launch Initiative (SLI) to serve as a theme for two emerging programs. The first of these, the Orbital Space Plane (OSP), was intended to provide crew-escape and crew-transfer functions for the ISS. The second, the NGLT Program, developed technologies needed for safe, routine space access for scientific exploration, commerce, and national defense. The NGLT Program was comprised of 12 projects, ranging from fundamental high-temperature materials research to full-scale engine system developments (turbine and rocket) to scramjet flight test. The Program included technology advancement activities with a broad range of objectives, ultimate applications/timeframes, and technology maturity levels. An over-arching Systems Engineering and Analysis (SE&A) approach was employed to focus technology advancements according to a common set of requirements. Investments were categorized into three segments of technology maturation: propulsion technologies, launch systems technologies, and SE&A.

Cook, Stephen↗

Establishing a Distance Learning Plan for International Space Station (ISS) Interactive Video Education Events (IVEE)

Educational outreach is an integral part of the International Space Station (ISS) mandate. In a few scant years, the International Space Station has already established a tradition of successful, general outreach activities. However, as the number of outreach events increased and began to reach school classrooms, those events came under greater scrutiny by the education community. Some of the ISS electronic field trips, while informative and helpful, did not meet the generally accepted criteria for education events, especially within the context of the classroom. To make classroom outreach events more acceptable to educators, the ISS outreach program must differentiate between communication events (meant to disseminate information to the general public) and education events (designed to facilitate student learning). In contrast to communication events, education events: are directed toward a relatively homogeneous audience who are gathered together for the purpose of learning, have specific performance objectives which the students are expected to master, include a method of assessing student performance, and include a series of structured activities that will help the students to master the desired skill(s). The core of the ISS education events is an interactive videoconference between students and ISS representatives. This interactive videoconference is to be preceded by and followed by classroom activities which help the students aftain the specified learning objectives. Using the interactive videoconference as the centerpiece of the education event lends a special excitement and allows students to ask questions about what they are learning and about the International Space Station and NASA. Whenever possible, the ISS outreach education events should be congruent with national guidelines for student achievement. ISS outreach staff should recognize that there are a number of different groups that will review the events, and that each group has different criteria for acceptance. For example, school administrators are more likely to be concerned about an event meeting national standards and the cost of the event. In contrast, a teacher's acceptance of an education event may be directly related to the amount of extra work the event imposes upon that teacher. ISS education events must be marketed differently to the different groups of educators, and must never increase the workload of the average teacher.

Wallington, Clint↗

Future applications of artificial intelligence to Mission Control Centers

Future applications of artificial intelligence to Mission Control Centers are presented in the form of the viewgraphs. The following subject areas are covered: basic objectives of the NASA-wide AI program; inhouse research program; constraint-based scheduling; learning and performance improvement for scheduling; GEMPLAN multi-agent planner; planning, scheduling, and control; Bayesian learning; efficient learning algorithms; ICARUS (an integrated architecture for learning); design knowledge acquisition and retention; computer-integrated documentation; and some speculation on future applications.

Friedland, Peter↗

Reinforcement Learning for Spacecraft Navigation & Environment Characterization in the Planar-Restricted Two-Body Problem

As science, exploration, and commercial space missions become increasingly complex, so does the need for efficient, autonomous, and integrated spacecraft navigation and operations techniques. Key operational functions, including data collection and transmission, environment characterization, systems constraints, human factors, and navigation, often are intertwined and conflicted. Deep Reinforcement Learning (DRL) offers a framework for addressing integrated spacecraft navigation and planning in an uncertain dynamical environment. The goal of this study is to evaluate the utility of DRL for integrated spacecraft navigation and planning. This is achieved by developing a simple environmental characterization training environment in the Planar-Restricted 2-Body Problem (PR2BP), establishing benchmarks and heuristic baselines, and designing a previously unstudied Markov Decision Process (MDP) formulation. This MDP formulation enables the spacecraft DRL agents to appropriately balance navigation and actuation capabilities. The resulting DRL-derived policy exceeds a random or untrained policy and meets or exceeds the level of performance of a heuristic without actuation. In the process, valuable intuition is gained about the problem with insight into how DRL methods could scale to increasingly more realistic scenarios, including net-work design and training architectures, efficient state space representations, and methods for encouraging exploration in a parametric action space, among others.

navigation↗

Design, Integration, Certification and Testing of the Orion Crew Module Propulsion System

The Orion Crew Module Propulsion Reaction Control System is currently complete and ready for flight as part of the Orion program's first flight test, Exploration Flight Test One (EFT-1). As part of the first article design, build, test, and integration effort, several key lessons learned have been noted and are planned for incorporation into the next build of the system. This paper provides an overview of those lessons learned and a status on the Orion propulsion system progress to date.

McKay, Heather↗

Lessons Learned In Developing Multiple Distributed Planning Systems for the International Space Station

The planning processes for the International Space Station (ISS) Program are quite complex. Detailed mission planning for ISS on-orbit operations is a distributed function. Pieces of the on-orbit plan are developed by multiple planning organizations, located around the world, based on their respective expertise and responsibilities. The "pieces" are then integrated to yield the final detailed plan that will be executed onboard the ISS. Previous space programs have not distributed the planning and scheduling functions to this extent. Major ISS planning organizations are currently located in the United States (at both the NASA Johnson Space Center (JSC) and NASA Marshall Space Flight Center (MSFC)), in Russia, in Europe, and in Japan. Software systems have been developed by each of these planning organizations to support their assigned planning and scheduling functions. Although there is some cooperative development and sharing of key software components, each planning system has been tailored to meet the unique requirements and operational environment of the facility in which it operates. However, all the systems must operate in a coordinated fashion in order to effectively and efficiently produce a single integrated plan of ISS operations, in accordance with the established planning processes. This paper addresses lessons learned during the development of these multiple distributed planning systems, from the perspective of the developer of one of the software systems. The lessons focus on the coordination required to allow the multiple systems to operate together, rather than on the problems associated with the development of any particular system. Included in the paper is a discussion of typical problems faced during the development and coordination process, such as incompatible development schedules, difficulties in defining system interfaces, technical coordination and funding for shared tools, continually evolving planning concepts/requirements, programmatic and budget issues, and external influences. Techniques that mitigated some of these problems will also be addressed, along with recommendations for any future programs involving the development of multiple planning and scheduling systems. Many of these lessons learned are not unique to the area of planning and scheduling systems, so may be applied to other distributed ground systems that must operate in concert to successfully support space mission operations.

Maxwell, Theresa G.↗

Lessons Learned in Developing Multiple Distributed Planning Systems for the International Space Station

The planning processes for the International Space Station (ISS) Program are quite complex. Detailed mission planning for ISS on-orbit operations is a distributed function. Pieces of the on-orbit plan are developed by multiple planning organizations, located around the world, based on their respective expertise and responsibilities. The pieces are then integrated to yield the final detailed plan that will be executed onboard the ISS. Previous space programs have not distributed the planning and scheduling functions to this extent. Major ISS planning organizations are currently located in the United States (at both the NASA Johnson Space Center (JSC) and NASA Marshall Space Flight Center (MSFC)), in Russia, in Europe, and in Japan. Software systems have been developed by each of these planning organizations to support their assigned planning and scheduling functions. Although there is some cooperative development and sharing of key software components, each planning system has been tailored to meet the unique requirements and operational environment of the facility in which it operates. However, all the systems must operate in a coordinated fashion in order to effectively and efficiently produce a single integrated plan of ISS operations, in accordance with the established planning processes. This paper addresses lessons learned during the development of these multiple distributed planning systems, from the perspective of the developer of one of the software systems. The lessons focus on the coordination required to allow the multiple systems to operate together, rather than on the problems associated with the development of any particular system. Included in the paper is a discussion of typical problems faced during the development and coordination process, such as incompatible development schedules, difficulties in defining system interfaces, technical coordination and funding for shared tools, continually evolving planning concepts/requirements, programmatic and budget issues, and external influences. Techniques that mitigated some of these problems will also be addressed, along with recommendations for any future programs involving the development of multiple planning and scheduling systems. Many of these lessons learned are not unique to the area of planning and scheduling systems, so may be applied to other distributed ground systems that must operate in concert to successfully support space mission operations.

Maxwell, Theresa G.↗

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗

Scheduling lessons learned from the Autonomous Power System

The Autonomous Power System (APS) project at NASA LeRC is designed to demonstrate the applications of integrated intelligent diagnosis, control, and scheduling techniques to space power distribution systems. The project consists of three elements: the Autonomous Power Expert System (APEX) for Fault Diagnosis, Isolation, and Recovery (FDIR); the Autonomous Intelligent Power Scheduler (AIPS) to efficiently assign activities start times and resources; and power hardware (Brassboard) to emulate a space-based power system. The AIPS scheduler was tested within the APS system. This scheduler is able to efficiently assign available power to the requesting activities and share this information with other software agents within the APS system in order to implement the generated schedule. The AIPS scheduler is also able to cooperatively recover from fault situations by rescheduling the affected loads on the Brassboard in conjunction with the APEX FDIR system. AIPS served as a learning tool and an initial scheduling testbed for the integration of FDIR and automated scheduling systems. Many lessons were learned from the AIPS scheduler and are now being integrated into a new scheduler called SCRAP (Scheduler for Continuous Resource Allocation and Planning). This paper will service three purposes: an overview of the AIPS implementation, lessons learned from the AIPS scheduler, and a brief section on how these lessons are being applied to the new SCRAP scheduler.

Ringer, Mark J.↗