Search NASA⌕ Search

Engineering topics

Brian Coltin

Publications and source records attributed to Brian Coltin.

15 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Robust Semantic Mapping and Localization on a Free-Flying Robot in Microgravity

We propose a system that uses semantic object detections to localize a microgravity free-flyer. Many applications require absolute localization in a known reference frame, such as the execution of waypoint trajectories defined by human operators. Classical geometric methods build a map of point features, which may not be able to be associated after lighting or environmental changes. By contrast, semantics remain invariant to changes up to the robustness of the detection algorithm and motion of the semantic objects. In this work, we describe our approaches for both offline semantic map generation as well as online localization against a semantic map, intended to run in real-time on the robot. We additionally demonstrate how our semantic localizer outperforms image-feature matching in some cases, and show the robustness of the algorithm to environmental changes. Crucially, we show in our experiments that when semantics are used to supplement point features, localization is always improved. To our knowledge, these experiments demonstrate the first use of learned semantics for localization on a free-flying robot in microgravity.

Localization↗
NASA NTRS2021

Integrated System for Autonomous and Adaptive Caretaking (ISAAC) Simulated Cargo Logistics Demo Video

This video shows a simulation of autonomous cargo logistics using ISAAC (Integrated System for Autonomous and Adaptive Caretaking). The video shows autonomous planning and execution of a cargo scenario in which the R2 robot moves a cargo bag from a stowed location to a transfer location in the US Lab on the International Space Station. The Astrobee robot then moves the bag from the transfer location to a new temporary stowage location in the Japanese Experiment Module (JEM).

ISAAC↗
NASA NTRSDate not supplied

Astrobee On-Orbit Commissioning Manuscript and Presentation

The Astrobee free flying robots operate autonomously inside the International Space Station (ISS) with oversight from a ground operator or ISS crew. They replace the Synchronized Position Hold Engage and Reorient Experimental Satellites (SPHERES) as research platforms for zero-g free-flying robotics. Astrobee can also serve as a mobile camera/sensor platform for flight and payload controllers to improve ISS operations. Development began in late 2014, and flight hardware deployed to the ISS on several launches starting from November 2018 to October 2019. Shortly after the first two robots arrived in April 2019, we began a series of commissioning activities to validate the Astrobee robots. This paper reviews the Astrobee system and describes the on-orbit commissioning activities and results.

ISS↗
NASA NTRSDate not supplied

Astrobee On-Orbit Commissioning

The Astrobee free flying robots operate autonomously inside the International Space Station (ISS) with oversight from a ground operator or ISS crew. They replace the Synchronized Position Hold Engage and Reorient Experimental Satellites (SPHERES) as research platforms for zero-g free-flying robotics. Astrobee can also serve as a mobile camera/sensor platform for flight and payload controllers to improve ISS operations. Development began in late 2014, and flight hardware deployed to the ISS on several launches starting from November 2018 to October 2019. Shortly after the first two robots arrived in April 2019, we began a series of commissioning activities to validate the Astrobee robots. This paper reviews the Astrobee system and describes the on-orbit commissioning activities and results.

ISS↗
NASA NTRSDate not supplied

DELTA: An Open-Source Framework to Simplify Deep Learning with Satellite Imagery

DELTA (Deep Earth Learning, Tools, and Analysis) is an open-source framework developed at NASA for deep learning on satellite imagery based on tensorflow. It helps simplify data engineering and preprocessing steps and reduces the need for a lot of the boilerplate code that needs written to make datasets palatable for machine learning. This lets data scientists focus on model development while DELTA handles the grunt work. This presentation will demonstrate DELTA’s functionality and share some examples from an active project using it for flood mapping.

Michael von Pohle↗
NASA NTRSDate not supplied

ISAAC: An Integrated System for Autonomous and Adaptive Caretaking

The Integrated System for Autonomous and Adaptive Caretaking (ISAAC) project is developing technology for autonomous caretaking of spacecraft, primarily during uncrewed mission phases. ISAAC aims to integrate autonomous intra-vehicular robots (IVR) with spacecraft infrastructure (power, life support, etc.) and ground control. It focuses on capabilities required for NASA’s Gateway cis-lunar outpost that also apply to human missions to Mars and beyond. Its development strategy is to test using existing IVR on the ISS (the Astrobee free-flyer and Robonaut dextrous manipulator) as an analog for future IVR on Gateway.

Trey Smith↗
NASA NTRSDate not supplied

DELTA: An Open-Source Framework to Simplify Machine Learning with Satellite Imagery

DELTA (Deep Earth Learning, Tools, and Analysis) is an open-source framework developed at NASA to simplify running and training machine learning (ML) models on satellite imagery. Users new to machine learning can run existing ML models on satellite imagery with minimal setup and configuration. For experienced ML users, DELTA helps simplify data engineering, preprocessing steps, and reduces the need for boilerplate code that needs written to make satellite imagery datasets palatable for machine learning. This lets data scientists focus on model development while DELTA handles the imagery manipulation. This presentation will demonstrate DELTA’s functionality and share some examples from an active project using it for flood mapping using imagery from multiple satellite sources.

deep learning↗
NASA NTRSDate not supplied

AstroLoc: An Efficient and Robust Localizer for a Free-flying Robot

We present AstroLoc, an efficient and robust monocular visual-inertial graph-based localization system used by the Astrobee free-flying robots onboard the International Space Station (ISS). We provide a novel localization system that limits the traditionally higher computation times for graph-based localization systems and enables the resource constrained Astrobee robots to benefit from their increased accuracy. We also introduce methods for handling cheirality issues for visual odometry and localization factors that further increase localization robustness. We evaluate the performance of AstroLoc on a dataset of ISS activities and show that it greatly improves pose, velocity, and IMU bias estimation accuracy while efficiently running in a limited computation environment. The source code for AstroLoc is released to the public.

Localization↗
NASA NTRSDate not supplied

Astrobee's Multi-year Activities at the International Space Station's Japanese Experimental Module

The Astrobee free-flying robots recently completed their third successful year of operations, housed in the Japanese Experimental Module (JEM) on the International Space Station. We summarize the three years of operation, giving special attention to JAXA's 1st and 2nd Kibo Robot Programming Challenge (RPC) and the mapping processes and tools that make Astrobees' autonomous operation possible. The JEM is an ever changing, dynamic environment where light settings, cargo, payloads, and crew members constantly move and interact with one another. The 1st JAXA Kibo RPC event, a collaboration between JAXA and NASA, was held in 2020. Students from several countries in the Asia-Pacific region competed in programming challenges with a simulated Astrobee. The finalists were then invited to run their code on an actual Astrobee in the JEM. For the final round, students programmed Astrobee to visit three different locations to obtain data that would instruct the robot to complete a final task with the participation of ISS crew. The first competition was a tremendous success, leading to an equally successful 2nd JAXA Kibo RPC in 2021 with even larger participation. The 3rd JAXA Kibo RPC will occur in 2022 expanding further to incorporate US participants. These activities led to several firsts in Astrobee’s history: operation of an Astrobee free-flying robot without crew supervision in preparation for on-orbit operations, autonomous image acquisition towards updates of the navigation map, non-NASA code running on the robot (both from JAXA and participating students), two heterogeneous free-flying robots from two different space agencies working together (Int-Ball and Astrobee) during the final event in 2020, the first payload using Astrobee, and having Astrobee controlled from a non-NASA location (Tsukuba Space Center). The preparation towards these activities involved constant evaluation of the different components of Astrobee's systems, specially mapping and localization. The paper describes the evolution of these systems such as the improvements made in localization to reduce localization drift by using graph-based optimization instead of the extended Kalman Filter localizer. Additionally, it reports on the mapping process and analysis tools created to validate map consistency across different activities in the constantly changing JEM environment. These enhancements have enabled the Astrobee facility to successfully execute over 100 ISS activities supporting over a dozen researchers and partners around the world.

Astrobee↗
NASA NTRSDate not supplied

Mapping the ISS with the Autonomous Free-Flying Astrobee Robots

Space intra-vehicular robots (IVR) can provide coverage survey and targeted close-up inspection imagery capabilities that improve ground operator situation awareness. We report on a series of activities using the autonomous Astrobee robotic free-flyers that mapped the interior of three ISS modules, generating a panoramic tour of the ISS interior without astronaut manual photography. For the ISS, these capabilities can complement or replace time-consuming crew inspection tasks, providing panoramic data products that are easier to navigate than the baseline crew safety videos. For future exploration vehicles with extended uncrewed mission phases, robotic inspection will play a more critical role of providing the primary “eyes in the sky” for vehicle operators.

robotics↗
NASA NTRSDate not supplied

Unsupervised Change Detection for Space Habitats Using 3D Point Clouds

This work presents an algorithm for scene change detection from point clouds to enable autonomous robotic caretaking in future space habitats. Autonomous robotic systems will help maintain future deep-space habitats, such as the Gateway space station, which will be uncrewed for extended periods. Existing scene analysis software used on the International Space Station (ISS) relies on manually-labeled images for detecting changes. In contrast, the algorithm presented in this work uses raw, unlabeled point clouds as inputs. The algorithm first applies modified Expectation-Maximization Gaussian Mixture Model (GMM) clustering to two input point clouds. It then performs change detection by comparing the GMMs using the Earth Mover’s Distance. The algorithm is validated quantitatively and qualitatively using a test dataset collected by an Astrobee robot in the NASA Ames Granite Lab comprising single frame depth images taken directly by Astrobee and full-scene reconstructed maps built with RGB-D and pose data from Astrobee. The runtimes of the approach are also analyzed in depth. The source code is publicly released to promote further development.

robotics↗
NASA NTRSDate not supplied

Astrobee: The International Space Station Robotic Freeflyer

The Astrobees are free-flying robots that operate inside the International Space Station (ISS) and were launched to the ISS in 2019. Designed as a mobile camera, an astronaut assistant, and a research platform, they have successfully performed hundreds of activities in space supporting dozens of research projects. The robots were designed to overcome multiple challenges unique to the ISS environment, including safety, upgradeability and maintainability, limited mass and computation, and unique localization challenges from lack of gravity and a constantly changing environment. Robots such as Astrobee, have the capacity to become caretakers for future spacecraft, working to monitor and keep systems operating smoothly while crew are away. This talk will give an overview of the Astrobee robots, with an emphasis on Astrobee’s development, robotic software, and its successful use on the ISS.

robotics↗