Search NASA⌕ Search

SEARCH · Search NASA

Results for “Augmented reality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

ExaDigiT/RAPS

ExaDigiT is a framework for developing comprehensive digital twins of liquid-cooled supercomputers, which has three main modules: (1) a python-based Resource Allocator and Power Simulator (RAPS), (2) a Modelica-based Thermo-Fluidic cooling model, and (3) a C++-based augmented reality model built on Unreal Engine 5. RAPS either simulates workloads or replays historical workloads from system telemetry, and is able to predict dynamic energy consumption, as well as interact with the cooling model to predict cooling system behavior. Such a tool can be used together with reinforcement learning algorithms to provide an end-to-end optimization tool for data centers.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗

datacenterCoolingModel

ExaDigiT is a framework for developing comprehensive digital twins of liquid-cooled supercomputers, which has three main modules: (1) a python-based Resource Allocator and Power Simulator (RAPS), (2) a Modelica-based Thermo-Fluidic cooling model, and (3) a C++-based augmented reality model built on Unreal Engine 5. The Modelica-based cooling model is primarily built-on the open-source Transient Simulation Framework of Reconfigurable Models (TRANSFORM) library and the open-source autocsm library. The library follows the templating architecture developed in the TRANSFORM and the autocsm libraries. This tool can be easily extended to model other Frontier-like liquid cooled supercomputers.

Kumar, Vineet [Oak Ridge National Laboratory (ORNL↗

Robust multiresonant nonlocal metasurfaces by rational design

Abstract Dielectric metasurfaces supporting optical resonances have become a promising platform for quantum and nonlinear optics. However, resonant metasurfaces remain limited in their capacity to independently control the behavior of many distinct resonances despite efforts in computational optimization and inverse design. In this work, we overcome longstanding limitations by introducing a generalized rational design paradigm based on symmetry. Specifically, we use symmetry-broken metasurfaces with periodic “quadromer” lattices comprised of four nanostructures per unit cell to enable extensive control of multiple optical resonances. The rationally designed metasurfaces are readily fabricable, and we experimentally demonstrate metasurfaces that support up to four high Q-factor resonances with deliberately chosen free-space polarizations, spectral separations, and mode profiles. Our design paradigm may unlock new applications for multiresonant metasurfaces in quantum and nonlinear optics, optical sensing, and augmented reality displays.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Spatially Adaptive Tunable Lighting Control System with Expanded Wellness and Energy Saving Benefits

Lighting design is becoming increasingly complex, including active dimming for energy savings and spectral tuning for human wellbeing. Modern commercial lighting control systems are already difficult to use, and maintain, and even with so-called Smart Lighting, optimizing light settings is rapidly exceeding the capabilities of direct human control, limiting the adoption of lighting control systems. The growing importance of Occupancy Centric Controls (OCC) for advanced building management systems can be applied to the development of autonomous lighting system controls needed to drive the adoption of advanced lighting controls for improved adaptive “sculpted illumination” for both greater energy savings and broader human wellbeing perspectives. This project, led by Rensselaer Polytechnic Institute, entitled “Spatially adaptive tunable lighting control system with expanded wellness and energy saving benefits” will develop and test an entirely new platform for automated optimized lighting design and control that requires little or no human engagement, yet contours lighting profiles automatically to minimize lighting energy use. We call this approach to lighting control “dynamic light sculpting” since the right amount of illumination is automatically delivered to occupants in real time. Since the system uses new, privacy-preserving occupant position and pose detection technologies developed by Rensselaer for broad OCC building applications, the control system will analyze how to deliver the right amount of the right type of illumination only where and when it is needed based these OCC platforms. To create these powerful autonomous lighting control platforms, the project will integrate evolving augmented reality (AR) and virtual reality (VR) tools with sophisticated lighting and interior design toolkits to create interactive lighting design and control simulators. Using the quickly growing paradigm of digital twins, these tools will integrate light fixture properties, occupancy sensor data, interior design data, and lighting specifications to accurately visualize how various design concepts interact with simulated yet realistic human activities that occur in commercial office buildings of various types. These advanced digital twin design and simulation tools will be combined with a new class of digitally-programmable LED lighting fixtures that can dynamically change the spectral content and direction of light emission. When fully integrated, the new autonomous lighting control system will take all of the guesswork out of optimizing light quality while minimizing energy consumption. Digital twin tools will simplify the design, installation, commissioning, operation, and maintenance of future energy-efficient lighting systems. It should be possible to reduce lighting energy costs from 40% to 70% with OCC based dynamic light sculpting systems. The work will be led by the Center for Lighting Enabled Systems and Applications (LESA) and the Center for Architectural Science and Ecology (CASE), both at Renssealer Polytechnic Institute; Lumileds, a global leader in the development of advanced LED systems; and HKS, a leading global architecture design firm. This interdisciplinary project team’s experience includes all of the design simulation tools, VR/AR digital twin visualization technology, advanced occupancy sensing technology, and complex control system design capabilities that will revolutionize lighting design and control technology to autonomously deliver high-quality lighting that improves human health and wellbeing while simultaneously maximizing lighting energy savings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Prototyping of 2D and 3D photonic nanodevices by Electron Beam Lithography (CRADA Final Report)

Team developed a large area and fast electron beam lithographic for patterning visible wavelength optical materials to create 2 or 3 dimensional gratings and metasurfaces. Metasurfaces,are arrays of nanoantennas or optical resonators, are dielectric or plasmonic structures that allow control of the amplitude, phase, and polarization of light by locally modulating its wavefront The team was able to write a 1cm2 pattern with resolution better than 30 nm without visible discontinuities in the pattern. Performance to this specification allowed prototyping of optical lenses for production applications such as augmented reality goggles (optical displays) and other applications for visible light manipulations

36 MATERIALS SCIENCE↗

Information Power Grid: Distributed High-Performance Computing and Large-Scale Data Management for Science and Engineering

We use the term "Grid" to refer to distributed, high performance computing and data handling infrastructure that incorporates geographically and organizationally dispersed, heterogeneous resources that are persistent and supported. This infrastructure includes: (1) Tools for constructing collaborative, application oriented Problem Solving Environments / Frameworks (the primary user interfaces for Grids); (2) Programming environments, tools, and services providing various approaches for building applications that use aggregated computing and storage resources, and federated data sources; (3) Comprehensive and consistent set of location independent tools and services for accessing and managing dynamic collections of widely distributed resources: heterogeneous computing systems, storage systems, real-time data sources and instruments, human collaborators, and communications systems; (4) Operational infrastructure including management tools for distributed systems and distributed resources, user services, accounting and auditing, strong and location independent user authentication and authorization, and overall system security services The vision for NASA's Information Power Grid - a computing and data Grid - is that it will provide significant new capabilities to scientists and engineers by facilitating routine construction of information based problem solving environments / frameworks. Such Grids will knit together widely distributed computing, data, instrument, and human resources into just-in-time systems that can address complex and large-scale computing and data analysis problems. Examples of these problems include: (1) Coupled, multidisciplinary simulations too large for single systems (e.g., multi-component NPSS turbomachine simulation); (2) Use of widely distributed, federated data archives (e.g., simultaneous access to metrological, topological, aircraft performance, and flight path scheduling databases supporting a National Air Space Simulation systems}; (3) Coupling large-scale computing and data systems to scientific and engineering instruments (e.g., realtime interaction with experiments through real-time data analysis and interpretation presented to the experimentalist in ways that allow direct interaction with the experiment (instead of just with instrument control); (5) Highly interactive, augmented reality and virtual reality remote collaborations (e.g., Ames / Boeing Remote Help Desk providing field maintenance use of coupled video and NDI to a remote, on-line airframe structures expert who uses this data to index into detailed design databases, and returns 3D internal aircraft geometry to the field); (5) Single computational problems too large for any single system (e.g. the rotocraft reference calculation). Grids also have the potential to provide pools of resources that could be called on in extraordinary / rapid response situations (such as disaster response) because they can provide common interfaces and access mechanisms, standardized management, and uniform user authentication and authorization, for large collections of distributed resources (whether or not they normally function in concert). IPG development and deployment is addressing requirements obtained by analyzing a number of different application areas, in particular from the NASA Aero-Space Technology Enterprise. This analysis has focussed primarily on two types of users: the scientist / design engineer whose primary interest is problem solving (e.g. determining wing aerodynamic characteristics in many different operating environments), and whose primary interface to IPG will be through various sorts of problem solving frameworks. The second type of user is the tool designer: the computational scientists who convert physics and mathematics into code that can simulate the physical world. These are the two primary users of IPG, and they have rather different requirements. The results of the analysis of the needs of these two types of users provides a broad set of requirements that gives rise to a general set of required capabilities. The IPG project is intended to address all of these requirements. In some cases the required computing technology exists, and in some cases it must be researched and developed. The project is using available technology to provide a prototype set of capabilities in a persistent distributed computing testbed. Beyond this, there are required capabilities that are not immediately available, and whose development spans the range from near-term engineering development (one to two years) to much longer term R&D (three to six years). Additional information is contained in the original.

Johnston, William E.↗

Fiber-Scanned Microdisplays

Helmet- and head-mounted display systems, denoted fiber-scanned microdisplays, have been proposed to provide information in an "augmented reality" format (meaning that the information would be optically overlaid on the user's field of view).

Crossman-Bosworth, Janet↗

Human-Robot Interaction Directed Research Project

Human-robot interaction (HRI) is about understanding and shaping the interactions between humans and robots (Goodrich & Schultz, 2007). It is important to evaluate how the design of interfaces and command modalities affect the human s ability to perform tasks accurately, efficiently, and effectively (Crandall, Goodrich, Olsen Jr., & Nielsen, 2005) It is also critical to evaluate the effects of human-robot interfaces and command modalities on operator mental workload (Sheridan, 1992) and situation awareness (Endsley, Bolt , & Jones, 2003). By understanding the effects of interface design on human performance, workload, and situation awareness, interfaces can be developed that support the human in performing tasks with minimal errors and with appropriate interaction time and effort. Thus, the results of research on human-robot interfaces have direct implications for design. Because the factors associated with interfaces and command modalities in HRI are too numerous to address in 3 years of research, the proposed research concentrates on three manageable areas applicable to National Aeronautics and Space Administration (NASA) robot systems. These topic areas emerged from the Fiscal Year (FY) 2011 work that included extensive literature reviews and observations of NASA systems. The three topic areas are: 1) video overlays, 2) camera views, and 3) command modalities. Each area is described in detail below, along with relevance to existing NASA human-robot systems. In addition to studies in these three topic areas, a workshop is proposed for FY12. The workshop will bring together experts in human-robot interaction and robotics to discuss the state of the practice as applicable to research in space robotics. Studies proposed in the area of video overlays consider two factors in the implementation of augmented reality (AR) for operator displays during teleoperation. The first of these factors is the type of navigational guidance provided by AR symbology. In the proposed studies, participants performance during teleoperation of a robot arm will be compared when they are provided with command-guidance symbology (that is, directing the operator what commands to make) or situation-guidance symbology (that is, providing natural cues so that the operator can infer what commands to make). The second factor for AR symbology is the effects of overlays that are either superimposed or integrated into the external view of the world. A study is proposed in which the effects of superimposed and integrated overlays on operator task performance during teleoperated driving tasks are compared

Rochlis, Jennifer↗

The NASA Tournament Laboratory (NTL): Improving Data Access at PDS while Spreading Joy and Engaging Students through 16 Micro-Contests

NASA PDS hosts terabytes of valuable data from hundreds of data sources and spans decades of research. Data is stored on flat-file systems regulated through careful meta dictionaries. PDS's data is available to the public through its website which supports data searches through drill-down navigation. While the system returns data quickly, result sets in response to identical input differ depending on the drill-down path a user follows. To correct this Issue, to allow custom searching, and to improve general accessibility, PDS sought to create a new data structure and API, and to use them to build applications that are a joy to use and showcase the value of the data to students, teachers and citizens. PDS engaged TopCoder and Harvard Business School through the NTL to pursue these objectives in a pilot effort. Scope was limited to Small Bodies Node data. NTL analyzed data, proposed a solution, and implemented it through a series of micro-contests. Contest focused on different segments of the problem; conceptualization, architectural design, implementation, testing, etc. To demonstrate the utility of the completed solution, NTL developed web-based and mobile applications that can compare targets, regardless of mission. To further explore the potential of the solution NTL hosted "Mash-up" challenges that integrated the API with other publically available assets, to produce consumer and teaching applications, including an Augmented Reality iPad tool. Two contests were also posted to middle and high school students via the NoNameSite.com platform, and as a result of these contests, PDS/SBN has initiated a Facebook program. These contests defined and implemented a data warehouse with the necessary migration tools to transform legacy data, produced a public web interface for the new search, developed a public API, and produced four mobile applications that we expect to appeal to users both within and, without the academic community.

LaMora, Andy↗

NASA Tech Briefs, September 2013

Topics include: ISS Ammonia Leak Detection Through X-Ray Fluorescence; A System for Measuring the Sway of the Vehicle Assembly Building; Fast, High-Precision Readout Circuit for Detector Arrays; Victim Simulator for Victim Detection Radar; Hydrometeor Size Distribution Measurements by Imaging the Attenuation of a Laser Spot; Quasi-Linear Circuit; High-Speed, High-Resolution Time-to-Digital Conversion; Li-Ion Battery and Supercapacitor Hybrid Design for Long Extravehicular Activities; Ultrasonic Low-Friction Containment Plate for Thermal and Ultrasonic Stir Weld Processes; High-Powered, Ultrasonically Assisted Thermal Stir Welding; Next-Generation MKIII Lightweight HUT/Hatch Assembly; Centrifugal Sieve for Gravity-Level-Independent Size; Segregation of Granular Materials; Ion Exchange Technology Development in Support of the Urine Processor Assembly; Nickel-Graphite Composite Compliant Interface and/or Hot Shoe Material; UltraSail CubeSat Solar Sail Flight Experiment; Mechanism for Deploying a Long, Thin-Film Antenna From a Rover; Counterflow Regolith Heat Exchanger; Acquisition and Retaining Granular Samples via a Rotating Coring Bit; Very-Low-Cost, Rugged Vacuum System; Medicine Delivery Device With Integrated Sterilization and Detection; FRET-Aptamer Assays for Bone Marker Assessment, C-Telopeptide, Creatinine, and Vitamin D; Multimode Directional Coupler for Utilization of Harmonic Frequencies from TWTAs; Dual-Polarization, Multi-Frequency Antenna Array for use with Hurricane Imaging Radiometer; Complementary Barrier Infrared Detector (CBIRD) Contact Methods; Autonomous Control of Space Nuclear Reactors; High-Power, High-Speed Electro-Optic Pockels Cell Modulator; Covariance Analysis Tool (G-CAT) for Computing Ascent, Descent, and Landing Errors; Enigma Version 12; Micrometeoroid and Orbital Debris (MMOD) Shield Ballistic Limit Analysis Program; Spitzer Telemetry Processing System; Planetary Protection Bioburden Analysis Program; Wing Leading Edge RCC Rapid Response Damage Prediction Tool (IMPACT2); ISSM: Ice Sheet System Model; Automated Loads Analysis System (ATLAS); Integrated Main Propulsion System Performance Reconstruction Process/Models. Phoenix Telemetry Processor; Contact Graph Routing Enhancements Developed in ION for DTN; GFEChutes Lo-Fi; Advanced Strategic and Tactical Relay Request Management for the Mars Relay Operations Service; Software for Generating Troposphere Corrections for InSAR Using GPS and Weather Model Data; Ionospheric Specifications for SAR Interferometry (ISSI); Implementation of a Wavefront-Sensing Algorithm; Sally Ride EarthKAM - Automated Image Geo-Referencing Using Google Earth Web Plug-In; Trade Space Specification Tool (TSST) for Rapid Mission Architecture (Version 1.2); Acoustic Emission Analysis Applet (AEAA) Software; Memory-Efficient Onboard Rock Segmentation; Advanced Multimission Operations System (ATMO); Robot Sequencing and Visualization Program (RSVP); Automating Hyperspectral Data for Rapid Response in Volcanic Emergencies; Raster-Based Approach to Solar Pressure Modeling; Space Images for NASA JPL Android Version; Kinect Engineering with Learning (KEWL); Spacecraft 3D Augmented Reality Mobile App; MPST Software: grl_pef_check; Real-Time Multimission Event Notification System for Mars Relay; SIM_EXPLORE: Software for Directed Exploration of Complex Systems; Mobile Timekeeping Application Built on Reverse-Engineered JPL Infrastructure; Advanced Query and Data Mining Capabilities for MaROS; Jettison Engineering Trajectory Tool; MPST Software: grl_suppdoc; PredGuid+A: Orion Entry Guidance Modified for Aerocapture; Planning Coverage Campaigns for Mission Design and Analysis: CLASP for DESDynl; and Space Place Prime.

Source record↗

Human-Robot Interaction Directed Research Project

Human-robot interaction (HRI) is a discipline investigating the factors affecting the interactions between humans and robots. It is important to evaluate how the design of interfaces and command modalities affect the human's ability to perform tasks accurately, efficiently, and effectively when working with a robot. By understanding the effects of interface design on human performance, workload, and situation awareness, interfaces can be developed to appropriately support the human in performing tasks with minimal errors and with appropriate interaction time and effort. Thus, the results of research on human-robot interfaces have direct implications for the design of robotic systems. This DRP concentrates on three areas associated with interfaces and command modalities in HRI which are applicable to NASA robot systems: 1) Video Overlays, 2) Camera Views, and 3) Command Modalities. The first study focused on video overlays that investigated how Augmented Reality (AR) symbology can be added to the human-robot interface to improve teleoperation performance. Three types of AR symbology were explored in this study, command guidance (CG), situation guidance (SG), and both (SCG). CG symbology gives operators explicit instructions on what commands to input, whereas SG symbology gives operators implicit cues so that operators can infer the input commands. The combination of CG and SG provided operators with explicit and implicit cues allowing the operator to choose which symbology to utilize. The objective of the study was to understand how AR symbology affects the human operator's ability to align a robot arm to a target using a flight stick and the ability to allocate attention between the symbology and external views of the world. The study evaluated the effects type of symbology (CG and SG) has on operator tasks performance and attention allocation during teleoperation of a robot arm. The second study expanded on the first study by evaluating the effects of the type of navigational guidance (CG and SG) on operator task performance and attention allocation during teleoperation of a robot arm through uplinked commands. Although this study complements the first study on navigational guidance with hand controllers, it is a separate investigation due to the distinction in intended operators (i.e., crewmembers versus ground-operators). A third study looked at superimposed and integrated overlays for teleoperation of a mobile robot using a hand controller. When AR is superimposed on the external world, it appears to be fixed onto the display and internal to the operators' workstation. Unlike superimposed overlays, integrated overlays often appear as three-dimensional objects and move as if part of the external world. Studies conducted in the aviation domain show that integrated overlays can improve situation awareness and reduce the amount of deviation from the optimal path. The purpose of the study was to investigate whether these results apply to HRI tasks, such as navigation with a mobile robot. HRP GAPS This HRI research contributes to closure of HRP gaps by providing information on how display and control characteristics - those related to guidance, feedback, and command modalities - affect operator performance. The overarching goals are to improve interface usability, reduce operator error, and develop candidate guidelines to design effective human-robot interfaces.

Sandor, Aniko↗

Commercial Off-The-Shelf (COTS) Graphics Processing Board (GPB) Radiation Test Evaluation Report

Large round trip communications latency for deep space missions will require more onboard computational capabilities to enable the space vehicle to undertake many tasks that have traditionally been ground-based, mission control responsibilities. As a result, visual display graphics will be required to provide simpler vehicle situational awareness through graphical representations, as well as provide capabilities never before done in a space mission, such as augmented reality for in-flight maintenance or Telepresence activities. These capabilities will require graphics processors and associated support electronic components for high computational graphics processing. In an effort to understand the performance of commercial graphics card electronics operating in the expected radiation environment, a preliminary test was performed on five commercial offthe- shelf (COTS) graphics cards. This paper discusses the preliminary evaluation test results of five COTS graphics processing cards tested to the International Space Station (ISS) low earth orbit radiation environment. Three of the five graphics cards were tested to a total dose of 6000 rads (Si). The test articles, test configuration, preliminary results, and recommendations are discussed.

Salazar, George A.↗

Human-Robot Interaction Directed Research Project

Human-robot interaction (HRI) is a discipline investigating the factors affecting the interactions between humans and robots. It is important to evaluate how the design of interfaces and command modalities affect the human's ability to perform tasks accurately, efficiently, and effectively when working with a robot. By understanding the effects of interface design on human performance, workload, and situation awareness, interfaces can be developed to appropriately support the human in performing tasks with minimal errors and with appropriate interaction time and effort. Thus, the results of research on human-robot interfaces have direct implications for the design of robotic systems. This DRP concentrates on three areas associated with interfaces and command modalities in HRI which are applicable to NASA robot systems: 1) Video Overlays, 2) Camera Views, and 3) Command Modalities. The first study focused on video overlays that investigated how Augmented Reality (AR) symbology can be added to the human-robot interface to improve teleoperation performance. Three types of AR symbology were explored in this study, command guidance (CG), situation guidance (SG), and both (SCG). CG symbology gives operators explicit instructions on what commands to input, whereas SG symbology gives operators implicit cues so that operators can infer the input commands. The combination of CG and SG provided operators with explicit and implicit cues allowing the operator to choose which symbology to utilize. The objective of the study was to understand how AR symbology affects the human operator's ability to align a robot arm to a target using a flight stick and the ability to allocate attention between the symbology and external views of the world. The study evaluated the effects type of symbology (CG and SG) has on operator tasks performance and attention allocation during teleoperation of a robot arm. The second study expanded on the first study by evaluating the effects of the type of navigational guidance (CG and SG) on operator task performance and attention allocation during teleoperation of a robot arm through uplinked commands. Although this study complements the first study on navigational guidance with hand controllers, it is a separate investigation due to the distinction in intended operators (i.e., crewmembers versus ground-operators). A third study looked at superimposed and integrated overlays for teleoperation of a mobile robot using a hand controller. When AR is superimposed on the external world, it appears to be fixed onto the display and internal to the operators' workstation. Unlike superimposed overlays, integrated overlays often appear as three-dimensional objects and move as if part of the external world. Studies conducted in the aviation domain show that integrated overlays can improve situation awareness and reduce the amount of deviation from the optimal path. The purpose of the study was to investigate whether these results apply to HRI tasks, such as navigation with a mobile robot.

Sandor, Aniko↗

Designing for Virtual Windows in a Deep Space Habitat

This paper discusses configurations and test analogs toward the design of a virtual window capability in a Deep Space Habitat. Long-duration space missions will require crews to remain in the confines of a spacecraft for extended periods of time, with possible harmful effects if a crewmember cannot cope with the small habitable volume. Virtual windows expand perceived volume using a minimal amount of image projection equipment and computing resources, and allow a limited immersion in remote environments. Uses for the virtual window include: live or augmented reality views of the external environment; flight deck, piloting, observation, or other participation in remote missions through live transmission of cameras mounted to remote vehicles; pre-recorded background views of nature areas, seasonal occurrences, or cultural events; and pre-recorded events such as birthdays, anniversaries, and other meaningful events prepared by ground support and families of the crewmembers.

habitation↗

Astrobee Free Flyers: Integrated and Tested. Ready for Launch!

As of March 2019, four Astrobee free-flying robots have been integrated, tested and shipped to NASA’s Johnson Space Center, Texas, ready to be launched to space. Two of these Astrobee units (Honey and Bumble) will be launched to the International Space Station (ISS) on April 17, 2019, on the Cygnus NG-11 (Northrop Grumman) cargo resupply spacecraft from the Mid-Atlantic Regional Spaceport (MARS) launch facility, at NASA's Wallops Flight Facility, in Virginia, while the third unit (Queen) will be launched in mid-July 2019, with the cargo resupply mission Space-X 18, from Cape Canaveral, at NASA’s Kennedy Space Center. The primary purpose of the Astrobee system is to provide an autonomous and flexible platform for research on zero-g free-flying robotics, with the ability to accommodate guest researchers looking to utilize the unique capabilities of this platform in the microgravity of low earth orbit, and to advance the state of the art of guest science software and research payloads, which range from gecko-inspired adhesives for perching on smooth surfaces, to augmented reality interfaces to help astronauts and robots work together effectively, to RFID reader, performing inventory of RFID tagged items inside the ISS. Astrobee will also serve utility functions, such as free-flying cameras to record video and provide assistance during astronaut activities, and as mobile sensor platforms to conduct surveys of the ISS. In this paper we will give an overview of the status of the Astrobee system, which includes the docking station already launched and mounted in the JAXA JEM module of the ISS, the Astrobee robots, the Astrobee robotic arms, the ground data system (GDS) and the development of the several guest science payloads.

Carlino, Roberto↗

Medics: Medical Decision Support System for Long-Duration Space Exploration

The Autonomous Medical Operations (AMO) group at NASA Ames is developing a “medical decision support system” to enable astronauts on long-duration exploration missions to operate autonomously. The system will support clinical actions by providing medical interpretation advice and procedural recommendations during emergent care and clinical work performed by crew. The current state of development of the system, called MedICS (Medical Interpretation Classification and Segmentation) includes two separate aspects: a set of machine learning diagnostic models trained to analyze organ images and patient health records, and an interface to ultrasound diagnostic hardware and to medical repositories. Three sets of images of different organs and medical records were utilized for training machine learning models for various analyses, as follows: 1. Pneumothorax condition (collapsed lung). The trained model provides a positive or negative diagnosis of the condition. 2. Carotid artery occlusion. The trained model produces a diagnosis of 5 different occlusion levels (including “normal”). 3. Ocular retinal images. The model extracts optic disc pixels (image segmentation). This is a precursor step for advanced autonomous fundus clinical evaluation algorithms to be implemented in FY20. 4. Medical health records. The model produces a differential diagnosis for any particular individual, based on symptoms and other health and demographic information. A probability is calculated for each of 25 most common conditions. The same model provides the likelihood of survival. All results are provided with a confidence level. Item 1 images were provided by the US Army and were part of a data set for the clinical treatment of injured battlefield soldiers. This condition is relevant to possible space mishaps, due to pressure management issues. Item 2 images were provided by Houston Methodist Hospital, and item 3 health records were acquired from the MIT laboratory of computational physiology. The machine learning technology utilized is deep multilayer networks (Deep Learning), and new models will continue to be produced, as relevant data is made available and specific health needs of astronaut crews are identified. The interfacing aspects of the system include a GUI for running the different models, and retrieving and storing data, as well as support for integration with an augmented reality (AR) system deployed at JSC by Tietronix Software Inc. (HoloLens). The AR system provides guidance for the placement of an ultrasound transducer that captures images to be sent to the MedICS system for diagnosis. The image captured and the associated diagnosis appear in the technician’s AR visual display.

Colombano, Silvano↗

X-HAB 2020: AR Field Treks Summary and Conclusions

As part of the FY20 X-Hab Challenge, BLiSS sought to create an Augmented Reality (AR) toolkit to help with analog field trek operations under the supervision of the Solar System Exploration Research Virtual Institute (SSERVI). These treks are operational and technical demonstrations at space-like destinations on Earth to test current extra-vehicular activity (EVA) techniques. While BLiSS as an organization has experience studying operational tasks such as this, it has never developed AR software at this scale. For that reason, another team at the University was brought on to work in parallel. The Collaborative Lab for Advancing Work in Space (CLAWS) is a veteran group of the NASA Spacesuit User Interface Technologies for Students (SUITS) challenge in which Hololens displays for astronauts are created within a year. The operational and technological pairing was ideally suited for tackling this problem. The team divided its responsibilities so that BLiSS would handle the research required to shape the project. As this deliverable had an end user, it was decided that interviewing these field geologists and operations specialists would provide the best insight. These interviews paired with literature review would reveal niche applications for AR that remained within feasible bounds. These science-driven EVAs in unknown terrain require more flexible tools than the current generation of EVA assistants. Rather than focus on sequential instructions, there instead needs to be a broad toolkit that's only called upon in specific instances. This AR Toolkit for Lunar Astronauts and Scientists (ATLAS) became the development goal of the project: create a non-intrusive assembly of tools that could be accessed in AR on the field. The current ATLAS design makes use of a geospatially and temporally annotated eld note system called GeoNotes. This allows for data to be collected and coordinated in a way that's synchronized across time, space, and different users. A Mission Control Center (MCC) and Mobile Support Equipment (MSE) were all needed to transport the AR headset into the field with the user. A network infrastructure was designed and set up within the University to enable this functionality. The software is based on a Protocol-Module structure that allows for modular development of each capability. A Protocol Manager coordinates different protocols that make use of modules. Each module tackles a different individual task while the protocol puts each one to use. The protocol manager coordinates when these are called to be used. This software is hosted on a head-mounted display (HMD) with the MCC acting as support from afar. While the software would be unit-tested at each level and each hardware component verified, a final demonstration would serve to prove the system's capabilities: an analog field trek. The team would prepare to support a user in a remote location from the MCC back at the University. A local area near campus would be tested before going out to do sample field geology further away. This unfortunately became impossible with the arrival of COVID-19. Access to all of the facilities to complete the project as planned were shut down. Our team was scattered across the globe and forced to complete the rest virtually. Adjustments were made to produce a small virtual concept in Adobe XD in the meantime. Even digital surveys were created based on the NASA task-load index (TLX) originally intended for testing actual users. The goal shifted towards completing software and getting feedback on the user interfaces (UI) and user experiences (UX). This team has reformed in response to COVID and its focus has shifted to what can be done remotely. There is still an intention to finish the original deliverable described in this report. The work has been expanded beyond the original X-Hab challenge and has instead become its own research e ort to be continued afterwards. This report collects the processes and knowledge gained from a year of studying and working at this problem with two teams. It should preserve it for the time until the world returns to normal and work can resume. CLAWS will be taking over full responsibility from that point forward, eventually surpassing the original needs of the project. While this document captures the work done towards an eventual end, the CLAWS team has written their own proposal alongside it. It outlines a new future for ATLAS beyond X-Hab, BLiSS, and hopefully beyond COVID-19. This project began as a vague goal hoping to place a new technology into the unique setting of exploration science. The project has since comfortably taken root and will hopefully bloom over the next year.

Alex Sena↗