A Flexible NASA Open Source Platform for Mixed Reality: Mixed Reality Exploration Toolkit (MRET) Architecture & Design
Explore the source record for details and available documents.
Thomas G Grubb↗
SEARCH · Search NASA
Results for “reality mixed reality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Mixed Reality Training and Workspace Integration
Mixed reality technology is being used across Idaho National Laboratory (INL) to enhance training and workforce development. Projects involving mixed reality such as lockout/tagout , the Industrial Control System Laboratory , and a Department of Defense microreactor demonstration have benefited from these innovative approaches. Several key concerns exist with integrating this new technology, including ease of use, ease of setup, and maintenance through the lifecycle. These aspects need to be properly addressed to ensure that this technology remains attractive and useful to operations teams as well as engineers and researchers at INL. Primarily, this requires buy-in from information technology management, designing software/hardware-agnostic tools, and reducing friction with technology integration. To be successful, regular operational use of this new and unfamiliar technology will require straightforward user interfaces, simple setup processes, and feature-rich experiences that reduce the time needed to benefit from the technology. INL is improving its toolset to allow for a streamlined deployment and use of mixed reality technologies that are directly available to researchers and engineers.
99 GENERAL AND MISCELLANEOUS↗
Immersive environment technologies for planetary exploration with applications for mixed reality
Immersive environments are successfully being used to support mission operations at JPL. This technology contributed to the Mars Pathfinder Mission in planning sorties for the Sojourner rover. Results and operational experiences with these tools are being incorporated into the development of the second generation of mission planning tools.
visualization rover system ground system Mars imme↗
Lunar Surface Mixed Reality and ARGOS Trainer
This Artemis focused Mixed Reality (MR) and ARGOS project extends the current VR/ARGOS CIF project to enhance and add VR/MR capabilities to support analysis, training and risk reduction for lunar surface EVA operations in the extreme South Pole lunar environment. This third year focused on establishing a functional trainer for crew that supports the ingress/egress of a lander as well as surface tasks such as sample collection and tool deployment and operation. Completed enhancements include improving hand tracking, the addition of supporting mixed reality surface operations, and the implementation of a higher fidelity body tracking capability through a motion capture system.
Lunar↗
Implementing Mixed Reality Tools to Support Mission Delivery at Hanford - 20520
Mission Support Alliance's (MSA) Public Works and Information Systems organizations have been working with Microsoft Corporation to develop and implement mixed reality operational solutions using their HoloLens technology. This collaborative effort is intended to drive innovative solutions and significantly improve MSA's efficiency in performing work at the Hanford Site. HoloLens, a mixed reality tool, is a commercial off-the-shelf technology that combines a head-mounted viewing screen with applications to help people and organizations learn, communicate, and collaborate more effectively through the use of mixed reality. https://microsoft.com/en-us/hololens MSA's initial pilot successfully demonstrated its ability to use mixed reality utilizing HoloLens' advanced features in a variety of ways, illustrating the potential efficiency gains previously stated as goals of this pilot. These achievements include the development of proprietary technology that allows synchronization between a HoloLens device and a mobile device or alternate global positioning system (GPS) device. This synchronization enables the HoloLens device to access real time GPS location data anywhere on the Hanford Site. This real time location data, combined with MSA's improved Hanford Geographical Information System (GIS) data, enables the mixed reality pilot application to identify underground utility systems as well as related utility attributes. This ability will aid utility workers in several areas, including excavation activities, the placement of large cranes, identifying interactions and conflicts with future underground utility placements and, eventually, recognizing and de-conflicting tank waste transfer system valve alignments. The HoloLens mixed reality application is also able to provide full or small scale holographic images of a facility's digital twin. This feature will allow MSA's Water and Sewer Utilities team to train and validate procedures associated with the new water treatment facility virtually, before it is constructed. This reduces the amount of time it takes to complete the training and procedure validation, and accelerates the overall construction schedule. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Effects of Motion Fidelity on Pilot Performance Using Mixed-Reality Visuals
This paper investigates the effects of simulator motion fidelity on pilot performance using mixed-reality visuals. In NASA’s Vertical Motion Simulator, eleven pilots flew a lift-plus-cruise electric vertical takeoff and landing vehicle in a vertiport approach and landing task under four different motion conditions: no motion, small hexapod motion, large hexapod motion, and full motion. The simulator motion condition statistically significantly affected: 1) the severity of cybersickness, 2) motion ratings, 3) final approach and take-off boundary crossing groundspeeds, 4) longitudinal touchdown point, and 5) battery time remaining after landing. These results suggest that the use of motion systems in simulators with virtual- or mixed-reality visuals is warranted.
simulation↗
Effects of Motion Fidelity on Pilot Performance Using Mixed-Reality Visuals
This paper investigates the effects of simulator motion fidelity on pilot performance using mixed-reality visuals. In NASA’s Vertical Motion Simulator, eleven pilots flew a lift-plus-cruise electric vertical takeoff and landing vehicle in a vertiport approach and landing task with four different motion conditions: no motion, small hexapod motion, large hexapod motion, and full motion. The simulator motion condition statistically significantly affected: 1) the severity of cybersickness, 2) motion ratings, 3) final approach and take-off boundary crossing groundspeed, 4) longitudinal touchdown point, and 5) battery time remaining after landing. The findings suggest that motion systems are essential in simulators with virtual- or mixed-reality visuals when it is required to have pilot flying performance similar to that in real flight.
mixed reality↗
Radsource Mr: Mixed Reality Planning Tool For Radioactive Recovery
The RadSource MR system leverages Meta Quest 3's advanced mixed reality capabilities to create a comprehensive spatial planning platform for end-of-life sealed radioactive source recovery operations. The application utilizes the Quest 3's high-resolution passthrough cameras and spatial mapping algorithms to generate accurate 3D environmental models. Core technical components include: (1) Real-time spatial measurement algorithms calculating distances, angles, slopes, and surface areas with sub-centimeter accuracy; (2) Virtual object placement system allowing users to position digital representations of recovery equipment (trailers, containment vessels, protective barriers) within the real environment; (3) Voice recording and annotation system for hands-free documentation in protective equipment; (4) 3D mesh capture and storage capabilities for post-operation analysis and regulatory documentation. (5) Procedure documentation is available for viewing in Mixed Reality, providing an innovative and convenient way to access the information during pre-visit and inspection activities. (6) Support for screen capture for the view for real world and virtual objects together to use it later for planning. The system integrates computer vision techniques for environmental understanding, spatial mathematics for precise measurements, and human-computer interaction principles optimized for hazardous environment operations. Data persistence allows teams to save and share planning sessions across multiple stakeholders while maintaining operational security requirements.
Khadka, Rajiv [Idaho National Laboratory (INL), Id↗
The Reality of Space: Exploring Virtual, Augmented, and Mixed Realities and the Future of Spaceflight
Explore the source record for details and available documents.
Yevgeniy Zhivotovskiy↗
Real-time mixed reality display of dual particle radiation detector data
Radiation source localization and characterization are challenging tasks that currently require complex analyses for interpretation. Mixed reality (MR) technologies are at the verge of wide scale adoption and can assist in the visualization of complex data. Herein, we demonstrate real-time visualization of gamma ray and neutron radiation detector data in MR using the Microsoft HoloLens 2 smart glasses, significantly reducing user interpretation burden. Radiation imaging systems typically use double-scatter events of gamma rays or fast neutrons to reconstruct the incidence directional information, thus enabling source localization. The calculated images and estimated ’hot spots’ are then often displayed in 2D angular space projections on screens. By combining a state-of-the-art dual particle imaging system with HoloLens 2, we propose to display the data directly to the user via the head-mounted MR smart glasses, presenting the directional information as an overlay to the user’s 3D visual experience. We describe an open source implementation using efficient data transfer, image calculation, and 3D engine. We thereby demonstrate for the first time a real-time user experience to display fast neutron or gamma ray images from various radioactive sources set around the detector. We also introduce an alternative source search mode for situations of low event rates using a neural network and simulation based training data to provide a fast estimation of the source’s angular direction. Using MR for radiation detection provides a more intuitive perception of radioactivity and can be applied in routine radiation monitoring, education & training, emergency scenarios, or inspections.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
The "Oracle Problem" and Its Impacts in Mixed Reality
Explore the source record for details and available documents.
Machine Learning↗
Leveraging Industrial Mixed Reality Successes for Safeguards
Explore the source record for details and available documents.
Westphal, Greg↗
The Next Wave: Humans, Computers, and Redefining Reality
The Augmented/Virtual Reality (AVR) Lab at KSC is dedicated to " exploration into the growing computer fields of Extended Reality and the Natural User Interface (it is) a proving ground for new technologies that can be integrated into future NASA projects and programs." The topics of Human Computer Interface, Human Computer Interaction, Augmented Reality, Virtual Reality, and Mixed Reality are defined; examples of work being done in these fields in the AVR Lab are given. Current new and future work in Computer Vision, Speech Recognition, and Artificial Intelligence are also outlined.
Extended Reality↗
An Evaluation of Extended Reality Technologies for Use in Verification Testing at NASA 2024 HRP IWS Abstract
BACKGROUND At NASA, verification testing is the formal process of ensuring that a product conforms to requirements set by a project or program. Some verification methods, such as Demonstrations and Test, require either the end product or a mockup of the product with sufficient fidelity to stand-in for the product during the test. Traditionally, these mockups have been physical (e.g., foam-core and wood) but there is growing interest in exploring new methods for testing with these mockups. These methods include virtual reality (VR), mixed reality, and augmented reality which are collectively referred to as eXtended Reality (XR) technologies. VR has already been adopted and used by many in the aerospace industry as a tool for use in early design phases (e.g., developmental testing) and may have the most potential for use in verification tests. Benefits of using VR mockups offer cost effectiveness, ease of iteration, simulation of hazardous conditions (e.g., an egress through a hatch with smoke obscuring vision), and the ability to simulate microgravity conditions, which are challenging to do with physical mockups. However, the validity of test results obtained from VR mockup demonstrations or testing, compared to the current gold standard of physical mockups, remains uncertain. It is unlikely that there is one clean answer as there are many different types of verification outcomes and each XR technology must be evaluated on its own merits. This is not an issue during developmental testing as the design is still in flux and the total success of the design is not dependent upon the results of a developmental test. Verification tests, however, only happen once, assuming no change to the design, and the results are used to certify the product. Therefore, establishing the validity of XR mockup-based verification outcomes is essential before considering them for any use in verification tests. OBJECTIVE AND METHOD To address this concern, the Human Research Program has funded a project to explore and qualify how XR technologies might be used in verification demonstration and testing at NASA. Currently, we are conducting a review of the literature on the utilization of XR mockups for design activities, prototyping, and user testing. We are employing the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis method to identify the pros, cons, and barriers to adoption of XR technologies for verification testing at NASA. Additionally, we are developing a framework to guide the deployment of XR mockups for verification tests. Building upon available evidence from the literature and subject-matter expert feedback, our goal for the framework is to provide guidelines for which forms of XR mockups are suitable for a given verification test, when only physical mockups should be employed and to highlight areas for which more evidence is needed. To further refine our framework and to contribute to the body of evidence, we are planning a lab-based experiment comparing a VR mockup to a physical twin for a set of select verification outcomes. ANTICIPATED RESULTS In this presentation, we will present the work we conducted to evaluate XR technologies for use in verification tests at NASA. We will summarize and report our findings from the SWOT analysis and our lab-based study, and we will present the current state of the XR Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project. Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project.
Extended Reality↗
Immersive Analytics in Critical Spatial Domains: From Materials to Energy Systems
Immersive analytics (IA) leverages virtual reality, augmented reality, and mixed reality to transform how users interact with complex datasets across domains such as science, industry, and education. These immersive technologies offer spatial and multimodal environments that foster intuitive exploration, but they also introduce challenges related to cognitive load, interface design, and system performance. Here, this article presents a comprehensive review of visualization techniques, interaction models, and multimodal inputs utilized in IA. Drawing on case studies in scientific visualization, industrial training, and educational communication, we examine both the potential and limitations of current systems. Finally, we propose future research directions, focusing on real‐time collaboration, adaptive user interfaces, and scalable data exploration strategies to advance the field.
99 - GENERAL AND MISCELLANEOUS↗
Crew Medical Training to Progressively Enable EIMO
Background. Onboard medical capabilities have greatly expanded over the history of the US space program. Newly identified space-related medical conditions, technological advances, and longer mission durations have led to an increasing need for on-demand medical expertise. Lengthy communications delays, lack of resupply and evacuation opportunities on exploration-class missions place an ever-increasing burden on the crew to provide medical care. Having adequate knowledge, skills, and abilities (KSA) available is an essential component of successful Earth Independent Medical Operations (EIMO). Without appropriate crew training and KSA, cutting-edge medical equipment has little value. Presumably, the crew will include a qualified physician; however, if the physician is incapacitated, a non-physician crew medical officer (CMO) will be needed. While more crew time is needed for medical training, there will be concomitant increases in preflight training demands for vehicle system management, operations, science, and contingencies. In truly independent operations, onboard resources such as just-in-time training, mixed reality, decision support tools, and AI-enabled chatbot “consultants” will be needed to augment KSA. Overview. Because of crew time constraints, topical priorities must be determined for preflight training. Curricula should be developed that emphasize management of conditions with relatively high incidence and morbidity/mortality. Defining the required KSA levels to treat each condition is essential, but all crewmembers should have basic lifesaving skills. Procedural and diagnostic training on live patients and simulators should be prioritized over classroom lectures. Crews must be trained with onboard equipment, resources, mixed reality, and AI-based decision support tools. Mission simulations should include medical problems with/without ground support and with appropriate communication delays. Certification guidelines for each level of KSA must be established. Skills rapidly decay for non-physician CMO’s; both pre-flight and in-flight refresher training will be needed. During spaceflight just-in-time training, simulations, and onboard CME with crew physician can help retain skills. Discussion. Medical technology, simulation design, mixed reality, and AI are advancing at a dizzying rate. Recognizing the severe constraints on crew time, it is critical that astronaut training is highly efficient and adapted to keep pace with new innovations both pre-flight and during exploration missions. These challenges will be discussed during this panel session.
Jay Lemery↗
Internship in Augmented and Virtual Reality - Rapid Model Import Tool
The integration of virtual and augmented reality, sometimes called mixed reality, is an emerging technology which will likely skyrocket overnight much in the way smartphones did a decade ago. Kennedy Space Center's Augmented and Virtual Reality (AVR) Lab is developing a Rapid Model Import Tool (RMIT) to create a quick and efficient way to bring NASA's complex engineering 3D models into virtual and augmented environments. The long-term objective is to create a tool that will ultimately benefit KSC engineers. Its various uses within NASA can potentially span from astronaut training, to marketing, to public outreach, to name a few. Unity is a prolific cross-platform game engine that allows users to build high quality 2D and 3D games for desktop, mobile, web, and game console platforms. It is perhaps also the most widely used software for virtual reality game development. At the AVR lab, we are looking at alternative uses of Unity to build tools for NASA engineers to perform design, development, testing, and training on spacecraft, rocket delivery systems, ground support equipment, and facilities at KSC. As an intern for the RMIT project, I am charged with the task of performing research on Unity-compatible file types to develop an efficient, affordable, preservative process to bring models from CATIA 3D engineering software into the Unity environment. With a tool called the NASA Enterprise Visualization Application (NEVA), developed by the Boeing Design Visualization group at KSC, we are able to easily convert CATIA's design models to. DAE (also known as COLLADA) and .OBJ file formats. I first reduce the polygon count of the model within CATIA itself, make any necessary tweaks to reduce the model further, and then export using NEVA. The .OBJ or. DAE files that I am left with are then converted by another intern to a Unity-compatible file format using a custom Python script. I have generated extensive documentation of this process in a NEVA User Guide. By the end of this semester, we will have built a solid framework for RMIT based on a thorough understanding of virtual reality specifications and file requirements, allowing future software development teams to go forward with development on the custom tool.
Leap Motion↗