Extended Reality Visualization at INL
Presentation detailing the use of extended reality at INL.
97 MATHEMATICS AND COMPUTING↗
SEARCH · Search NASA
Results for “Augmented reality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cross-domain digital twin architecture for predictive maintenance via machine learning and Large Language Models
This research introduces a comprehensive framework for creating and deploying a digital twin platform for continuous monitoring and predictive maintenance within industrial settings. Through utilizing advanced technologies, including Unreal Engine 5, Unity 3D, the Message Queue Telemetry Transport protocol, Random Forest machine learning algorithms, and Large Language Models (LLMs), we establish a platform that digitally reproduces physical equipment and translates digital controls into real-world actions. This facilitates preventive maintenance approaches and improves operational effectiveness. The digital twin platform gathers sensor data from operational equipment, analyzes it using machine learning, and delivers practical insights to prevent potential malfunctions and enhance equipment performance. Furthermore, the incorporation of a web portal enables efficient monitoring and access to historical data, educational materials, and equipment status information. Preliminary findings indicate that digital twins can transform industrial equipment management and maintenance methodologies.
97 MATHEMATICS AND COMPUTING↗
Data visualization for effective rover sequencing
The Rover Sequencing and Visualization Program is a suite of toos for the commanding of planetary rovers which are subject to significant light time delay and thus are unsuitable for tele-operation.
surface operations↗
The Evolution of Three Dimensional Visualization for Commanding the Mars Rovers
NASA's Jet Propulsion Laboratory has built and operated four rovers on the surface of Mars. Two and three dimensional visualization has been extensively employed to command both the mobility and robotic arm operations of these rovers. Stereo visualization has been an important component in this set of visualization techniques. This paper discusses the progression of the implementation and use of visualization techniques for in-situ operations of these robotic missions. Illustrative examples will be drawn from the results of using these techniques over more than ten years of surface operations on Mars.
stereo visualization↗
Deep Learning Applications in Manned Spaceflight
This presentation discusses a basic overview of Deep Machine Learning (DML) fundamentals, and two JSC applications of DML to create an Intelligent Personal Coach for exercise applications on deep-space missions, and the training of a neural network using the SingleShotPose algorithm from Microsoft to detect object 6 degree of freedom pose information from 2D image data for use in an Intelligent Procedure Assistant. The presentation concludes with a discussion about conceptual future uses of DML for space missions.
augmented reality↗
Deep Learning for Space Applications
Explore the source record for details and available documents.
Augmented Reality↗
Discovering Digital Engineering Needs Through Human-Centered Approaches
Digital engineering has the power to transform organizations in profound ways, but digitizing legacy workflows and infusing new technologies into existing practices may not yield the intended results or address the underlying problems or bottlenecks. This paper describes the human-centered approaches used to understand stakeholder challenges and identify digital engineering needs during the design-build-test portion of the systems development lifecycle. Through a series of interactive stakeholder workshops and need-finding activities, key stakeholder needs were identified to help guide the integration of digital engineering technologies. The overarching need was for downstream system development steps to be considered further upstream in the project lifecycle in a manner that allows for proactive and interactive co-constructive idea exploration, problem resolution, and requirements considerations in a collective engagement with engineers and technicians from multiple disciplines to increase efficiency and reduce risks while offering opportunities to improve the final engineered system. Supporting this need was a wide-ranging desire from stakeholders for digital and non-digital support structures for efficient rapid iteration during system development. This work yielded several improvements for system development processes and for digital engineering infusion.
Human-centered Design↗
FY23 ION Based Approaches to Address Labor and Knowledge Retention
This study will seek to outline the current problem facing the nuclear industry related to staffing, training, and retaining the necessary workforce for long-term sustainability. A review of published reports and knowledge gained by working in the nuclear energy sector will form the basis for this scoping study. This study will not seek to draw any conclusions regarding the underlying causes of these personnel issues but will attempt to ask questions and help outline future research. The Integrated Operations for Nuclear business model provides opportunities to partially mitigate some of the current issues facing staffing the nuclear fleet. Upgrading safety and control systems from analog to digital as advocated by the Integrated Operations for Nuclear process allows the nuclear facility to compete for the best and brightest candidates and offers them a broad and sustainable career path where their skills can be valued and utilized in the broader industrial sectors. This study will seek to identify the primary reasons that the nuclear industry is facing this labor crisis and seek to identify possible solutions that will be fully evaluated in future research.
42 ENGINEERING↗
REMOTELY: A HOME ENERGY SCORE APP
Explore the source record for details and available documents.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Vertical full-colour micro-LEDs via 2D materials-based layer transfer
Micro-LEDs (µLEDs) have been explored for augmented and virtual reality display applications that require extremely high pixels per inch and luminance. However, conventional manufacturing processes based on the lateral assembly of red, green and blue (RGB) µLEDs have limitations in enhancing pixel density. Recent demonstrations of vertical µLED displays have attempted to address this issue by stacking freestanding RGB LED membranes and fabricating top-down, but minimization of the lateral dimensions of stacked µLEDs has been difficult. Here we report full-colour, vertically stacked µLEDs that achieve, to our knowledge, the highest array density (5,100 pixels per inch) and the smallest size (4 µm) reported to date. This is enabled by a two-dimensional materials-based layer transfer technique that allows the growth of RGB LEDs of near-submicron thickness on two-dimensional material-coated substrates via remote or van der Waals epitaxy, mechanical release and stacking of LEDs, followed by top-down fabrication. The smallest-ever stack height of around 9 µm is the key enabler for record high µLED array density. We also demonstrate vertical integration of blue µLEDs with silicon membrane transistors for active matrix operation. Furthermore, these results establish routes to creating full-colour µLED displays for augmented and virtual reality, while also offering a generalizable platform for broader classes of three-dimensional integrated devices.
42 ENGINEERING↗
Roadmap for Optical Metasurfaces
Metasurfaces have recently risen to prominence in optical research, providing unique functionalities that can be used for imaging, beam forming, holography, polarimetry, and many more, while keeping device dimensions small. Despite the fact that a vast range of basic metasurface designs has already been thoroughly studied in the literature, the number of metasurface-related papers is still growing at a rapid pace, as metasurface research is now spreading to adjacent fields, including computational imaging, augmented and virtual reality, automotive, display, biosensing, nonlinear, quantum and topological optics, optical computing, and more. At the same time, the ability of metasurfaces to perform optical functions in much more compact optical systems has triggered strong and constantly growing interest from various industries that greatly benefit from the availability of miniaturized, highly functional, and efficient optical components that can be integrated in optoelectronic systems at low cost. This creates a truly unique opportunity for the field of metasurfaces to make both a scientific and an industrial impact. Furthermore, the goal of this Roadmap is to mark this “golden age” of metasurface research and define future directions to encourage scientists and engineers to drive research and development in the field of metasurfaces toward both scientific excellence and broad industrial adoption.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Flexible and Accessible 4D Subsurface Visualization Using a Web-Based Platform
4D subsurface visualization using a web platform can provide improved communication, education and outreach to non-experts and stakeholders. It allows for the improved understanding of complex relationships and interactions that occur in inaccessible locations of which many experimental testbeds exist in. The use of a web-based visualization tool lessens the cognitive impact further by reducing the need of specialized software. Users can define their own exploration of a 3D or 4D scene, adding and removing data as needed and are able to make informed decisions based on accurate model visualizations, while providing a collaborative tool that can be accessed on any platform using a browser window. The tool is also able to point to real time streams to display up-to-the-second data as an experiment is ongoing. This level of latency can also provide operators with essential information that can direct an experiment’s progress. Additionally, this tool can leverage augmented and virtual reality (AR, VR) capabilities of certain mobile devices and head-mounted displays, providing further engaging visualization possibilities.
Pratt, Martin J.↗
Modeling and Simulation - Virtual
The purpose of this report is to describe the body of work I have produced as a NASA Office of Science Technology Engineering and Math (OSTEM) intern in the Fall 2020 semester. My mentor during this session was Antonio Pego and my main task was to research and develop solutions in augmented and virtual reality technologies for use in training simulations. A main focus of this project was the development of human models for use in the game engine, with the purpose of creating simulations that would help with realistic analysis of tasks that will be performed by personnel. A steep learning curve had to be tackled in order to learn and catch up with current MR technologies. Along the way I created documentation of the different techniques utilized to support future continuation of the project.
Peter Leroy Santana Rodriguez↗
NASA's Research & Development in Airspace Management for Drones, Air Taxis, and Beyond
Today's National Airspace System is extremely safe. Multiple people and organizations are involved in the handling of flights at all phases. We are, however, at the dawn of a new era in aviation where new entrants are emerging from multiple domains. These new entrants will need to be integrated into the airspace safely and given the flexibility to scale. With that integration and flexibility comes the need and responsibility to develop concepts and systems that will work with the regulatory and operational needs of the Federal Aviation Administration while accounting for the needs of industry. NASA is at the leading edge of research, development, and testing across multiple domains to enable the realization of new entrants such as drones, air cargo and taxis, and high altitude vehicles. Addressing the potential for public good coming from these new capabilities is also a NASA focus area. While there has been considerable effort in the engineering aspects of this research, another thrust that is of importance is that of data integration and visualization. With respect to visualization, NASA Ames Research Center has been leading the advancement in data visualization for these new domains and has spent considerable efforts in the exploration of incorporating Augmented and Mixed Reality as a situation awareness platform. This presentation will outline the new developments in airspace integration and tie in NASA's efforts in visualizing the associated operational data.
airspace management↗
Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations
NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop testing plays a crucial role in NASA's mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts' situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA's testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop testing for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive training experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.
Simulation Modeling Virtual Reality Immersive Tech↗
Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations
NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for immersing users in realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop simulations play a crucial role in NASA’s mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments, data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures, and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts’ situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA’s testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop simulations for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive simulation experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.
HITL↗
Virtual reality and planetary exploration
NASA-Ames is intensively developing virtual-reality (VR) capabilities that can extend and augment computer-generated and remote spatial environments. VR is envisioned not only as a basis for improving human/machine interactions involved in planetary exploration, but also as a medium for the more widespread sharing of the experience of exploration, thereby broadening the support-base for the lunar and planetary-exploration endeavors. Imagery representative of Mars are being gathered for VR presentation at such terrestrial sites as Antarctica and Death Valley.
Mcgreevy, Michael W.↗
Virtual reality and planetary exploration
Exploring planetary environments is central to NASA's missions and goals. A new computing technology called Virtual Reality has much to offer in support of planetary exploration. This technology augments and extends human presence within computer-generated and remote spatial environments. Historically, NASA has been a leader in many of the fundamental concepts and technologies that comprise Virtual Reality. Indeed, Ames Research Center has a central role in the development of this rapidly emerging approach to using computers. This ground breaking work has inspired researchers in academia, industry, and the military. Further, NASA's leadership in this technology has spun off new businesses, has caught the attention of the international business community, and has generated several years of positive international media coverage. In the future, Virtual Reality technology will enable greatly improved human-machine interactions for more productive planetary surface exploration. Perhaps more importantly, Virtual Reality technology will democratize the experience of planetary exploration and thereby broaden understanding of, and support for, this historic enterprise.
Mcgreevy, Michael W.↗