Results for “Augmented reality” — Page 2

DOE OSTI2023

Systems, devices, and methods for performing augmented reality responsive to monitoring user behavior

Systems, devices, and methods are described for performing augmented reality (AR) to assist user performing a task in an environment. An AR device may be configured to capture real-time data. An AR engine may be configured to monitor user behavior from the real-time data responsive to feature extraction from the real-time data, compare the user behavior to pre-defined work procedures, and generate augmented reality objects to be output by the AR device.

Yoon, SuJong↗

DOE OSTI2024

System and method for transparent augmented reality

A system and method for displaying augmented reality information for a real object. A transparent display displays the information for the real object within a field of view, such that the information and the real object are simultaneously viewable by a user. Data-gathering peripheral devices gather data of the field of view, including a camera gathering visual data for tracking the real object. An articulating arm moveably supports and facilitates repositioning the display and redirecting the data-gathering peripheral devices in a particular direction. A security interlock disables the data-gathering peripherals when the articulating arm is moved beyond a pre-established threshold, thereby preventing the data-gathering peripherals from gathering data of a real environment outside of a limit of the field of view as determined by the pre-established threshold. The security interlock may include an adjustable contact switch which adjustably defines the pre-established threshold in at least one direction.

Runyan, Paul Ryan↗

DOE OSTI2024

Data Visualization: Augmented Reality

In recent years, there has been an increasing interest in developing new technologies for automated characterization and visualization of condition monitoring data. Augmented Reality (AR) is a technology that is being developed to improve such data visualization. Augmented reality has been defined as a technology that merges virtual and physical components in real-time, and in three dimensions. Wearable, commercially-available AR devices allow onsite engineers and technicians to perform inspection tasks with significantly more available information such as comparisons of past and present sensor and imager data, onsite data analysis and result displays, and various forms of metadata including technical drawings, previous inspection reports and maintenance histories, operation manuals, codes and standards, and holograms representing data analysis results superimposed onto the in situ monitored system.

42 ENGINEERING↗

NASA NTRS2002

Augmented Reality in a Simulated Tower Environment: Effect of Field of View on Aircraft Detection

An optical see-through, augmented reality display was used to study subjects' ability to detect aircraft maneuvering and landing at the Dallas Ft. Worth International airport in an ATC Tower simulation. Subjects monitored the traffic patterns as if from the airport's western control tower. Three binocular fields of view (14 deg, 28 deg and 47 deg) were studied in an independent groups' design to measure the degradation in detection performance associated with the visual field restrictions. In a second experiment the 14 deg and 28 deg fields were presented either with 46% binocular overlap or 100% overlap for separate groups. The near asymptotic results of the first experiment suggest that binocular fields of view much greater than 47% are unlikely to dramatically improve performance; and those of the second experiment suggest that partial binocular overlap is feasible for augmented reality displays such as may be used for ATC tower applications.

Ellis, Stephen R.↗

DOE OSTI2023

Augmented reality to guide pedestrian evacuation in earthquake events

Augmented Reality is a promising technology to support safe pedestrian evacuation from buildings during earthquakes. The widespread availability of phone cameras makes this idea feasible. QR codes, for instance, can place virtual three-dimensional objects to guide people along a prescribed route. In this work, a prototypical system based on this premise was created in response to the 2020 sudden surge in seismic activity in Puerto Rico. The results point towards feasibility as well as to a series of areas that require improvement for a real-life implementation.

Rivera-Hernández, Diego F.↗

NASA NTRS2023

Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics (Joint AR) Product

Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate and tight choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. Additionally, EVA operations, aside from the Apollo Lunar surface missions, have predominately focused on maintenance and construction tasks where success criteria are clearly measurable. However, future exploration missions expect to enable crew to carry out scientific objectives in increasingly Earth-independent ways. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. This paper highlights the project’s experience with a product-focused management style and use-case centered systems engineering approach to iteratively design, build, and test. The Joint AR product features were defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, graphics engine software and analog test beds. We outline the defining architectural design decisions, including safety criticality considerations, suit mounting interfaces, computer architectures, and partnership contracting mechanisms. The outcomes of these studies, architecture decisions, and management requirements result in a recommended design which is the Joint AR product. We discuss the evolution, development of these system components, and what work remains. We hope to share a unified understanding of various design decisions and how they impact the future of crew members’ access to data during Lunar and Martian EVAs. This ongoing effort can enable a community-wide discovery process toward realizing necessary AR features and capabilities for future missions.

Augmented Reality↗

NASA NTRS2023

Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics System (Joint AR) Product

Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) project team at NASA Johnson Space Center (JSC) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. The Joint AR product was defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, and graphics engine software. This paper outlines the defining architectural design decisions, including safety criticality considerations, interfaces, and computer architectures. The outcomes of these studies result in a prototype design which is defined here as the Joint AR product. This work aims to enable a community-wide discussion toward realizing necessary suit-compatible AR features and capabilities for future missions.

Paromita Mitra↗

NASA NTRS2023

Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics System (Joint AR) Product

Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) project team at NASA Johnson Space Center (JSC) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. The Joint AR product was defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, and graphics engine software. This paper outlines the defining architectural design decisions, including safety criticality considerations, interfaces, and computer architectures. The outcomes of these studies result in a prototype design which is defined here as the Joint AR product. This work aims to enable a community-wide discussion toward realizing necessary suit-compatible AR features and capabilities for future missions.

Paromita Mitra↗

DOE OSTI2026

Augmented Reality Technologies for Radiation Safety Training: A Systematic Review of Sensor Integration and Visualization Approaches

This paper presents a comprehensive systematic review examining the application of augmented reality (AR) and sensor technologies for visualizing ionizing radiation in virtual training environments. The review methodology involved systematic identification and analysis of the relevant literature based on predetermined criteria including publication type, year of publication, application domain, and technological approach. The literature search encompassed publications from 2011 to 2021 across four major academic databases: Web of Science, Google Scholar, IEEE Xplore, and Scopus. Through rigorous screening following PRISMA 2020 guidelines, 23 research articles met the inclusion criteria for detailed analysis. From 404 initial database records, 360 were excluded during title/abstract screening (primarily for lacking AR components, radiation focus, or training applications) and 4 during full-text assessment (all for lacking sensor integration). The findings reveal that AR-based ionizing radiation visualization has been successfully implemented across diverse domains, including nuclear facility operations, medical procedures, CERN research activities, and educational and monitoring applications. The analysis identified multiple dimensions of impact, encompassing distinct benefits, emerging opportunities, and implementation challenges associated with AR deployment for ionizing radiation training. Each of these dimensions is comprehensively examined and documented within this review. Additionally, this study identifies critical research gaps that currently limit the full potential of AR technology in supporting ionizing radiation training programs. These gaps are systematically analyzed and discussed to establish clear directions for future research endeavors in this emerging field.

61 - RADIATION PROTECTION AND DOSIMETRY↗

NASA NTRS2024

Enhancing Metal Additive Manufacturing Training with the Advanced Vision Language Model: A Pathway to Immersive Augmented Reality Training for Non-Experts

This paper introduces an innovative training system for the Renishaw AM400 metal printer, leveraging the synergy of the advanced Vision Language Model (VLM) with Augmented Reality (AR) within the Digital Twins (DT) framework. Aimed at overcoming the limitations of conventional training methods in metal additive manufacturing (AM), our system integrates AR to provide an immersive learning environment, enhancing the real-world experience with interactive digital overlays. The core of the system lies in its use of VLM, which, pre-trained on diverse datasets, excels in processing multi-modal data, thereby offering nuanced and contextually relevant guidance for trainees. Key experiments demonstrate the system’s effectiveness, particularly highlighting the usage of VLM as an Artificial Intelligence (AI) agent to integrate external tools like YOLO-v7 for valve state classification and CRAFT for control panel text recognition. This approach significantly improves recognition accuracy, operational understanding, and human–machine interaction, especially for non-expert users, making complex metal AM operations more accessible. The research not only showcases the potential of AR and VLM in industrial training but also sets a new standard for smart manufacturing practices, indicating broader applications in various industrial domains.

Metal additive manufacturing↗

NASA NTRS2008

Handheld Augmented Reality Project [HARP]

The project goal was to design and study engaging and effective augmented reality learning environments using wireless handheld computers equipped with global positioning system (GPS) receivers.

Dunleavy, Matt↗

NASA NTRS2023

Augmented Reality Tool for Operationally Relevant 3D Sensorimotor Assessments

This project will leverage existing capabilities of an augmented reality (AR) sensorimotor assessment tool. Currently, the tool mimics the capabilities of validated 2-dimensional (2D) assessments. While this modality is well-aligned with seated interactions with flight-deck displays, extravehicular activity (EVA) tasks will require full body mobility wherein astronauts reach and bend in a 3-dimensional (3D) space. These types of motions, naturally including head movements, are known to elicit experiences of disorientation. Thus, sensorimotor assessments that include these stimulations and larger movements are important for understanding astronaut readiness prior to initial EVAs. The proposed effort will extend the current 2D eye-hand coordination task in AR to a 3D task designed to evaluate dynamic balance and eye-hand-body coordination. The new module will extend the range of motion and interaction space such that it better aligns with operational task needs for astronaut readiness evaluation.

Sarah Catherine Moudy↗

NASA NTRS1999

Wireless Augmented Reality Prototype (WARP)

Initiated in January, 1997, under NASA's Office of Life and Microgravity Sciences and Applications, the Wireless Augmented Reality Prototype (WARP) is a means to leverage recent advances in communications, displays, imaging sensors, biosensors, voice recognition and microelectronics to develop a hands-free, tetherless system capable of real-time personal display and control of computer system resources. Using WARP, an astronaut may efficiently operate and monitor any computer-controllable activity inside or outside the vehicle or station. The WARP concept is a lightweight, unobtrusive heads-up display with a wireless wearable control unit. Connectivity to the external system is achieved through a high-rate radio link from the WARP personal unit to a base station unit installed into any system PC. The radio link has been specially engineered to operate within the high- interference, high-multipath environment of a space shuttle or space station module. Through this virtual terminal, the astronaut will be able to view and manipulate imagery, text or video, using voice commands to control the terminal operations. WARP's hands-free access to computer-based instruction texts, diagrams and checklists replaces juggling manuals and clipboards, and tetherless computer system access allows free motion throughout a cabin while monitoring and operating equipment.

Devereaux, A. S.↗

NASA NTRS2023

Joint Augmented Reality Visual Informatics System: Concept of Operations

NASA proposed requirements for a digital display for an EVA spacesuit to provide relevant information to the crew member. The Joint Augmented Reality Visual Informatics System (Joint AR) project pursued four years of research and development towards a suit-display system in a near-eye, AR form factor. The project was responsible for developing software (custom graphics engine and core flight software), physical hardware prototyping (controls, projection display optics, suited display platform), virtual prototyping platform (a virtual reality testbed), and human-in-the-loop (HITL) operational testing informed by EVA flight controllers, crew members, and human factors engineers for con-ops definition. This document contains substantial updates to CTSD-ADV-1788 Rev. Basic. This revision was produced by the project to summarize the use-cases and and user experiences developed throughout the project, and refine the Basic revision originally drafted at the beginning of the project life cycle. The primary purpose of this document is to summarize and make available the scenario development efforts that have been pursued and explored within the Joint AR project. This includes descriptions of the scenarios themselves as well as corresponding potential of advanced informatics displays to support those specified scenarios. In doing so, this document provides a variety of approaches to deconstruct and hypothesize how future technological capabilities so that with future EVA work demands can be satisfied within future human planetary spaceflight missions.

Matthew Miller↗

NASA NTRS2013

Spacecraft 3D Augmented Reality Mobile App

The Spacecraft 3D application allows users to learn about and interact with iconic NASA missions in a new and immersive way using common mobile devices. Using Augmented Reality (AR) techniques to project 3D renditions of the mission spacecraft into real-world surroundings, users can interact with and learn about Curiosity, GRAIL, Cassini, and Voyager. Additional updates on future missions, animations, and information will be ongoing. Using a printed AR Target and camera on a mobile device, users can get up close with these robotic explorers, see how some move, and learn about these engineering feats, which are used to expand knowledge and understanding about space. The software receives input from the mobile device's camera to recognize the presence of an AR marker in the camera's field of view. It then displays a 3D rendition of the selected spacecraft in the user's physical surroundings, on the mobile device's screen, while it tracks the device's movement in relation to the physical position of the spacecraft's 3D image on the AR marker.

Hussey, Kevin J.↗

DOE OSTI2022

Bringing chemical structures to life with augmented reality, machine learning, and quantum chemistry

Visualizing 3D molecular structures is crucial to understanding and predicting their chemical behavior. However, static 2D hand-drawn skeletal structures remain the preferred method of chemical communication. Here, we combine cutting-edge technologies in augmented reality (AR), machine learning, and computational chemistry to develop MolAR, an open-source mobile application for visualizing molecules in AR directly from their hand-drawn chemical structures. Users can also visualize any molecule or protein directly from its name or protein data bank ID and compute chemical properties in real time via quantum chemistry cloud computing. MolAR provides an easily accessible platform for the scientific community to visualize and interact with 3D molecular structures in an immersive and engaging way.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS2018

Occluded Object Reconstruction for First Responders with Augmented Reality Glasses Using Conditional Generative Adversarial Networks

Firefighters suffer a variety of life-threatening risks, including line-of-duty deaths, injuries, and exposures to hazardous substances. Support for reducing these risks is important. We built a partially occluded object reconstruction method on augmented reality glasses for first responders. We used a deep learning based on conditional generative adversarial networks to train associations between the various images of flammable and hazardous objects and their partially occluded counterparts. Our system then reconstructed an image of a new flammable object. Finally, the reconstructed image was superimposed on the input image to provide "transparency". The system imitates human learning about the laws of physics through experience by learning the shape of flammable objects and the flame characteristics.

Chow, Edward↗

NASA NTRS2017

Fireballs in the Sky: An Augmented Reality Citizen Science Program

Fireballs in the Sky is an innovative Australian citizen science program that connects the public with the research of the Desert Fireball Network (DFN). This research aims to understand the early workings of the solar system, and Fireballs in the Sky invites people around the world to learn about this science, contributing fireball sightings via a user-friendly augmented reality mobile app. Tens of thousands of people have downloaded the app world-wide and participated in the science of meteoritics. The Fireballs in the Sky app allows users to get involved with the Desert Fireball Network research, supplementing DFN observations and providing enhanced coverage by reporting their own meteor sightings to DFN scientists. Fireballs in the Sky reports are used to track the trajectories of meteors - from their orbit in space to where they might have landed on Earth. Led by Phil Bland at Curtin University in Australia, the Desert Fireball Network (DFN) uses automated observatories across Australia to triangulate trajectories of meteorites entering the atmosphere, determine pre-entry orbits, and pinpoint their fall positions. Each observatory is an autonomous intelligent imaging system, taking 1000 by 36 megapixel all-sky images throughout the night, using neural network algorithms to recognize events. They are capable of operating for 12 months in a harsh environment, and store all imagery collected. We developed a completely automated software pipeline for data reduction, and built a supercomputer database for storage, allowing us to process our entire archive. The DFN currently stands at 50 stations distributed across the Australian continent, covering an area of 2.5 million square kilometers. Working with DFN's partners at NASA's Solar System Exploration Research Virtual Institute, the team is expanding the network beyond Australia to locations around the world. Fireballs in the Sky allows a growing public base to learn about and participate in this exciting research.

Day, Brian↗