Search NASA⌕ Search

SEARCH · Search NASA

Results for “Augmented Reality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Towards multi-platform software architecture for Collaborative Teleoperation

Augmented Reality (AR) can provide to a Human Operator (HO) a real help in achieving complex tasks, such as remote control of robots and cooperative teleassistance. Using appropriate augmentations, the HO can interact faster, safer and easier with the remote real world. In this paper, we present an extension of an existing distributed software and network architecture for collaborative teleoperation based on networked human-scaled mixed reality and mobile platform. The first teleoperation system was composed by a VR application and a Web application. However the 2 systems cannot be used together and it is impossible to control a distant robot simultaneously. Our goal is to update the teleoperation system to permit a heterogeneous collaborative teleoperation between the 2 platforms. An important feature of this interface is based on the use of different Virtual Reality platforms and different Mobile platforms to control one or many robots.

99 GENERAL AND MISCELLANEOUS↗

Exploration ToolKit (ExTK)

The Exploration Toolkit (ExTK) is a reusable Extended Reality (XR) system developed for incorporating and exploring 3-Dimensional (3-D) computer aided design (CAD) models in XR, with a primary focus on Augmented Reality (AR). The ExTK consists of a Developer Mode and a User Mode. In Developer Mode, ExTK provides developers with the ability to easily import 3-D CAD models and activate desired exploration functionality and layout. Multiple models can be added to a single instantiation of the ExTK using Unity's Scene capability. Exploration functions include scaling, rotating, explode/contract, animations, hiding parts, submodules, and measurement functions. In User Mode, ExTK provides a menu system that allows users to select models and initiate exploration functions. ExTK is architected for reusability and developers can customize the ExTK layout and functions according to application needs. ExTK is designed to be hardware agnostic, although initial development focused on the Microsoft HoloLens as the primary deployment platform. The ExTK is developed using the Unity Game Engine Development Platform. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-1506 O

Klein, BrandonThorin↗

Evaluation Challenges for the Application of Extended Reality Devices in Medicine

Augmented and virtual reality devices are being actively investigated and implemented for a wide range of medical uses. However, significant gaps in the evaluation of these medical devices and applications hinder their regulatory evaluation. Addressing these gaps is critical to demonstrating the devices’ safety and effectiveness. We outline the key technical and clinical evaluation challenges discussed during the US Food and Drug Administration’s public workshop, “Medical Extended Reality: Toward Best Evaluation Practices for Virtual and Augmented Reality in Medicine” and future directions for evaluation method development. Evaluation challenges were categorized into several key technical and clinical areas. Finally, we highlight current efforts in the standards communities and illustrate connections between the evaluation challenges and the intended uses of the medical extended reality (MXR) devices. Participants concluded that additional research is needed to assess the safety and effectiveness of MXR devices across the use cases.

60 APPLIED LIFE SCIENCES↗

Integrating extended reality with inspection systems

An extended reality (“XR”) inspection system for inspecting a target is provided. The XR inspection system includes an inspection system and an XR device. The inspection system includes an inspection main unit and an inspection probe. The inspection main unit collects inspection data from the inspection probe, receives commands input by an inspector using the XR device, performs functions associated with the commands, and sends display data to the XR device. The XR device provides an augmented reality display and sensors. The XR device displays display data received from the inspection main unit, receives commands via the sensors from the inspector, and sends the commands to the inspection main unit.

Keene, Lionel↗

Longitudinal Socio-Emotional Learning Intervention for Autism via Smartglasses: Qualitative School Teacher Descriptions of Practicality, Usability, and Efficacy in General and Special Education Classroom Settings

There is considerable demand for special education services for the over half a million students with autism in the United States. While assistive technology may augment educational services, its implementation is often prevented by a number of practical and attitudinal barriers. These barriers are especially pertinent for the newest and thus least familiar digital systems, such as computerized smartglasses loaded with specialized software modules. Computerized smartglasses are a technology that has already been shown to have an ability to deliver educational interventions through augmented reality. With this in mind, we sought to understand how school educators received and assessed the practicality of a smartglasses-based educational intervention in a single-subject study. The intervention was designed to aid with attention and social educational learning in autism. The intervention was delivered twice a day during a two-week study on a 13-year-old student with autism who was attending a mainstream middle school in Massachusetts. Three different school educators delivered the intervention: the student’s general education teacher, special education teacher, and paraprofessional. Educators recorded their attitudes, the practicality of the technology, and its impact on the student and their classroom through the use of a digital log and a series of in-person interviews. Overall, the school educators experienced a positive view of the smartglasses. The smartglasses intervention was found to be logistically practical to implement, easily usable by both the educator and student, and not time-consuming to learn or implement. Educators also identified the experience as being fun for the student, and felt that the student demonstrated improvement in his verbal and non-verbal skills. There were no adverse effects on the other students or the classroom, and the technology did not result in a distraction. These findings suggest that social skills interventions delivered by smartglasses may be practical, useful, and may lead to improvements in social communication skills. Further research on smartglasses may help to clarify the future role for augmenting special education in students with autism.

Keshav, Neha↗

Ultra‐Flexible Pixelated Perovskite Photodetectors Enabled by Honeycomb Polymer Grids for High‐Resolution Imaging

Abstract A nature‐inspired fabrication method based on a photolithography‐free flexible polymer grid is reported for high‐resolution pixelation of perovskite photodiode arrays with exceptional mechanical ductility and a morphology resembling that of natural compound eyes. The resulting pixelated perovskite photosensitive layer has a ≈1 µm pixel size with 2000 Pixels per inch (PPI) resolution when fully assembled as a photodetector array, delivering a detectivity of >10 13 Jones while providing cross‐talk free imaging. Using a polymer grid effectively releases stress on the perovskite platform, greatly increasing the mechanical agility of the otherwise brittle perovskite film. This novel fabrication methodology and device design offer new possibilities for applications in robotics, biomedical imaging, and virtual and augmented reality.

Zheng, Ding [Department of Chemistry and the Mater↗

What limits the efficiency of GaN-based superluminescent light-emitting diodes (SLEDs)?

Gallium-nitride-based SLEDs are attractive light sources for augmented reality displays and other applications. However, the electrical-to-optical power conversion efficiency (PCE) of SLEDs is still far below the record-high values reported for LEDs. Utilizing advanced numerical device simulation, this paper investigates the internal physical processes that cause the low PCE of SLEDs. The poor hole conductivity strongly reduces the electrical efficiency, similar to laser diodes. However, in contrast to laser diodes, the rising carrier density in the active layers is identified as main reason for enhanced Auger recombination that severely limits the internal quantum efficiency. Design improvement options are demonstrated.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

In vivo virtual intraoperative surgical photoacoustic microscopy

We developed a virtual intraoperative surgical photoacoustic microscopy system by combining with a commercial surgical microscope and photoacoustic microscope (PAM). By sharing the common optical path in the microscope and PAM system, we could acquire the PAM and microscope images simultaneously. Moreover, by employing a beam projector to back-project 2D PAM images onto the microscope view plane as augmented reality, the conventional microscopic and 2D cross-sectional PAM images are concurrently mapped on the plane via an ocular lens of the microscope in real-time. Further, we guided needle insertion into phantom ex vivo and mice skins in vivo.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Integration of 5G and Time Sensitive Networks in Fossil Energy Generation Systems: A Case Study

Precise timing and data transmission within stringent time constraints are critical for numerous applications, such as robotics, virtual and augmented reality, industrial automation, energy and medical business, and various other sectors. Time-sensitive networks (TSN) and fifth-generation wireless communications (5G) are crucial for industrial communications, enabling convergent communication for various services using a common network core. Applications that are time-sensitive and require deterministic communications with low latency fall into this category, such as generation plant systems. This paper presents a simulated model of 5G-TSN for a fossil power plant based on the wired network parameters implemented in situ. Metrics analysis, comparison and future work are presented. © 2024 IEEE.

01 COAL, LIGNITE, AND PEAT↗

A Digital Twin Framework for Liquid-cooled Supercomputers as Demonstrated at Exascale

We present ExaDigiT, an open-source framework for developing comprehensive digital twins of liquid-cooled supercomputers. It integrates three main modules: (1) a resource allocator and power simulator, (2) a transient thermo-fluidic cooling model, and (3) an augmented reality model of the supercomputer and central energy plant. The framework enables the study of "what-if" scenarios, system optimizations, and virtual prototyping of future systems. Using Frontier as a case study, we demonstrate the framework's capabilities by replaying six months of system telemetry for systematic verification and validation. Such a comprehensive analysis of a liquid-cooled exascale supercomputer is the first of its kind. ExaDigiT elucidates complex transient cooling system dynamics, runs synthetic or real workloads, and predicts energy losses due to rectification and voltage conversion. Throughout our paper, we present lessons learned to benefit HPC practitioners developing similar digital twins. We envision the digital twin will be a key enabler for sustainable, energy-efficient supercomputing.

Brewer, Wes↗

Fast-MICP for frameless image-guided surgery

Purpose: In image-guided surgery (IGS) systems, image-to-physical registration is critical for reliable anatomical information mapping and spatial guidance. Conventional stereotactic frame-based or fiducial-based approaches provide accurate registration but are not patient-friendly. This study proposes a frameless cranial IGS system that uses computer vision techniques to replace the frame or fiducials with the natural features of the patient. Methods: To perform a cranial surgery with the proposed system, the facial surface of the patient is first reconstructed by stereo vision. Accuracy is ensured by capturing parallel-line patterns projected from a calibrated LCD projector. Meanwhile, another facial surface is reconstructed from preoperative computed tomography (CT) images of the patient. The proposed iterative closest point (ICP)-based algorithm [fast marker-added ICP (Fast-MICP)] is then used to register the two facial data sets, which transfers the anatomical information from the CT images to the physical space. Results: Experimental results reveal that the Fast-MICP algorithm reduces the computational cost of marker-added ICP (J.-D. Lee et al., ''A coarse-to-fine surface registration algorithm for frameless brain surgery,'' in Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society, 2007, pp. 836-839) to 10% and achieves comparable registration accuracy, which is under 3 mm target registration error (TRE). Moreover, two types of optical-based spatial digitizing devices can be integrated for further surgical navigation. Anatomical information or image-guided surgical landmarks can be projected onto the patient to obtain an immersive augmented reality environment. Conclusion: The proposed frameless IGS system with stereo vision obtains TRE of less than 3 mm. The proposed Fast-MICP registration algorithm reduces registration time by 90% without compromising accuracy.

60 APPLIED LIFE SCIENCES↗

Interactive initialization of 2D/3D rigid registration

Purpose: Registration is one of the key technical components in an image-guided navigation system. A large number of 2D/3D registration algorithms have been previously proposed, but have not been able to transition into clinical practice. The authors identify the primary reason for the lack of adoption with the prerequisite for a sufficiently accurate initial transformation, mean target registration error of about 10 mm or less. In this paper, the authors present two interactive initialization approaches that provide the desired accuracy for x-ray/MR and x-ray/CT registration in the operating room setting. Methods: The authors have developed two interactive registration methods based on visual alignment of a preoperative image, MR, or CT to intraoperative x-rays. In the first approach, the operator uses a gesture based interface to align a volume rendering of the preoperative image to multiple x-rays. The second approach uses a tracked tool available as part of a navigation system. Preoperatively, a virtual replica of the tool is positioned next to the anatomical structures visible in the volumetric data. Intraoperatively, the physical tool is positioned in a similar manner and subsequently used to align a volume rendering to the x-ray images using an augmented reality (AR) approach. Both methods were assessed using three publicly available reference data sets for 2D/3D registration evaluation. Results: In the authors' experiments, the authors show that for x-ray/MR registration, the gesture based method resulted in a mean target registration error (mTRE) of 9.3 ± 5.0 mm with an average interaction time of 146.3 ± 73.0 s, and the AR-based method had mTREs of 7.2 ± 3.2 mm with interaction times of 44 ± 32 s. For x-ray/CT registration, the gesture based method resulted in a mTRE of 7.4 ± 5.0 mm with an average interaction time of 132.1 ± 66.4 s, and the AR-based method had mTREs of 8.3 ± 5.0 mm with interaction times of 58 ± 52 s. Conclusions: Based on the authors' evaluation, the authors conclude that the registration approaches are sufficiently accurate for initializing 2D/3D registration in the OR setting, both when a tracking system is not in use (gesture based approach), and when a tracking system is already in use (AR based approach)

62 RADIOLOGY AND NUCLEAR MEDICINE↗

ExaDigiT/RAPS

ExaDigiT is a framework for developing comprehensive digital twins of liquid-cooled supercomputers, which has three main modules: (1) a python-based Resource Allocator and Power Simulator (RAPS), (2) a Modelica-based Thermo-Fluidic cooling model, and (3) a C++-based augmented reality model built on Unreal Engine 5. RAPS either simulates workloads or replays historical workloads from system telemetry, and is able to predict dynamic energy consumption, as well as interact with the cooling model to predict cooling system behavior. Such a tool can be used together with reinforcement learning algorithms to provide an end-to-end optimization tool for data centers.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗

datacenterCoolingModel

ExaDigiT is a framework for developing comprehensive digital twins of liquid-cooled supercomputers, which has three main modules: (1) a python-based Resource Allocator and Power Simulator (RAPS), (2) a Modelica-based Thermo-Fluidic cooling model, and (3) a C++-based augmented reality model built on Unreal Engine 5. The Modelica-based cooling model is primarily built-on the open-source Transient Simulation Framework of Reconfigurable Models (TRANSFORM) library and the open-source autocsm library. The library follows the templating architecture developed in the TRANSFORM and the autocsm libraries. This tool can be easily extended to model other Frontier-like liquid cooled supercomputers.

Kumar, Vineet [Oak Ridge National Laboratory (ORNL↗

Robust multiresonant nonlocal metasurfaces by rational design

Abstract Dielectric metasurfaces supporting optical resonances have become a promising platform for quantum and nonlinear optics. However, resonant metasurfaces remain limited in their capacity to independently control the behavior of many distinct resonances despite efforts in computational optimization and inverse design. In this work, we overcome longstanding limitations by introducing a generalized rational design paradigm based on symmetry. Specifically, we use symmetry-broken metasurfaces with periodic “quadromer” lattices comprised of four nanostructures per unit cell to enable extensive control of multiple optical resonances. The rationally designed metasurfaces are readily fabricable, and we experimentally demonstrate metasurfaces that support up to four high Q-factor resonances with deliberately chosen free-space polarizations, spectral separations, and mode profiles. Our design paradigm may unlock new applications for multiresonant metasurfaces in quantum and nonlinear optics, optical sensing, and augmented reality displays.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Two 175 ton geothermal chiller heat pumps for leed platinum building technology demonstration project. Operation data, data collection and marketing

The activities funded by this grant helped educate and inform approximately six thousand individuals who participated in guided tours of the geothermal chiller plant at Johnson Controls Corporate Headquarters in Glendale, Wisconsin over the three year term of the project. In addition to those who took the formal tour, thousands more were exposed to hands-on learning at the self-service video kiosks located in the headquarters building and augmented reality tablet app that allowed for self-guided tours. The tours, video, and app focused on the advantages of geothermal heat pump chillers, including energy savings and environmental impact. The overall tour and collateral also demonstrated the practical application of this technology and how it can be designed into a system that includes many other sustainable technologies without sacrificing comfort or health of building occupants Among tour participants were nearly 1,000 individuals, representing 130 organizations identified as potential purchasers of geothermal heat pump chillers. In addition to these commercial clients, tours were well attended by engineering, facilities, and business trade groups. This has also been a popular tour for groups from Universities around the Midwest and K-12 schools from Wisconsin and Northern Illinois A sequence of operations was put into place to control the chillers and they have been tuned and maintained to optimize the benefit from the geothermal water loop. Data on incoming and outgoing water temperature and flow from the geothermal field was logged and sent to DOE monthly during the grant period to demonstrate energy savings.

15 GEOTHERMAL ENERGY↗

Concrete Inspection in Remote Areas with Robotic Arms

The use of non-destructive examination (NDE) to assess the conditions of concrete structures has received extensive attention in recent years. NDE analysis techniques are used to evaluate the properties of materials and to ensure the integrity of structures without causing damages to them. NDE inspection is often performed manually by certified technicians, however in remote and high hazard environments this is impossible or undesirable for human safety. Development of remote and robotic technologies to perform these essential inspections is of interest to the DOE EM complex due to its aging infrastructure and hazardous environments. This goal of this seedling was to develop techniques to remotely examine inaccessible concrete structures and to evaluate results collected remotely to those obtained by human hand. Additionally, two promising state of the art NDE instruments were identified to be used for the characterization to include the new Proceq Ground Penetrating Radar (GPR) which generates results in 3D using Augmented Reality. A test bed was built to develop and evaluate deployment techniques of the NDE instruments with a robotic arm. Concrete test forms simulating the rough concrete surfaces, strength, composition and potential structural defects that can be found at our DOE EM facilities were built for the examination. Initial NDE instrumentation data collection has been performed on the concrete test forms.

47 OTHER INSTRUMENTATION↗

Spatially Adaptive Tunable Lighting Control System with Expanded Wellness and Energy Saving Benefits

Lighting design is becoming increasingly complex, including active dimming for energy savings and spectral tuning for human wellbeing. Modern commercial lighting control systems are already difficult to use, and maintain, and even with so-called Smart Lighting, optimizing light settings is rapidly exceeding the capabilities of direct human control, limiting the adoption of lighting control systems. The growing importance of Occupancy Centric Controls (OCC) for advanced building management systems can be applied to the development of autonomous lighting system controls needed to drive the adoption of advanced lighting controls for improved adaptive “sculpted illumination” for both greater energy savings and broader human wellbeing perspectives. This project, led by Rensselaer Polytechnic Institute, entitled “Spatially adaptive tunable lighting control system with expanded wellness and energy saving benefits” will develop and test an entirely new platform for automated optimized lighting design and control that requires little or no human engagement, yet contours lighting profiles automatically to minimize lighting energy use. We call this approach to lighting control “dynamic light sculpting” since the right amount of illumination is automatically delivered to occupants in real time. Since the system uses new, privacy-preserving occupant position and pose detection technologies developed by Rensselaer for broad OCC building applications, the control system will analyze how to deliver the right amount of the right type of illumination only where and when it is needed based these OCC platforms. To create these powerful autonomous lighting control platforms, the project will integrate evolving augmented reality (AR) and virtual reality (VR) tools with sophisticated lighting and interior design toolkits to create interactive lighting design and control simulators. Using the quickly growing paradigm of digital twins, these tools will integrate light fixture properties, occupancy sensor data, interior design data, and lighting specifications to accurately visualize how various design concepts interact with simulated yet realistic human activities that occur in commercial office buildings of various types. These advanced digital twin design and simulation tools will be combined with a new class of digitally-programmable LED lighting fixtures that can dynamically change the spectral content and direction of light emission. When fully integrated, the new autonomous lighting control system will take all of the guesswork out of optimizing light quality while minimizing energy consumption. Digital twin tools will simplify the design, installation, commissioning, operation, and maintenance of future energy-efficient lighting systems. It should be possible to reduce lighting energy costs from 40% to 70% with OCC based dynamic light sculpting systems. The work will be led by the Center for Lighting Enabled Systems and Applications (LESA) and the Center for Architectural Science and Ecology (CASE), both at Renssealer Polytechnic Institute; Lumileds, a global leader in the development of advanced LED systems; and HKS, a leading global architecture design firm. This interdisciplinary project team’s experience includes all of the design simulation tools, VR/AR digital twin visualization technology, advanced occupancy sensing technology, and complex control system design capabilities that will revolutionize lighting design and control technology to autonomously deliver high-quality lighting that improves human health and wellbeing while simultaneously maximizing lighting energy savings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗