Search NASA⌕ Search

SEARCH · Search NASA

Results for “Microsoft Kinect”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

SU-E-J-66: Evaluation of a Real-Time Positioning Assistance Simulator System for Skull Radiography Using the Microsoft Kinect

Purpose: The purpose of this study is to investigate the feasibility of a low cost, small size positioning assistance simulator system for skull radiography using the Microsoft Kinect sensor. A conventional radiographic simulator system can only measure the three-dimensional coordinates of an x-ray tube using angle sensors, but not measure the movement of the subject. Therefore, in this study, we developed a real-time simulator system using the Microsoft Kinect to measure both the x-ray tube and the subject, and evaluated its accuracy and feasibility by comparing the simulated and the measured x-ray images. Methods: This system can track a head phantom by using Face Tracking, which is one of the functions of the Kinect. The relative relationship between the Kinect and the head phantom was measured and the projection image was calculated by using the ray casting method, and by using three-dimensional CT head data with 220 slices at 512 × 512 pixels. X-ray images were thus obtained by using a computed radiography (CR) system. We could then compare the simulated projection images with the measured x-ray images from 0 degrees to 45 degrees at increments of 15 degrees by calculating the cross correlation coefficient C. Results: The calculation time of the simulated projection images was almost real-time (within 1 second) by using the Graphics Processing Unit(GPU). The cross-correlation coefficients C are: 0.916; 0.909; 0.891; and, 0.886 at 0, 15, 30, and 45 degrees, respectively. As a result, there were strong correlations between the simulated and measured images. Conclusion: This system can be used to perform head positioning more easily and accurately. It is expected that this system will be useful for learning radiographic techniques by students. Moreover, it could also be used for predicting the actual x-ray image prior to x-ray exposure in clinical environments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

SU-G-JeP4-01: An Assessment of a Microsoft Kinect V2 Sensor for Voluntary Breath-Hold Monitoring in Radiotherapy

Purpose: To determine whether the Microsoft Kinect Version 2 (Kinect v2), a commercial off-the-shelf (COTS) depth sensors designed for entertainment purposes, were robust to the radiotherapy treatment environment and could be suitable for monitoring of voluntary breath-hold compliance. This could complement current visual monitoring techniques, and be useful for heart sparing left breast radiotherapy. Methods: In-house software to control Kinect v2 sensors, and capture output information, was developed using the free Microsoft software development kit, and the Cinder creative coding C++ library. Each sensor was used with a 12m USB 3.0 active cable. A solid water block was used as the object. The depth accuracy and precision of the sensors was evaluated by comparing Kinect reported distance to the object with a precision laser measurement across a distance range of 0.6m to 2.0 m. The object was positioned on a high-precision programmable motion platform and moved in two programmed motion patterns and Kinect reported distance logged. Robustness to the radiation environment was tested by repeating all measurements with a linear accelerator operating over a range of pulse repetition frequencies (6Hz to 400Hz) and dose rates 50 to 1500 monitor units (MU) per minute. Results: The complex, consistent relationship between true and measured distance was unaffected by the radiation environment, as was the ability to detect motion. Sensor precision was < 1 mm and the accuracy between 1.3 mm and 1.8 mm when a distance correction was applied. Both motion patterns were tracked successfully with a root mean squared error (RMSE) of 1.4 and 1.1 mm respectively. Conclusion: Kinect v2 sensors are capable of tracking pre-programmed motion patterns with an accuracy <2 mm and appear robust to the radiotherapy treatment environment. A clinical trial using Kinect v2 sensor for monitoring voluntary breath hold has ethical approval and is open to recruitment. The authors are supported by a National Institute of Health Research (NIHR) Career Development Fellowship (CDF-2013-06-005). Microsoft Corporation donated three sensors. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2013

Rapid Characterization of Vegetation Structure with a Microsoft Kinect Sensor

The importance of vegetation structure and biomass in controlling land-atmosphere exchange is widely recognized, but measurements of canopy structure are challenging, time consuming, and often rely on destructive methods. The Microsoft Kinect is an infrared sensor designed for video gaming that outputs synchronized color and depth images and that has the potential to allow rapid characterization of vegetation structure. We compared depth images from a Kinect sensor with manual measurements of plant structure and size for two species growing in a California grassland. The depth images agreed well with the horizontal and vertical measurements of plant size made manually. Similarly, the plant volumes calculated with a three-dimensional convex hulls approach was well related to plant biomass. The Kinect showed some limitations for ecological observation associated with a short measurement range and daytime light contamination. Nonetheless, the Kinect’s light weight, fast acquisition time, low power requirement, and cost make it a promising tool for rapid field surveys of canopy structure, especially in small-statured vegetation.

Microsoft Kinect↗
DOE OSTI2014

Sci—Thur AM: YIS - 10: Use of the Microsoft Kinect for applications of patient surface data to radiotherapy

Current techniques to acquire patient surface data are often very expensive and lack flexibility. In this study, the use of the Microsoft Kinect to reliably acquire 3D scans of patient surface is investigated. A design is presented to make the system easily applicable to the clinic. Potential applications of the device to radiotherapy are also presented. Scan reproducibility was tested by repeatedly scanning an anthropomorphic phantom. Scan accuracy was tested by comparing Kinect scans to the surface extracted from a CT dataset of a Rando® anthropomorphic phantom, which was considered as the true reference surface. Average signed distances of 0.12 ± 2.34 mm and 0.13 ± 2.04 mm were obtained between the compared surfaces for reproducibility and accuracy respectively. This is conclusive, since it indicates that the variations observed come largely from noise distributed around an average distance close to 0 mm. Moreover, the range of the noise is small enough for the system to reliably capture a patient's surface. A system was also designed using two Kinects used together to acquire 3D surfaces in a quick and stable way that is applicable to the clinic. Finally, applications of the device to radiotherapy are demonstrated. Its use to detect local positioning errors is presented, where small local variations difficult to see with the naked eye are clearly visible. The system was also used to predict collisions using gantry and patient scans and thus ensure the safety of unconventional trajectories.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking

Purpose: Investigate the use of the Kinect 2.0 for patient motion tracking during radiotherapy by studying spatial and depth resolution capabilities. Methods: Using code written in C#, depth map data was abstracted from the Kinect to create an initial depth map template indicative of the initial position of an object to be compared to the depth map of the object over time. To test this process, simple setup was created in which two objects were imaged: a 40 cm × 40 cm board covered in non reflective material and a 15 cm × 26 cm textbook with a slightly reflective, glossy cover. Each object, imaged and measured separately, was placed on a movable platform with object to camera distance measured. The object was then moved a specified amount to ascertain whether the Kinect’s depth camera would visualize the difference in position of the object. Results: Initial investigations have shown the Kinect depth resolution is dependent on the object to camera distance. Measurements indicate that movements as small as 1 mm can be visualized for objects as close as 50 cm away. This depth resolution decreases linearly with object to camera distance. At 4 m, the depth resolution had decreased to observe a minimum movement of 1 cm. Conclusion: The improved resolution and advanced hardware of the Kinect 2.0 allows for increase of depth resolution over the Kinect 1.0. Although obvious that the depth resolution should decrease with increasing distance from an object given the decrease in number of pixels representing said object, the depth resolution at large distances indicates its usefulness in a clinical setting.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2011

Microsoft Kinect Sensor Evaluation

My summer project evaluates the Kinect game sensor input/output and its suitability to perform as part of a human interface for a spacecraft application. The primary objective is to evaluate, understand, and communicate the Kinect system's ability to sense and track fine (human) position and motion. The project will analyze the performance characteristics and capabilities of this game system hardware and its applicability for gross and fine motion tracking. The software development kit for the Kinect was also investigated and some experimentation has begun to understand its development environment. To better understand the software development of the Kinect game sensor, research in hacking communities has brought a better understanding of the potential for a wide range of personal computer (PC) application development. The project also entails the disassembly of the Kinect game sensor. This analysis would involve disassembling a sensor, photographing it, and identifying components and describing its operation.

Billie, Glennoah↗
NASA NTRS2018

Developing a Natural User Interface and Facial Recognition System With OpenCV and the Microsoft Kinect

The task for this project was to design, develop, test, and deploy a facial recognition system for the Kennedy Space Center Augmented/Virtual Reality Lab. This system will serve as a means of user authentication as part of the NUI of the lab. The overarching goal is to create a seamless user interface that will allow the user to initiate and interact with AR and VR experiences without ever needing to use a mouse or keyboard at any step in the process.

Human Interfaces↗
DOE OSTI2014

A real-time applicator position monitoring system for gynecologic intracavitary brachytherapy

Purpose: To develop a real-time applicator position monitoring system (RAPS) for intracavitary brachytherapy using an infrared camera and reflective markers. Methods: 3D image-guided brachytherapy requires high accuracy of applicator localization; however, applicator displacement can happen during patient transfer for imaging and treatment delivery. No continuous applicator position monitoring system is currently available. The RAPS system was developed for real-time applicator position monitoring without additional radiation dose to patients. It includes an infrared camera, reflective markers, an infrared illuminator, and image processing software. After reflective markers are firmly attached to the applicator and the patient body, applicator displacement can be measured by computing the relative change in distance between the markers. The reflective markers are magnetic resonance imaging (MRI) compatible, which is suitable for MRI-guided HDR brachytherapy paradigm. In our prototype, a Microsoft Kinect sensor with a resolution of 640 by 480 pixels is used as an infrared camera. A phantom study was carried out to compare RAPS' measurements with known displacements ranging from −15 to +15 mm. A reproducibility test was also conducted. Results: The RAPS can achieve 4 frames/s using a laptop with Intel{sup ®} Core™2 Duo processor. When the pixel size is 0.95 mm, the difference between RAPS' measurements and known shift values varied from 0 to 0.8 mm with the mean value of 0.1 mm and a standard deviation of 0.44 mm. The system reproducibility was within 0.6 mm after ten reposition trials. Conclusions: This work demonstrates the feasibility of a real-time infrared camera based gynecologic intracavitary brachytherapy applicator monitoring system. Less than 1 mm accuracy is achieved when using an off-the-shelf infrared camera.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
NASA NTRS2015

Augmented Virtual Reality Laboratory

Real time motion tracking hardware has for the most part been cost prohibitive for research to regularly take place until recently. With the release of the Microsoft Kinect in November 2010, researchers now have access to a device that for a few hundred dollars is capable of providing redgreenblue (RGB), depth, and skeleton data. It is also capable of tracking multiple people in real time. For its original intended purposes, i.e. gaming, being used with the Xbox 360 and eventually Xbox One, it performs quite well. However, researchers soon found that although the sensor is versatile, it has limitations in real world applications. I was brought aboard this summer by William Little in the Augmented Virtual Reality (AVR) Lab at Kennedy Space Center to find solutions to these limitations.

Resolved Skeleton↗
NASA NTRS2017

The NASA Augmented/Virtual Reality Lab: The State of the Art at KSC

The NASA Augmented Virtual Reality (AVR) Lab at Kennedy Space Center is dedicated to the investigation of Augmented Reality (AR) and Virtual Reality (VR) technologies, with the goal of determining potential uses of these technologies as human-computer interaction (HCI) devices in an aerospace engineering context. Begun in 2012, the AVR Lab has concentrated on commercially available AR and VR devices that are gaining in popularity and use in a number of fields such as gaming, training, and telepresence. We are working with such devices as the Microsoft Kinect, the Oculus Rift, the Leap Motion, the HTC Vive, motion capture systems, and the Microsoft Hololens. The focus of our work has been on human interaction with the virtual environment, which in turn acts as a communications bridge to remote physical devices and environments which the operator cannot or should not control or experience directly. Particularly in reference to dealing with spacecraft and the oftentimes hazardous environments they inhabit, it is our hope that AR and VR technologies can be utilized to increase human safety and mission success by physically removing humans from those hazardous environments while virtually putting them right in the middle of those environments.

Little, William↗
DOE OSTI2019

Dataset: Inferring Thermal Comfort using Body Shape Information Utilizing Depth Sensors

Thermal comfort is very important for well-being and productivity of building occupants. It has been shown that body shape is a useful feature to determine thermal comfort of individuals [2]. It is because, the heat dissipation rate of individuals depends on the body surface area. As a result, a tall and skinny person can tolerate higher room temperature than a rounded body shape person [5]. In order to test this hypothesis, we performed a year-long experiment in 2017, where we recruited 77 participants and put each of them in a thermally controlled conference room in CMU for 3 hours and recorded their subjective responses regarding thermal comfort at different temperature ranging from 60°F to 80°F. In addition, we collected depth data of individuals using a vertically mounted Microsoft Kinect for XBOX One at the entrance of the conference room to capture their body shape. We also collected biometric features (e.g., Galvanic Skin Response (GSR), skin temperature) using a Microsoft Health Band worn by the subjects. The resulting dataset provides rich information regarding how different features can be used to infer thermal comfort of the individuals.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

A dual-Kinect approach to determine torso surface motion for respiratory motion correction in PET

Purpose: Respiratory gating is commonly used to reduce blurring effects and attenuation correction artifacts in positron emission tomography (PET). Established clinically available methods that employ body-attached hardware for acquiring respiration signals rely on the assumption that external surface motion and internal organ motion are well correlated. In this paper, the authors present a markerless method comprising two Microsoft Kinects for determining the motion on the whole torso surface and aim to demonstrate its validity and usefulness—including the potential to study the external/internal correlation and to provide useful information for more advanced correction approaches. Methods: The data of two Kinects are used to calculate 3D representations of a patient’s torso surface with high spatial coverage. Motion signals can be obtained for any position by tracking the mean distance to a virtual camera with a view perpendicular to the surrounding surface. The authors have conducted validation experiments including volunteers and a moving high-precision platform to verify the method’s suitability for providing meaningful data. In addition, the authors employed it during clinical {sup 18}F-FDG-PET scans and exemplarily analyzed the acquired data of ten cancer patients. External signals of abdominal and thoracic regions as well as data-driven signals were used for gating and compared with respect to detected displacement of present lesions. Additionally, the authors quantified signal similarities and time shifts by analyzing cross-correlation sequences. Results: The authors’ results suggest a Kinect depth resolution of approximately 1 mm at 75 cm distance. Accordingly, valid signals could be obtained for surface movements with small amplitudes in the range of only few millimeters. In this small sample of ten patients, the abdominal signals were better suited for gating the PET data than the thoracic signals and the correlation of data-driven signals was found to be stronger with abdominal signals than with thoracic signals (average Pearson correlation coefficients of 0.74 ± 0.17 and 0.45 ± 0.23, respectively). In all cases, except one, the abdominal respiratory motion preceded the thoracic motion—a maximum delay of approximately 600 ms was detected. Conclusions: The method provides motion information with sufficiently high spatial and temporal resolution. Thus, it enables meaningful analysis in the form of comparisons between amplitudes and phase shifts of signals from different regions. In combination with a large field-of-view, as given by combining the data of two Kinect cameras, it yields surface representations that might be useful in the context of motion correction and motion modeling.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Data fusion for a vision-aided radiological detection system: Calibration algorithm performance

In order to improve the ability to detect, locate, track and identify nuclear/radiological threats, the University of Florida nuclear detection community has teamed up with the 3D vision community to collaborate on a low cost data fusion system. The key is to develop an algorithm to fuse the data from multiple radiological and 3D vision sensors as one system. The system under development at the University of Florida is being assessed with various types of radiological detectors and widely available visual sensors. A series of experiments were devised utilizing two EJ-309 liquid organic scintillation detectors (one primary and one secondary), a Microsoft Kinect for Windows v2 sensor and a Velodyne HDL-32E High Definition LiDAR Sensor which is a highly sensitive vision sensor primarily used to generate data for self-driving cars. Each experiment consisted of 27 static measurements of a source arranged in a cube with three different distances in each dimension. The source used was Cf-252. The calibration algorithm developed is utilized to calibrate the relative 3D-location of the two different types of sensors without need to measure it by hand; thus, preventing operator manipulation and human errors. The algorithm can also account for the facility dependent deviation from ideal data fusion correlation. Use of the vision sensor to determine the location of a sensor would also limit the possible locations and it does not allow for room dependence (facility dependent deviation) to generate a detector pseudo-location to be used for data analysis later. Using manually measured source location data, our algorithm-predicted the offset detector location within an average of 20 cm calibration-difference to its actual location. Calibration-difference is the Euclidean distance from the algorithm predicted detector location to the measured detector location. The Kinect vision sensor data produced an average calibration-difference of 35 cm and the HDL-32E produced an average calibration-difference of 22 cm. Using NaI and He-3 detectors in place of the EJ-309, the calibration-difference was 52 cm for NaI and 75 cm for He-3. In conclusion, the algorithm is not detector dependent; however, from these results it was determined that detector dependent adjustments are required.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2014

Continuous-scanning laser Doppler vibrometry: Extensions to arbitrary areas, multi-frequency and 3D capture

To date, differing implementations of continuous scan laser Doppler vibrometry have been demonstrated by various academic institutions, but since the scan paths were defined using step or sine functions from function generators, the paths were typically limited to 1D line scans or 2D areas such as raster paths or Lissajous trajectories. The excitation was previously often limited to a single frequency due to the specific signal processing performed to convert the scan data into an ODS. In this paper, a configuration of continuous-scan laser Doppler vibrometry is demonstrated which permits scanning of arbitrary areas, with the benefit of allowing multi-frequency/broadband excitation. Various means of generating scan paths to inspect arbitrary areas are discussed and demonstrated. Further, full 3D vibration capture is demonstrated by the addition of a range-finding facility to the described configuration, and iteratively relocating a single scanning laser head. Here, the range-finding facility was provided by a Microsoft Kinect, an inexpensive piece of consumer electronics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2015

SU-E-J-12: An Image-Guided Soft Robotic Patient Positioning System for Maskless Head-And-Neck Cancer Radiotherapy: A Proof-Of-Concept Study

Purpose: We propose a surface-image-guided soft robotic patient positioning system for maskless head-and-neck radiotherapy. The ultimate goal of this project is to utilize a soft robot to realize non-rigid patient positioning and real-time motion compensation. In this proof-of-concept study, we design a position-based visual servoing control system for an air-bladder-based soft robot and investigate its performance in controlling the flexion/extension cranial motion on a mannequin head phantom. Methods: The current system consists of Microsoft Kinect depth camera, an inflatable air bladder (IAB), pressured air source, pneumatic valve actuators, custom-built current regulators, and a National Instruments myRIO microcontroller. The performance of the designed system was evaluated on a mannequin head, with a ball joint fixed below its neck to simulate torso-induced head motion along flexion/extension direction. The IAB is placed beneath the mannequin head. The Kinect camera captures images of the mannequin head, extracts the face, and measures the position of the head relative to the camera. This distance is sent to the myRIO, which runs control algorithms and sends actuation commands to the valves, inflating and deflating the IAB to induce head motion. Results: For a step input, i.e. regulation of the head to a constant displacement, the maximum error was a 6% overshoot, which the system then reduces to 0% steady-state error. In this initial investigation, the settling time to reach the regulated position was approximately 8 seconds, with 2 seconds of delay between the command start of motion due to capacitance of the pneumatics, for a total of 10 seconds to regulate the error. Conclusion: The surface image-guided soft robotic patient positioning system can achieve accurate mannequin head flexion/extension motion. Given this promising initial Result, the extension of the current one-dimensional soft robot control to multiple IABs for non-rigid positioning control will be pursued.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

SU-C-209-06: Improving X-Ray Imaging with Computer Vision and Augmented Reality

Purpose: To determine the feasibility of using a computer vision algorithm and augmented reality interface to reduce repeat rates and improve consistency of image quality and patient exposure in general radiography. Methods: A prototype device, designed for use with commercially available hardware (Microsoft Kinect 2.0) capable of depth sensing and high resolution/frame rate video, was mounted to the x-ray tube housing as part of a Philips DigitalDiagnost digital radiography room. Depth data and video was streamed to a Windows 10 PC. Proprietary software created an augmented reality interface where overlays displayed selectable information projected over real-time video of the patient. The information displayed prior to and during x-ray acquisition included: recognition and position of ordered body part, position of image receptor, thickness of anatomy, location of AEC cells, collimated x-ray field, degree of patient motion and suggested x-ray technique. Pre-clinical data was collected in a volunteer study to validate patient thickness measurements and x-ray images were not acquired. Results: Proprietary software correctly identified ordered body part, measured patient motion, and calculated thickness of anatomy. Pre-clinical data demonstrated accuracy and precision of body part thickness measurement when compared with other methods (e.g. laser measurement tool). Thickness measurements provided the basis for developing a database of thickness-based technique charts that can be automatically displayed to the technologist. Conclusion: The utilization of computer vision and commercial hardware to create an augmented reality view of the patient and imaging equipment has the potential to drastically improve the quality and safety of x-ray imaging by reducing repeats and optimizing technique based on patient thickness. Society of Pediatric Radiology Pilot Grant; Washington University Bear Cub Fund.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2013

Implementation of Headtracking and 3D Stereo with Unity and VRPN for Computer Simulations

This paper explores low-cost hardware and software methods to provide depth cues traditionally absent in monocular displays. The use of a VRPN server in conjunction with a Microsoft Kinect and/or Nintendo Wiimote to provide head tracking information to a Unity application, and NVIDIA 3D Vision for retinal disparity support, is discussed. Methods are suggested to implement this technology with NASA's EDGE simulation graphics package, along with potential caveats. Finally, future applications of this technology to astronaut crew training, particularly when combined with an omnidirectional treadmill for virtual locomotion and NASA's ARGOS system for reduced gravity simulation, are discussed.

Noyes, Matthew A.↗