Search NASA⌕ Search

SEARCH · Search NASA

Results for “multi-view”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Centimeter human skeleton pose estimation

A method, apparatus and system for human skeleton pose estimation includes synchronously capturing images of a human moving through an area from a plurality of different points of view, for each of the plurality of captured images, determining a bounding box that bounds the human in the captured image and identifying pixel locations of the bounding box in the image, for each of the plurality of captured images, determining 2D and single-view 3D skeletons from the pixel locations of the bounding box, determining a first, multi-view 3D skeleton using a combination of the 2D and single-view 3D skeletons, and optimizing the first, multi-view 3D skeleton to determine a final 3D skeleton pose for the human. The method, apparatus and system can further include illuminating the area with structured light during the capturing of the images of the human moving through the area.

Brookshire, Jonathan D.↗

Levenberg–Marquardt multi-classification using hinge loss function

Incorporating higher-order optimization functions, such as Levenberg-Marquardt (LM) have revealed better generalizable solutions for deep learning problems. However, these higher-order optimization functions suffer from very large processing time and training complexity especially as training datasets become large, such as in multi-view classification problems, where finding global optima is a very costly problem. To solve this issue, we develop a solution for LM-enabled classification with, to the best of knowledge first-time implementation of hinge loss, for multiview classification. Hinge loss allows the neural network to converge faster and perform better than other loss functions such as logistic or square loss rates. Here we prove our method by experimenting with various multiclass classification challenges of varying complexity and training data size. The empirical results show the training time and accuracy rates achieved, highlighting how our method outperforms in all cases, especially when training time is limited. Our paper presents important results in the relationship between optimization and loss functions and how these can impact deep learning problems.

97 MATHEMATICS AND COMPUTING↗

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this paper a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

3D instance segmentation↗

Structure of Self-Generated Magnetic Fields in Laser-Solid Interaction from Proton Tomography

Self-generated magnetic fields in laser-solid interactions are experimentally characterized to reveal the 3D location and local field strength, rather than path-integrated quantities, using multi-view proton radiography and tomographic inversion. We infer magnetic fields that extend several millimeters off the target into the hot, rarefied corona, sufficient to strongly magnetize the plasma (Ω e τ e ≫ 1). The data are compared to MHD simulations incorporating recent improvements in modeling magnetic field generation and transport; the volume-averaged coronal field strength and magnetic flux agree to within 25% using a model with magnetic re-localization of transport, although the near-target morphology is not reproduced. This work demonstrates tomographic proton radiography as a valuable tool for investigating magnetic fields in laser-produced plasmas.

High-energy-density plasmas↗

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system

In this study, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

42 ENGINEERING↗

Applications Of Machine Learning to Gas Plume Analysis In Longwave Infrared Hyperspectral Images

Longwave infrared hyperspectral images can be used for gas plume analysis, as many gases exhibit distinct absorption features in this portion of the electromagnetic spectrum. In practice, accurately identifying weak gas signatures is difficult because the observed radiance is dominated by background radiance, which varies with material, temperature, and viewing conditions. Many gas plume analysis pipelines operate on single images, limiting the ability to leverage spatial and multi-view information that could enhance the analysis. The goal of this dissertation is to explore how machine learning and deep learning methods can complement classical approaches to improve gas plume identification in longwave infrared hyperspectral imagery, and to investigate the use of neural radiance fields for hyperspectral scene reconstruction.

3D Scene Reconstruction↗

Multi-modal dynamic radiography using short-pulse laser-generated probe beams

Radiography is an important tool for the interrogation of dynamic experiments in the fields of dynamic properties of materials, and in condensed matter, high explosive, and high-energy-density physics. Multi-modal radiography advances the hypothesis that combining the information delivered by multiple radiographic modalities can lead to more constrained (improved) “reconstruction” of the scene than can be obtained from a single probe. We identify four modalities: multi-probe, time sequence, multi-view, and multi-messenger. Multi-probe radiography is a promising candidate for a next-generation dynamic radiographic facility. High-energy X-rays are the most frequently used probe for dynamic radiography, although recent developments show the utility of proton (pRad), electron (eRad), and neutron probe beams. Because each probing species interacts with material in the radiographic scene through quantitatively different mechanisms, each returns independent information about the scene, which can add extra constraints to the reconstruction process. How to conduct detailed, quantitative “co-analysis” of multiple data streams remains an area of active research. Multi-beam, short-pulse, laser-generated probes offer sufficient dose, an appropriate spectrum, and appropriate spatio-temporal resolution to produce high-quality dynamic radiographs. This paper reports on technology development to advance the state of the art of multi-modal/multi-probe radiography and the pursuit of both deterministic and inferential (AI/ML assisted) co-analysis methodologies to produce more constrained reconstructions from multi-modal data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Vehicular Re-Identification from Uncontrolled Multiple Views

Vehicle re-identification (re-ID) across disparate sensing modalities remains a fundamental challenge for transportation research. In this work, we introduce a deep multi-view vehicle re-ID framework that leverages Siamese networks to compare pairs of vehicle images and produce matching scores, enabling robust association across drastically different viewpoints such as those from UAVs, surveillance cameras, and ground sensors. The model exploits convolutional neural networks to learn features that remain discriminative under changes in angle, distance, and illumination, supporting more generalizable re-ID performance. As part of this effort, we also developed an automated pipeline to synchronize roadside and UAV video streams, producing a multi-perspective dataset that complements preexisting real collections and a synthetic dataset generated in this study. Together, these contributions advance the capability to re-identify vehicles across wide viewing baselines; establish a foundation for scalable, reproducible research in vehicle re-ID; and open pathways for future applications, such as inferring routine behaviors, movement patterns, and daily habits of the individual associated with the vehicle.

convolutional neural networks↗

Deep Learning and Photogrammetric Reconstruction for Automated Crack Detection and Dimensional Measurement in Mining Operations

Surface crack detection and dimensional measurement at active mining sites present significant safety and operational challenges. Manual inspection methods are labor-intensive, spatially incomplete, and expose personnel to hazardous environments, while existing automated approaches have been developed primarily for concrete civil infrastructure and have not been validated on the complex, variable surfaces characteristic of mining environments. This dissertation presents an automated pipeline that integrates deep learning semantic segmentation with Structure-from-Motion photogrammetry to detect surface cracks and measure their aperture, length, and vertical displacement from standard RGB imagery acquired during routine Uncrewed Aerial Vehicle (UAV) survey operations, without requiring additional sensor hardware or manual measurement. The pipeline combines a U-Net architecture with an EfficientNet-B0 encoder, pretrained on the SDNET2018 concrete crack dataset and fine-tuned on a mining-specific dataset spanning laboratory concrete specimens, coal refuse impoundment embankments, and post-blast limestone quarry benches. Photogrammetric reconstruction is performed using COLMAP Structure-from-Motion and Multi-View Stereo, with crack segmentation masks projected into the reconstructed point cloud to enable three-dimensional vertical displacement measurement through local plane fitting and bimodal surface detection. The pipeline was validated across 36 controlled laboratory specimens at three imaging distances and four vertical displacement levels, achieving aperture measurement RMSE of 0.047 cm and R² of 0.954, and vertical displacement RMSE of 0.140 cm and R² of 0.966, against independent caliper measurements. Field application at a coal refuse impoundment in southwestern Pennsylvania detected 71 crack components across the embankment crest, with a dominant longitudinal crack exhibiting aperture values reaching 28 cm and a 95th percentile vertical displacement of 35.53 cm, consistent in magnitude and spatial distribution with simultaneously acquired LiDAR-derived estimates. Application across four post-blast limestone quarry bench datasets in California successfully characterized blast-induced fracture networks at ground sampling distances ranging from 0.59 to 1.23 cm/pixel, with detected crack geometries physically consistent with observable surface conditions at each site. The results demonstrate that deep learning-based crack detection and photogrammetric measurement can be integrated into routine UAV inspection workflows at mining sites, providing repeatable, scalable, and quantitative crack characterization across surface types, crack scales, and displacement magnitudes not previously addressed in the literature. The pipeline requires no dedicated surveying equipment beyond the UAV platforms already deployed at mine sites for survey and monitoring purposes, supporting practical adoption within existing operational workflows.

Crack detection, Dimensional Measurement↗

pccp.c1

Point Cloud of Cloud Points (PCCP) VAP provides a stereo reconstructed Point Cloud of Cloud Points (PCCP) that correspond to the x,y,z coordinates of cloud features detected from stereo image pairs genrated from a multi-view stereo system deployed to collect exclusive set of data on shallow cumulus clouds overhead.

54 ENVIRONMENTAL SCIENCES↗

pccpa.c1

Point Cloud of Cloud Points (PCCP) VAP provides a stereo reconstructed Point Cloud of Cloud Points (PCCP) that correspond to the x,y,z coordinates of cloud features detected from stereo image pairs genrated from a multi-view stereo system deployed to collect exclusive set of data on shallow cumulus clouds overhead.

54 ENVIRONMENTAL SCIENCES↗

pccpb.c1

Point Cloud of Cloud Points (PCCP) VAP provides a stereo reconstructed Point Cloud of Cloud Points (PCCP) that correspond to the x,y,z coordinates of cloud features detected from stereo image pairs generated from a multi-view stereo system deployed to collect exclusive set of data on shallow cumulus clouds overhead.

54 ENVIRONMENTAL SCIENCES↗

Advanced NDE research in electromagnetic, thermal, and coherent optics

A new inspection technology called magneto-optic/eddy current imaging was investigated. The magneto-optic imager makes readily visible irregularities and inconsistencies in airframe components. Other research observed in electromagnetics included (1) disbond detection via resonant modal analysis; (2) AC magnetic field frequency dependence of magnetoacoustic emission; and (3) multi-view magneto-optic imaging. Research observed in the thermal group included (1) thermographic detection and characterization of corrosion in aircraft aluminum; (2) a multipurpose infrared imaging system for thermoelastic stress detection; (3) thermal diffusivity imaging of stress induced damage in composites; and (4) detection and measurement of ice formation on the space shuttle main fuel tank. Research observed in the optics group included advancements in optical nondestructive evaluation (NDE).

Skinner, S. Ballou↗

Simulator Investigation of Pilot Aids for Helicopter Terminal Area Operations with One Engine Inoperative

Two recent piloted simulation experiments have investigated advanced display concepts applied to civil transport helicopter terminal area operations. Civil Category A helicopter operations apply to multi-engine helicopters wherein a safe recovery (land or fly out) is required in the event of a single engine failure. The investigation used the NASA Ames Research Center Vertical Motion Simulator, which has a full six degrees of freedom, to simulate the flight task as closely as possible. The goal of these experiments was to use advanced cockpit displays to improve flight safety and enhance the mission performance of Category A terminal area operations in confined areas. The first experiment investigated the use of military display formats to assist civil rotorcraft in performing a Category A takeoff in confined terminal areas. Specifically, it addressed how well a difficult hovering backup path could be followed using conventional instruments in comparison to panel mounted integrated displays. The hovering backup takeoff, which enables pilots to land back to the confined area pad in the event of an engine failure, was chosen since it is a difficult task to perform. Seven NASA and Army test pilots participated in the experiment. Evaluations, based on task performance and pilot workload, showed that an integrated display enabled the pilot to consistently achieve adequate or desired performance with reasonable pilot workload. Use of conventional instruments, however, frequently resulted in unacceptable performance (poor flight path tracking), higher pilot workload, and poor situational awareness. Although OEI landbacks were considered a visual task, the improved performance on the backup portion, in conjunction with increased situational awareness resulting from use of integrated displays, enabled the pilots to handle an engine failure and land back safely. In contrast, use of conventional instruments frequently led to excessive rates of sink at touchdown. A second simulation (in progress - July - August) is being conducted to investigate the use of advanced displays to perform vertical and short takeoffs and landings. One Engine Inoperative trajectories, which were optimized based on safety of flight restrictions, are utilized. Based on comments from the first experiment and further analytic development, appropriate fly out and approach guidance was added. Displays include conventional instruments with raw data, and the following integrated displays: multi-view and side-view hover displays based on the Apache Pilot Night Vision System, and variations of the pathway-in-the-sky displays with a flight-path-vector, a leader and flight director modifications. Panel mounted and head-up displays are being evaluated. Engine modifications have been incorporated to simulate 30 second and 2 minute contingency power ratings. Evaluations are based on task performance and pilot workload. NASA, Army, FAA, and industry test pilots participated. Details concerning the design, conduct, and the results of the experiment will be reported in the proposed paper.

Iseler, Laura↗

Photogrammetry Toolbox Reference Manual

Specialized photogrammetric and image processing MATLAB functions useful for wind tunnel and other ground-based testing of aerospace structures are described. These functions include single view and multi-view photogrammetric solutions, basic image processing to determine image coordinates, 2D and 3D coordinate transformations and least squares solutions, spatial and radiometric camera calibration, epipolar relations, and various supporting utility functions.

Liu, Tianshu↗

Neural Radiance Fields for Shadow Removal and Relighting of Satellite Imagery

Multi-view stereo photogrammetric techniques have been conventionally utilized to generate Global Digital Elevation Models (GDEM) of planetary and lunar surfaces. However, these methods, relying on conventional feature detectors, are often subject to inaccuracies caused by changes in lighting conditions, including diffuse reflection and harsh shading. This has limited the ability of these methods to accurately reconstruct shadowed regions in orbital imagery, such as highly shaded urban areas and the permanently shadowed regions (PSRs) located in the lunar surface, which are critical targets for NASA's Artemis program. Neural Radiance Fields (NeRFs) offer a novel solution to these limitations by breaking away from traditional photogrammetric assumptions of ridged, opaque surfaces. NeRFs are capable of reconstructing 3D objects with variably transmissive properties, such as fluids, PSRs, and reflective surfaces. In this work, we demonstrate the robustness of NeRFs in generating high-fidelity 3D models of terrain from highly shaded orbital imagery acquired from small satellites in low earth orbit (LEO). Our results showcase the ability of NeRF to overcome the limitations of traditional photogrammetric methods and provide a promising solution for reconstructing complex scenes in challenging lighting conditions.

Caleb Ashmore Adams↗

3D Cloud Tomography and Droplet Size Retrieval from Multi-Angle Polarimetric Imaging of Scattered Sunlight from Above

Tomography aims to recover a three-dimensional (3D) density map of a medium or an object. In medical imaging,it is extensively used for diagnostics via X-ray computed tomography (CT). We define and derive a tomographyof cloud droplet distributions via passive remote sensing. We use multi-view polarimetric images to fit a 3Dpolarized radiative transfer (RT) forward model. Our motivation is 3D volumetric probing of vertically-developedconvectively-driven clouds that are ill-served by current methods in operational passive remote sensing. Currenttechniques are indeed based on strictly 1D RT modeling and applied to a single cloudy pixel, where cloud geometrydefaults to that of a plane-parallel slab. Incident unpolarized sunlight, once scattered by cloud droplets, changesits polarization state according to droplet size. Therefore, polarimetric measurements in the rainbow and gloryangular regions can be used to infer the droplet size distribution. This work defines and derives a framework for afull 3D tomography of cloud droplets for both their mass concentration in space and their distribution across arange of sizes. This gridded 3D retrieval of key microphysical properties is made tractable by our novel approachthat involves a restructuring and partial linearization of an open-source polarized 3D RT code to accommodate aspecial two-step iterative optimization technique. Physically-realistic synthetic clouds are used to demonstrate themethodology with rigorous uncertainty quantification, while a real-world cloud imaged by AirMSPI is processedto illustrate the new remote sensing capability

Schechner, Yoav Y.↗

A Real2Sim Digital Twin Pipeline for Photorealistic Robot Simulation: Evaluating VLA Policy Deployment on a Bimanual Mobile Robot

Digital twins that are automatically constructed from robot sensor data offer a promising pathway for scalable Real2Sim and Sim2Real transfer. However, it remains an open question whether photorealistic reconstruction alone is sufficient to support reliable deployment of vision-language-action (VLA) policies. We present a generative-AI-assisted Real2Sim pipeline that generates simulation-ready digital twins from real-world RGB observations with minimal manual intervention. The pipeline uses prompted segmentation to isolate scene components and a generative 3D model to directly produce simulation assets, eliminating the need for traditional multi-view reconstruction or manual 3D modeling.\r\nTo evaluate simulation fidelity, we deploy and compare policies from two VLA models in both the real robot and the reconstructed\r\nsimulation under identical tasks and initial conditions. We compare joint-level action trajectories and analyze how divergence evolves over time in closed-loop execution. Although the reconstructed environments are visually accurate, we observe increasing trajectory divergence during closedloop operation. These results indicate that photorealistic reconstruction alone is insufficient to preserve closed-loop control behavior\r\nin VLA policies, particularly in contact-rich manipulation settings where small perceptual errors compound over time.

97 MATHEMATICS AND COMPUTING↗