MS-POFT: multiscale phase-orientation guided feature transform for multi-modal image matching
Not Available
Zhang, Zhizheng (ORCID:0009000494711555)↗
SEARCH · Search NASA
Results for “image feature matching”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A bi-channel aided stitching of atomic force microscopy images
Microscopy is an essential tool in scientific research, enabling the visualization of structures at micro- and nanoscale resolutions. However, the field of microscopy often encounters limitations in field-of-view (FOV), restricting the amount of sample that can be imaged in a single capture. To overcome this limitation, image stitching techniques have been developed to seamlessly merge multiple overlapping images into a single, high-resolution composite. The images collected from microscope need to be optimally stitched before accurate physical information can be extracted from post analysis. However, the existing stitching tools either struggle to stitch images together when the microscopy images are feature sparse or cannot address all the transformations of images when performing image stitching. To address these issues, we propose a bi-channel aided feature-based image stitching method and demonstrate its use on Atomic Force Microscopy (AFM) generated Pantoea sp. YR343 biofilm and PTO thin film sample images as experimental data. The topographical channel image of AFM data captures the morphological details of the sample, and a stitched topographical image is desired for researchers. We utilize the amplitude and phase channels of AFM data to maximize the matching features and to estimate the position of the original topographical images and show that the proposed bi-channel aided stitching method outperforms the traditional direct stitching approach in AFM topographical image stitching task. Here, we demonstrated the application on AFM, but similar approaches could be employed of optical microscopy with brightfield and fluorescence channels. We believe this proposed workflow can serve as a valuable augmentation strategy for microscopy image stitching tasks and will benefit the experimentalist to avoid erroneous analysis and discovery due to incorrect stitching.
Atomic force microscopy↗
Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection
Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.
46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
Automatic Image Point Matching
Sparse Image Point Matching (SIPM) is a foundational technology for photo triangulation, structure from motion (SfM), Simultaneous Location and Mapping (SLAM), and data fusion. The goal of the matching is to automatically generate sets of image coordinates that identify the same feature across images. Ideally, the process should be robust to lighting, scale, perspective, and modality changes. The scope of the image matching topic in the field of remote sensing (RS) is enormous because of the variety of collection platforms, modalities, sensor types, applications, and subjects. In this work, we report the history of and assess the state of the art of visible-spectrum (panchromatic and color) image matching of the Earth’s surface. Work specific to large-format images (LFI) (e.g., metric aerial cameras and Earth-observing satellites) will be highlighted. However, the state of the art in this century will mostly be traced through machine vision research and benchmarks because research specific to LFI is rare.
97 MATHEMATICS AND COMPUTING↗
INR-TEM: Robust cavity detection in multifocus TEM images via implicit neural representations
When characterizing materials using transmission electron microscopy (TEM) images, detecting and quantifying small features in microstructures, such as cavities, pose significant challenges. Off-the-shelf object detection models, including YOLOv8, show considerable performance degradation, particularly when images vary in resolution and the objects of interest possess a low percentage of the total image region of interest. In this study, we introduce a novel detection pipeline that incorporates an implicit neural representation (INR)-based detection method, INR-TEM, and two-modality imaging (e.g., under-focused and over-focused images typically acquired during materials characterization) to improve object detection performance. The INR-TEM method incorporates a pixel-wise prediction principle inspired by pixel-wise centerness weighting. INR-TEM demonstrates superior robustness to resolution variability, maintaining high detection accuracy even at low image resolutions compared to YOLOv8. To leverage INR-TEM effectively in real-world two-modality characterization applications, we further integrate a two-stage motion correction pipeline designed explicitly for aligning multifocus TEM images. The alignment process, comprising keypoint (based on scale-invariant feature transform, SIFT) and intensity matching, significantly mitigates the adverse effects of perceived motion-induced image degradation during through-focal TEM imaging, directly enhancing INR-TEM’s detection capability over conventional single-focus images. Our integrated INR-TEM cavity detection framework notably improves performance across various cavity sizes, outperforming off-the-shelf YOLOv8 detections that rely on a single image modality.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
You Only Look Once v5 and Multi-Template Matching for Small-Crack Defect Detection on Metal Surfaces
This paper compares the performance of Deep Learning (DL) and multi-template matching (MTM) models for detecting small defects. DL models extract distinguishing features of objects but require a large dataset of images. In contrast, alternative computer vision techniques like MTM need a relatively small dataset. The lack of large datasets for small metal-surface defects has inhibited the adoption of automation in small-defect detection in remanufacturing settings. This motivated this preliminary study to compare template-based approaches, like MTM, with feature-based approaches, such as DL models, for small-defect detection on an initial laboratory and remanufacturing industry dataset. This study used You Only Look Once v5 (YOLOv5) as the DL model and compared its performance against the MTM model for small-crack detection. The findings of our preliminary investigation are as follows: (i) YOLOv5 demonstrated higher performance than MTM in detecting small cracks; (ii) an extra-large variant of YOLOv5 outperformed a small-size variant; (iii) the size and object variety of the data are crucial in achieving robust pre-trained weights for use in transfer learning; and (iv) enhanced image resolution contributes to precise object detection.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Report on the AAPM grand challenge on deep generative modeling for learning medical image statistics
Abstract Background The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain‐relevant assessments via the analysis of relevant image statistics. Methods As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108 000 images of size 512 512. For the evaluation of submissions to the Challenge, an ensemble of 10 000 DGM‐generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance [FID]) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics, and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results Fifty‐eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top‐ranked submission employed a conditional latent diffusion model, whereas the joint runners‐up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID‐based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions This Grand Challenge highlighted the need for domain‐specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.
Radiology, Nuclear Medicine & Medical Imaging↗
Vehicular Re-Identification from Uncontrolled Multiple Views
Vehicle re-identification (re-ID) across disparate sensing modalities remains a fundamental challenge for transportation research. In this work, we introduce a deep multi-view vehicle re-ID framework that leverages Siamese networks to compare pairs of vehicle images and produce matching scores, enabling robust association across drastically different viewpoints such as those from UAVs, surveillance cameras, and ground sensors. The model exploits convolutional neural networks to learn features that remain discriminative under changes in angle, distance, and illumination, supporting more generalizable re-ID performance. As part of this effort, we also developed an automated pipeline to synchronize roadside and UAV video streams, producing a multi-perspective dataset that complements preexisting real collections and a synthetic dataset generated in this study. Together, these contributions advance the capability to re-identify vehicles across wide viewing baselines; establish a foundation for scalable, reproducible research in vehicle re-ID; and open pathways for future applications, such as inferring routine behaviors, movement patterns, and daily habits of the individual associated with the vehicle.
convolutional neural networks↗
A deep learning approach to fast analysis of collective Thomson scattering spectra
Fast analysis of collective Thomson scattering ion acoustic wave features using a deep convolutional neural network model is presented. The network was trained from spectra to predict the plasma parameters, including ion velocities, population fractions, and ion and electron temperatures. A fully kinetic particle-in-cell simulation was used to model a laboratory astrophysics experiment and simulate a diagnostic image of the ion acoustic wave feature. Network predictions were compared with Bayesian inference of the plasma model parameters for both the simulated and experimentally measured images. Both approaches were fairly accurate predicting the simulated image and the network predictions matched a good portion of the Bayesian results for the experimentally measured image. The Bayesian approach is more robust to noise and motivates future work to train deep learning models with realistic noise. The advantage of the deep learning model is making thousands of predictions in a few hundred milliseconds, compared to a few seconds to minutes per prediction for the optimization and Bayesian approaches presented here. The results demonstrate promising capabilities of deep learning models to analyze Thomson data orders of magnitude faster than conventional methods when using the neural network for standalone analysis. If more rigorous analysis is needed, neural network predictions can be used to quickly initialize other optimization methods and increase chances of success. This is especially useful when the dataset becomes very large or highly dimensional and manually refining initial conditions for the entire dataset are no longer tractable.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
SAM-I-Am: Semantic boosting for zero-shot atomic-scale electron micrograph segmentation
Image segmentation is a critical enabler for tasks ranging from medical diagnostics to autonomous driving. However, the correct segmentation semantics — where are boundaries located? what segments are logically similar? — change depending on the domain, such that state-of-the-art foundation models can generate meaningless and incorrect results. Moreover, in certain domains, fine-tuning and retraining techniques are infeasible: obtaining labels is costly and time-consuming; domain images (micrographs) can be exponentially diverse; and data sharing (for third-party retraining) is restricted. To enable rapid adaptation of the best segmentation technology, we propose the concept of semantic boosting: given a zero-shot foundation model, guide its segmentation and adjust results to match domain expectations. Here, we apply semantic boosting to the Segment Anything Model (SAM) to obtain microstructure segmentation for transmission electron microscopy. Our booster, SAM-I-Am, serves as a post-processing engine that extracts geometric and textural features of various intermediate masks to perform mask removal and mask merging operations. We demonstrate a zero-shot performance increase of (absolute) +21.35%, +12.6%, +5.27% in mean IoU, and a -9.91%, -18.42%, -4.06% drop in mean false positive masks across images of three difficulty classes over vanilla SAM (ViT-L).
36 MATERIALS SCIENCE↗
Utah FORGE: Well 16B(78)-32 Drill Core Fracture Analysis Images and Data
This dataset contains drilling core data from well 16B(78)-32, including PDF documents with flattened core images annotated by feature type and core interval, as well as spreadsheets detailing feature morphologies by depth, planar feature measurements, and planar feature orientations rotated to in situ conditions. Core was recovered from three intervals, one per stimulation stage, in the crystalline rocks affected by the stimulation of well 16A(78)-32. Seven core runs were conducted, yielding 135.8 feet of recovered core. Features in the core were categorized into planar fractures, semi-planar fractures, unbroken mineralized fractures, rough fractures, curviplanar fractures, concave-convex surfaces, and planar compositional features such as mylonite or dike-like structures. Planar features were measured while the core was positioned horizontally, with the core axis aligned to a downhole azimuth of 42 degrees. Planar core measurements from stimulations 2 and 3 that could be confidently correlated with FMI data were rotated to in situ orientations. This was done by rotating the planes along vertical and horizontal axes to match the azimuth and inclination data recorded in the directional survey of well 16B(78)-32, as well as applying an axial rotation to resemble the fracture orientations observed in the FMI log at corresponding depths. Coherent sets of planar fracture measurements were made by aligning the core within each 3-foot section of the dissected core barrel, and between adjacent 3-foot sections within a core run by matching rock fabrics, saw cuts and/or tool marks. Where coherent fracture measurements could not be made within a core run, data sets are denoted by a subscript (i.e. 2-Ta and 2-Tb both come from tangent core run number 2).
15 GEOTHERMAL ENERGY↗
A Computational Framework to design 3D stiffness gradient acoustic metamaterials for impedance matching
Acoustic waves play a crucial role in various applications, including medical imaging, non-destructive testing, and sonar systems. One of the significant challenges in these applications is impedance matching, which is essential for minimizing reflections and maximizing the transfer of acoustic energy between different media. Acoustic metamaterials offer a promising solution to this challenge. In addition to impedance control, gradient stiffness can enhance structural efficiency and enable spatial control of wave propagation, making it a valuable feature in acoustic metamaterial design. In this pa- per, we present our developed computational method to design 3D stiffness gradient acoustic metamaterials for impedance matching. The key steps in our approach include generating initial designs using a periodic covariance function to provide unit cells that are both periodic on the boundaries and randomly formed inside the unit cell. Furthermore, we integrated manufacturing constraints into the design process, ensuring that the structures are interconnected for fabrication. We propose two computational optimization algorithms: GenUnit, based on a non-dominated sorting genetic algorithm (NSGA-II), and MLMatch, which leverages differentiable machine learning. The two approaches are not separate contributions but complementary com- ponents of a unified framework. GenUnit requires no training data and directly interfaces with physics-based simulations, making it highly accurate but slower for large-scale exploration. In contrast, MLMatch is data-hungry during training but, once trained, enables near-instantaneous inference and broad design-space coverage. Together, they form a hybrid strategy: ML- Match rapidly explores the global design space, and GenUnit provides local refinement with high-fidelity accuracy. This balance between training cost, inference time, and precision is the motivation for including both methods in the same study. We applied this dual-algorithm framework to generate two metallic-based metamaterial designs that match the acoustic impedance of water while exhibiting a controlled gradient in stiffness (from stiff to soft). The stiffness gradient is particularly advantageous in applications where one side of the structure must interface with soft or sensitive surfaces, such as human tissue or delicate components. Here, this work paves the way for improved materials in various acoustic applications, particularly in ultrasound devices, by providing better impedance.
Metamaterial↗
Production of microchannel plates using nano-scale additive manufacturing
Microchannel plate (MCP) detectors have been the workhorse detector for many applications, including space borne ultra-violet imaging and spectrographic instruments. Recent advances in additive manufacturing (AM) have enabled fabrication of complex structures with nano-scale resolution facilitating the production of highly customizable MCPs. Using AM to produce MCPs potentially has many advantages over traditional fused glass substrates, including better material control (e.g., more robust glasses or ceramics), better control of microscopic features (e.g., unique pore geometries to improve performance), and better control of macroscopic features (e.g., printing precision curved surfaces for focal plane matching). Through a collaboration with industry, national laboratory, and university partners, small format microcapillary array substrates were produced using a standard polymer photoresin. In conclusion, these substrates were functionalized using atomic layer deposition and their performance was compared to current state-of-the-art Pb-glass and borosilicate-glass MCPs.
36 MATERIALS SCIENCE↗
In-orbit performance of the soft X-ray imaging telescope Xtend aboard XRISM
Here, we present a summary of the in-orbit performance of the soft X-ray imaging telescope Xtend onboard the X-Ray Imaging and Spectroscopy Mission (XRISM), based on in-flight observation data, including first-light celestial objects, calibration sources, and results from the cross-calibration campaign with other currently operating X-ray observatories. XRISM/Xtend has a large field of view of ${38{^{\prime }_{.}}5}$ $\times$ ${38{^{\prime }_{.}}5}$, covering an energy range of 0.4–13 keV, as demonstrated by the first-light observation of the galaxy cluster Abell 2319. It also features an energy resolution of 170–180 eV at 6 keV, which meets the mission requirement and enables us to resolve He-like and H-like Fe K$\alpha$ lines. Throughout the observation during the performance verification phase, we confirm that two issues identified in the Soft X-ray Imager (SXI) onboard the previous Hitomi mission—light leakage and crosstalk events—are addressed and suppressed in the case of Xtend. A joint cross-calibration observation of the bright quasar 3C 273 results in an effective area measured to be $\sim$420 cm$^{2}$ at1.5 keV and $\sim$310 cm$^{2}$ at 6.0 keV, which matches values obtained in ground tests. We also continuously monitor the health of Xtend by analyzing overclocking data, calibration source spectra, and day-Earth observations; the readout noise is stable and low, and contamination is negligible even one year after launch. A low background level compared with other major X-ray instruments onboard satellites, combined with the largest grasp ($\Omega _{\rm eff}\sim 60$ cm$^2$ deg$^2$) of Xtend, will not only support Resolve analysis, but also enable significant scientific results on its own. This includes near-future follow-up observations and transient searches in the context of time-domain and multi-messenger astrophysics.
instrumentation: detectors: Xtend↗
Multiscale maps of Active Layer Depth for Teller site Mile Marker 27 and Kougarok Mile Marker 80, Seward Peninsula, AK
Remote sensing maps of active layer depth derived from Unmanned Areal System (UAS) data. The UAS datasets were stepwise scaled until matching the AVIRIS-NG (Airborne Visible / Infrared Imaging Spectrometer - Next Generation) and Sentinel-2 spatial resolutions. Using the field observed Active Layer Depth (ALD) measurement in combination with spectral and topographic predictors derivatives from DJI UAS imagery, we used a spatially explicit RF regression model to predict and map ALD across our study landscapes. This package includes maps for Next-Generation Ecosystem Experiment Arctic (NGEE Arctic)’s Teller Mile Marker (MM) 27, and Kougarok MM80 (aka Mile 80) watersheds. The field, map data, and metadata are provided as geoTIF and text (*.csv) formats. These datasets are provided in support of Hantson et al., 2024 (accepted) “Scaling Arctic landscape and permafrost features improves active layer depth modeling”
54 ENVIRONMENTAL SCIENCES↗
Path-Integrated X-Ray Digital Image Correlation using Synthetic Reference Images
X-rays can provide images when an object is visibly obstructed, allowing for motion measurements via x-ray digital image correlation (DIC). However, x-ray images are path-integrated and contain data for all objects between the source and detector. If multiple objects are present in the x-ray path, conventional DIC algorithms may fail to correlate the x-ray images. A new DIC algorithm called path-integrated (PI)-DIC addresses this issue by reformulating the matching criterion for DIC to account for multiple, independently-moving objects. PI-DIC requires a set of reference x-ray images of each independent object. However, due to experimental constraints, such reference images might not be obtainable from the experiment. Here, this work focuses on the reliability of synthetically-generated reference images, in such cases. A simplified exemplar is used for demonstration purposes, consisting of two aluminum plates with tantalum x-ray DIC patterns undergoing independent rigid translations. Synthetic reference images based on the “as-designed” DIC patterns were generated. However, PI-DIC with the synthetic images suffered some biases due to manufacturing defects of the patterns. A systematic study of seven identified defect types found that an incorrect feature diameter was the most influential defect. Synthetic images were re-generated with the corrected feature diameter, and PI-DIC errors were improved by a factor of 3-4. Final biases ranged from 0.00-0.04 px, and standard uncertainties ranged from 0.06-0.11 px. In conclusion, PI-DIC accurately measured the independent displacement of two plates from a single series of path-integrated x-ray images using synthetically-generated reference images, and the methods and conclusions derived here can be extended to more generalized cases involving stereo PI-DIC for arbitrary specimen geometry and motion. This work thus extends the application space of x-ray imaging for full-field DIC measurements of multiple surfaces or objects in extreme environments where optical DIC is not possible.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Mapping Glacier Basal Sliding Applying Machine Learning
During the RESOLVE project (“High-resolution imaging in subsurface geophysics: development of a multi-instrument platform for interdisciplinary research”), continuous surface displacement and seismic array observations were obtained on Glacier d’Argentière in the French Alps for 35 days in May 2018. The data set is used to perform a detailed study of targeted processes within the highly dynamic cryospheric environment. In particular, the physical processes controlling glacial basal motion are poorly understood and remain challenging to observe directly. Especially in the Alpine region for temperate based glaciers where the ice rapidly responds to changing climatic conditions and thus, processes are strongly intermittent in time and heterogeneous in space. Spatially dense seismic and Global Positioning System (GPS) measurements are analyzed applying machine learning to gain insight into the processes controlling glacial motions of Glacier d’Argentière. Using multiple bandpass-filtered copies of the continuous seismic waveforms, we compute energy-based features, develop a matched field beamforming catalog and include meteorological observations. Features describing the data are analyzed with a gradient boosting decision tree model to directly estimate the GPS displacements from the seismic noise. We posit that features of the seismic noise provide direct access to the dominant parameters that drive displacement on the highly variable and unsteady surface of the glacier. The machine learning model infers daily fluctuations and longer term trends. The results show on-ice displacement rates are strongly modulated by activity at the base of the glacier. The techniques presented provide a new approach to study glacial basal sliding and discover its full complexity.
58 GEOSCIENCES↗
Direct Observation of the ππ* to nπ* Transition in 2-Thiouracil via Time-Resolved NEXAFS Spectroscopy
The photophysics of nucleobases has been the subject of both theoretical and experimental studies over the past decades due to the challenges posed by resolving the steps of their radiationless relaxation dynamics, which cannot be described in the framework of the Born–Oppenheimer approximation (BOA). In this context, the ultrafast dynamics of 2-thiouracil has been investigated with a time-resolved NEXAFS study at the Free Electron Laser FLASH. Near Edge X-ray Absorption Fine Structure spectroscopy (NEXAFS) can be used to observe electronic transitions in ultrafast molecular relaxation. We performed time-resolved UV-pump/X-ray probe absorption measurements at the sulfur 2s (L1) and 2p (L2/3) edges. We are able to identify absorption features corresponding to the S2 (ππ*) and S1 (nπ*) electronic states. We observe a delay of 102 ± 11 fs in the population of the nπ* state with respect to the initial optical excitation and interpret the delay as the time scale for the S2 → S1 internal conversion. We furthermore identify oscillations in the absorption signal that match a similar observation in our previous X-ray photoelectron spectroscopy study on the same molecule.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗