Search NASA⌕ Search

SEARCH · Search NASA

Results for “image retrieval”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Fast Local Spatial Verification for Feature-Agnostic Large-Scale Image Retrieval

Images from social media can reflect diverse viewpoints, heated arguments, and expressions of creativity, adding new complexity to retrieval tasks. Researchers working on Content-Based Image Retrieval (CBIR) have traditionally tuned their algorithms to match filtered results with user search intent. However, we are now bombarded with composite images of unknown origin, authenticity, and even meaning. With such uncertainty, users may not have an initial idea of what the search query results should look like. For instance, hidden people, spliced objects, and subtly altered scenes can be difficult for a user to detect initially in a meme image, but may contribute significantly to its composition. It is pertinent to design systems that retrieve images with these nuanced relationships in addition to providing more traditional results, such as duplicates and near-duplicates — and to do so with enough efficiency at large scale. In this work, we propose a new approach for spatial verification that aims at modeling object-level regions using image keypoints retrieved from an image index, which is then used to accurately weight small contributing objects within the results, without the need for costly object detection steps. We call this method the Objects in Scene to Objects in Scene (OS2OS) score, and it is optimized for fast matrix operations, which can run quickly on either CPUs or GPUs. It performs comparably to state-of-the-art methods on classic CBIR problems (Oxford 5K, Paris 6K, and Google-Landmarks), and outperforms them in emerging retrieval tasks such as image composite matching in the NIST MFC2018 dataset and meme-style imagery from Reddit.

42 ENGINEERING↗
DOE OSTI2024

Retrieving images from tarnished daguerreotypes using X-ray fluorescence imaging with an X-ray micro beam with tunable energy

We report recent observations using a synchrotron X-ray micro-beam to retrieve images from tarnished 19th century daguerreotypes. We confirm that high quality image can always be retrieved from tarnished plates using Hg L α XRF as long as the bulk of the image particles and their distribution remains intact. We also report results from using tunable tender X-rays (2 - 7 keV) to conduct imaging in high vacuum at energy above the Ag L-edge and the Hg M-edge, extracting images using Ag L α and Hg M α , respectively among others (e.g., S to track corrosion). Images obtained with the surface sensitive total electron yield (TEY) and the bulk sensitive fluorescence yield (FLY) as well as corresponding micro-XANES are reported. Flux tolerance to high intensity X-beam is also explored. These results and their implications for cultural heritage research are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Retrieving Tarnished Daguerreotype Content Using X-ray Fluorescence Imaging—Recent Observations on the Effect of Chemical and Electrochemical Cleaning Methods

We report a study on the effect of chemical and electrochemical cleaning of tarnished daguerreotypes observed using X-ray fluorescence (XRF) microscopy with a micro-focussed X-ray beam from a synchrotron source. It has been found that, while both techniques result in some success depending on the condition of the plate and the experimental parameters (chemical concentration, voltage, current, etc.) the effect varies, and cleaning is often incomplete. The XRF images using Hg Lα,β at an excitation energy just above the L3 edge threshold produce fine images, regardless of the treatment. This finding confirms previous observations that if the bulk of the image particles remains intact, the surface tarnish has little effect on the quality of the original daguerreotype image retrievable from XRF.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Exploring Mission Design for Imaging Spectroscopy Retrievals for Land and Aquatic Ecosystems

The retrieval algorithms used for optical remote sensing satellite data to estimate Earth's geophysical properties have specific requirements for spatial resolution, temporal revisit, spectral range and resolution, and instrument signal-to-noise ratio (SNR) performance to meet biogeoscience objectives. Studies to estimate surface properties from hyperspectral data use a range of algorithms sensitive to various sources of spectroscopic uncertainty, which are in turn influenced by mission architecture choices. Retrieval algorithms vary across scientific fields and may be more or less sensitive to mission architecture choices that affect spectral, spatial, or temporal resolutions and spectrometer SNR. We used representative remote sensing algorithms across terrestrial and aquatic study domains to inform aspects of mission design that are most important for impacting accuracy in each scientific area. We simulated the propagation of uncertainties in the retrieval process including the effects of different instrument configuration choices. We found that retrieval accuracy and information content degrade consistently at >10 nm spectral resolution, >30 m spatial resolution, and >8-day revisit. In these studies, the noise reduction associated with lower spatial resolution improved accuracy vis à vis high spatial resolution measurements. The interplay between spatial resolution, temporal revisit, and SNR can be quantitatively assessed for imaging spectroscopy missions and used to identify key components of algorithm performance and mission observing criteria.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Fresnel diffraction imaging of surface nanostructure using coherent resonant x-ray scattering

We investigated surface nanostructures on an antiferromagnet MnBi 2 Te 4 using a novel imaging technique, direct (real)-space and real time coherent x-ray imaging (direct-CXI). This technique has provided new insights into antiferromagnetic textures, including the formation of anti-phase antiferromagnetic (AFM) domains and thermal dynamics of AFM domains and domain walls. While this method produces real-space images of AFM textures without requiring a complex imaging retrieval process, its underlying imaging mechanism has not been fully understood, limiting a deep understanding of AFM textures and the information they contain. By investigating the well-defined structural characteristics of the nanostructures fabricated on MnBi 2 Te 4 ⁠, we elucidate the imaging principle of this novel technique. We find that the observed images can be well explained by the Fresnel diffraction integral. Using a simple model from classical optics, our calculations successfully reproduce the experimentally observed images of the nanostructures. This demonstrates that direct-CXI not only provides straightforward real-space imaging but also contains phase information through its Fresnel diffraction integral.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Expert evaluation of LLM world models: A high-T c superconductivity case study

Large Language Models (LLMs) show great promise as a powerful tool for scientific literature exploration. However, their effectiveness in providing scientifically accurate and comprehensive answers to complex questions within specialized domains remains an active area of research. Using the field of high-temperature cuprates as an exemplar, we evaluate the ability of LLM systems to understand the literature at the level of an expert. We construct an expert-curated database of 1,726 scientific papers that covers the history of the field, and a set of 67 expert-formulated questions that probe deep understanding of the literature. We then evaluate six different LLM-based systems for answering these questions, including both commercially available closed models and a custom retrieval-augmented generation (RAG) system capable of retrieving images alongside text. Experts then evaluate the answers of these systems against a rubric that assesses balanced perspectives, factual comprehensiveness, succinctness, and evidentiary support. Among the six systems, two using RAG on curated literature outperformed existing closed models across key metrics, particularly in providing comprehensive and well-supported answers. We discuss promising aspects of LLM performances as well as critical short-comings of all the models. The set of expert-formulated questions and the rubric will be valuable for assessing expert level performance of LLM based reasoning systems.

36 MATERIALS SCIENCE↗
DOE OSTI2023

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Abstract Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Atomic resolution coherent x-ray imaging with physics-based phase retrieval

Coherent x-ray imaging and scattering from accelerator based sources such as synchrotrons continue to impact biology, medicine, technology, and materials science. Many synchrotrons around the world are currently undergoing major upgrades to increase their available coherent x-ray flux by approximately two orders of magnitude. The improvement of synchrotrons may enable imaging of materials in operando at the atomic scale which may revolutionize battery and catalysis technologies. Current algorithms used for phase retrieval in coherent x-ray imaging are based on the projection onto sets method. These traditional iterative phase retrieval methods will become more computationally expensive as they push towards atomic resolution and may struggle to converge. Additionally, these methods do not incorporate physical information that may additionally constrain the solution. In this work, we present an algorithm which incorporates molecular dynamics into Bragg coherent diffraction imaging (BCDI). This algorithm, which we call PRAMMol (Phase Retrieval with Atomic Modeling and Molecular Dynamics) combines statistical techniques with molecular dynamics to solve the phase retrieval problem. We present several examples where our algorithm is applied to simulated coherent diffraction from 3D crystals and show convergence to the correct solution at the atomic scale.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Two-dimensional retrieval methods for ultrafast imaging of molecular structure using laser-induced electron diffraction

Molecular structural retrieval based on electron diffraction has been proposed to determine the atomic positions of molecules with sub-angstrom spatial and femtosecond temporal resolutions. Given its success on small molecular systems, in this work, we point out that the accuracy of structure retrieval is constrained by the availability of a wide range of experimental data in the momentum space in all molecular systems. To mitigate the limitations, for laser-induced electron diffraction, here we retrieve molecular structures using two-dimensional (energy and angle) electron momentum spectra in the laboratory frame for a number of small molecular systems, which have previously been studied with 1D methods. Compared to the conventional single-energy or single-angle analysis, our 2D methods effectively expand the momentum range of the measured data. Besides utilization of the 2D data, two complementary methods are developed for consistency check on the retrieved results. Further, the 2D nature of our methods also offers a way of estimating the error from retrieval, which has never been explored before. Comparing with results from prior experiments, our findings show evidence that our 2D methods outperform the conventional 1D methods. Paving the way to the retrieval of large molecular systems, in which their tunneling ionization rates are challenging to obtain, we estimate the error of using the isotropic model in place of including the orientation-dependent ionization rate.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2022

AutoPhaseNN: unsupervised physics-aware deep learning of 3D nanoscale Bragg coherent diffraction imaging

Abstract The problem of phase retrieval underlies various imaging methods from astronomy to nanoscale imaging. Traditional phase retrieval methods are iterative and are therefore computationally expensive. Deep learning (DL) models have been developed to either provide learned priors or completely replace phase retrieval. However, such models require vast amounts of labeled data, which can only be obtained through simulation or performing computationally prohibitive phase retrieval on experimental datasets. Using 3D X-ray Bragg coherent diffraction imaging (BCDI) as a representative technique, we demonstrate AutoPhaseNN, a DL-based approach which learns to solve the phase problem without labeled data. By incorporating the imaging physics into the DL model during training, AutoPhaseNN learns to invert 3D BCDI data in a single shot without ever being shown real space images. Once trained, AutoPhaseNN can be effectively used in the 3D BCDI data inversion about 100× faster than iterative phase retrieval methods while providing comparable image quality.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Generative modeling enables molecular structure retrieval from Coulomb explosion imaging

Capturing the structural changes that molecules undergo during chemical reactions in real space and time is a long-standing dream and an essential prerequisite for understanding and ultimately controlling femtochemistry. A key approach to tackle this challenging task is Coulomb explosion imaging, which benefited decisively from recently emerging high-repetition-rate X-ray free-electron laser sources. With this technique, information on the molecular structure is inferred from the momentum distributions of the ions produced by the rapid Coulomb explosion of molecules. Retrieving molecular structures from these distributions poses a highly non-linear inverse problem that remains unsolved for molecules consisting of more than a few atoms. Here, we address this challenge using a diffusion-based Transformer neural network. We show that the network reconstructs unknown molecular geometries from ion-momentum distributions with a mean absolute error below one Bohr radius, which is half the length of a typical chemical bond.

Artificial Intelligence (cs.AI)↗
DOE OSTI2026

Fidelity-preserving enhancement of ptychography with foundational text-to-image models

Ptychographic phase retrieval enables high-resolution imaging of complex samples but often suffers from artifacts such as grid pathology and multislice crosstalk, which degrade reconstructed images. We propose a plug-and-play (PnP) framework that integrates physics model-based phase retrieval with text-guided image editing using foundational diffusion models. By employing the alternating direction method of multipliers, our approach ensures consensus between data fidelity and artifact removal subproblems, maintaining physical consistency while enhancing image quality. Artifact removal is achieved using a text-guided diffusion image editing method (LEDITS++) with a pre-trained foundational diffusion model, allowing users to specify artifacts for removal in natural language. Demonstrations on simulated and experimental datasets show significant improvements in artifact suppression and structural fidelity, validated by metrics such as peak signal-to-noise ratio and diffraction pattern consistency. This work highlights the combination of text-guided generative models and model-based phase retrieval algorithms as a transferable and fidelity-preserving method for high-quality diffraction imaging.

image editing↗
DOE OSTI2023

Reconstruction of simulated electrostatic potentials by automatic differentiation-based phase retrieval in electron microscopy imaging

Measuring interfacial electrostatic potentials is vital to understanding many fundamental materials properties. A variety of TEM methods exist for measuring electric potentials from the phase shift produced on an electron wave as it passes through a sample. However, most are either experimentally challenging or poorly suited to resolving nanoscale features. Here, we demonstrate the viability of a simple, automatic differentiation-based exit wave reconstruction from a focal series of images to accurately measure nanoscale electric potentials. Here, the analysis suggests that under optimal measurement conditions, electric potentials can be resolved to less than 0.06 V in magnitude and less than 1 nm in spatial extent.

36 MATERIALS SCIENCE↗
DOE OSTI2021

High-Resolution Imaging of Unstained Polymer Materials

Electron microscopy has played an important role in polymer characterization. Traditionally, electron diffraction is used to study crystalline polymers while transmission electron microscopy is used to study microphase separation in stained block copolymers and other multiphase systems. We describe developments that eliminate the barrier between these two approaches - it is now possible to image polymer crystals with atomic resolution. The focus of this Review is on high-resolution imaging (30 Å and smaller) of unstained polymers. Recent advances in hardware allow for capturing numerous (as many as 105) low-dose images from an unperturbed specimen; beam damage is a significant barrier to high-resolution electron microscopy of polymers. Machine-learning-based software is then used to sort and average the images to retrieve pristine structural information from a collection of noisy images. Acknowledging the heterogeneity in polymer samples prior to averaging is essential. Molecular conformations in a wide range of amphiphilic block copolymers, polymerized ionic liquids, and conjugated polymers can be gleaned from two-dimensional projections (2D), three-dimensional (3D) tomograms, and four-dimensional (4D) scanning transmission electron microscopy (STEM) data sets where 2D diffraction patterns are taken as a function of position. Some methods such as phase contrast STEM have been used to image closely related materials such as metal-organic frameworks but not polymers. With improvements in hardware and software, such methods may soon be applied to polymers. Our goal is to provide a comprehensive understanding of the strategies toward the high-resolution imaging of radiation sensitive polymer materials at different length scales.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Quantitative phase retrieval and characterization of magnetic nanostructures via Lorentz (scanning) transmission electron microscopy

Magnetic materials phase reconstruction using Lorentz transmission electron microscopy (LTEM) measurements have traditionally been achieved using longstanding methods such as off-axis holography (OAH) fast-Fourier transform technique and the transport-of-intensity equation (TIE). The increase in access to processing power alongside the development of advanced algorithms have allowed for phase retrieval of nanoscale magnetic materials with greater efficacy and resolution. Specifically, reverse-mode automatic differentiation (RMAD) and the extended electron ptychography iterative engine (ePIE) are two recent developments of phase retrieval that can be applied to analyzing micro-to-nano- scale magnetic materials. This work evaluates phase retrieval using TIE, RMAD, and ePIE in simulations of Permalloy (Ni 80 Fe 20 ) nanoscale islands, or nanomagnets. Extending beyond simulations, we demonstrate total phase retrieval and image reconstructions of a NiFe nanowire using OAH and RMAD in LTEM and ePIE in Lorentz-mode-4D scanning transmission electron microscopy experiments and determine the saturation magnetization through corroborations with micromagnetic modeling. Finally, we demonstrate the efficacy of these methods in retrieving the total phase and highlight its use in characterizing and analyzing the proximity effect of the magnetic nanostructures.

Lorentz transmission electron microscopy↗
DOE OSTI2025

DeepAndes: A Self-Supervised Vision Foundation Model for Multispectral Remote Sensing Imagery of the Andes

By mapping sites at large scales usingremotely sensed data, archaeologists can generate unique insights into long-term demographic trends, interregional social networks, and human adaptations in the past. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. In addition, while recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multispectral satellite imagery, such as the eight-band data used in our study. In this article, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multispectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for eight-band multispectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pretrained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pretraining in archaeological remote sensing.

Guo, Junlin [Vanderbilt Univ., Nashville, TN (Unit↗