SEARCH · Search NASA

Results for “Computer Vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2026

A Smart Vision-Aided RICH (Robotic Interface Control and Handling) System for VULCAN

High-flux neutron beams and high-efficiency detectors enable rapid neutron diffraction measurements at the Engineering Materials Diffractometer (VULCAN) at the Spallation Neutron Source (SNS), Oak Ridge National Laboratory (ORNL). To optimize beam time utilization, efficient sample exchange, alignment, and automated measurements are essential. Recent advances in artificial intelligence (AI) have expanded the capabilities of robotic systems. Here, we report the development of a Robotic Interactive Control and Handling (RICH) system for sample handling at VULCAN, designed to support high-throughput experiments and reduce overhead time. The RICH system employs a six-axis desktop robot integrated with AI-based computer vision models capable of recognizing and localizing samples in real time from instrument and depth-resolving cameras. Vision algorithms combine these detections to align samples with designated measurement positions or place them within complex sample environments such as furnaces. This integration of machine learning-assisted vision with robotic handling demonstrates the feasibility of autonomous sample detection and preparation, offering a pathway toward fully unmanned neutron scattering experiments.

automation↗

NASA NTRS2008

Reduced-Order Kalman Filtering for Processing Relative Measurements

A study in Kalman-filter theory has led to a method of processing relative measurements to estimate the current state of a physical system, using less computation than has previously been thought necessary. As used here, relative measurements signifies measurements that yield information on the relationship between a later and an earlier state of the system. An important example of relative measurements arises in computer vision: Information on relative motion is extracted by comparing images taken at two different times. Relative measurements do not directly fit into standard Kalman filter theory, in which measurements are restricted to those indicative of only the current state of the system. One approach heretofore followed in utilizing relative measurements in Kalman filtering, denoted state augmentation, involves augmenting the state of the system at the earlier of two time instants and then propagating the state to the later time instant.While state augmentation is conceptually simple, it can also be computationally prohibitive because it doubles the number of states in the Kalman filter. When processing a relative measurement, if one were to follow the state-augmentation approach as practiced heretofore, one would find it necessary to propagate the full augmented state Kalman filter from the earlier time to the later time and then select out the reduced-order components. The main result of the study reported here is proof of a property called reduced-order equivalence (ROE). The main consequence of ROE is that it is not necessary to augment with the full state, but, rather, only the portion of the state that is explicitly used in the partial relative measurement. In other words, it suffices to select the reduced-order components first and then propagate the partial augmented state Kalman filter from the earlier time to the later time; the amount of computation needed to do this can be substantially less than that needed for propagating the full augmented Kalman state filter.

Bayard, David S.↗

NASA NTRS2011

Image Registration for Stability Testing of MEMS

Image registration, or alignment of two or more images covering the same scenes or objects, is of great interest in many disciplines such as remote sensing, medical imaging. astronomy, and computer vision. In this paper, we introduce a new application of image registration algorithms. We demonstrate how through a wavelet based image registration algorithm, engineers can evaluate stability of Micro-Electro-Mechanical Systems (MEMS). In particular, we applied image registration algorithms to assess alignment stability of the MicroShutters Subsystem (MSS) of the Near Infrared Spectrograph (NIRSpec) instrument of the James Webb Space Telescope (JWST). This work introduces a new methodology for evaluating stability of MEMS devices to engineers as well as a new application of image registration algorithms to computer scientists.

Memarsadeghi, Nargess↗

NASA NTRS2016

Comparison of Cross Correlation and Optical Flow Methods for Processing Retroreflective and Natural Background BOS Data

Background oriented schlieren images have historically been generated by calculating the observed pixel displacement between an image pair using normalized cross-correlation methods. This work uses optical flow data reduction methods to solve the displacement fields. A well established method used in the computer vision community, optical flow is the apparent brightness motion in an image sequence. The regularization method of Horn and Schunck is used to create schlieren images using two data sets: a supersonic shockjet plume interaction at the NASA Ames Unitary Plan Wind Tunnel, and a transonic flight test of a T-38 using a naturally occurring background performed in conjunction with NASA Ames and Armstrong research centers. Results are presented and contrasted with those using normalized cross-correlation methods. The optical flow images are found to provided significantly more detail at a decreased computational time.

schlieren↗

NASA NTRS1991

Passive range estimation for rotorcraft low-altitude flight

The automation of rotorcraft low-altitude flight presents challenging problems in control, computer vision and image understanding. A critical element in this problem is the ability to detect and locate obstacles, using on-board sensors, and modify the nominal trajectory. This requirement is also necessary for the safe landing of an autonomous lander on Mars. This paper examines some of the issues in the location of objects using a sequence of images from a passive sensor, and describes a Kalman filter approach to estimate the range to obstacles. The Kalman filter is also used to track features in the images leading to a significant reduction of search effort in the feature extraction step of the algorithm. The method can compute range for both straight line and curvilinear motion of the sensor. A laboratory experiment was designed to acquire a sequence of images along with sensor motion parameters under conditions similar to helicopter flight. Range estimation results using this imagery are presented.

Sridhar, B.↗

NASA NTRS1993

Passive range estimation for rotorcraft low-altitude flight

Sridhar, B.↗

NASA NTRS1993

Passive range estimation for rotorcraft low-altitude flight

Sridhar, B.↗

NASA NTRS2001

Where to look? Automating attending behaviors of virtual human characters

This research proposes a computational framework for generating visual attending behavior in an embodied simulated human agent. Such behaviors directly control eye and head motions, and guide other actions such as locomotion and reach. The implementation of these concepts, referred to as the AVA, draws on empirical and qualitative observations known from psychology, human factors and computer vision. Deliberate behaviors, the analogs of scanpaths in visual psychology, compete with involuntary attention capture and lapses into idling or free viewing. Insights provided by implementing this framework are: a defined set of parameters that impact the observable effects of attention, a defined vocabulary of looking behaviors for certain motor and cognitive activity, a defined hierarchy of three levels of eye behavior (endogenous, exogenous and idling) and a proposed method of how these types interact.

NASA Discipline Space Human Factors↗

NASA NTRSDate not supplied

Automated Registration of Multi-Mode Nondestructive Evaluation Data

Registration techniques play a central role in applications of image processing to computer vision, medical imaging, and automatic target tracking. Feature-based techniques such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF) are commonly used to register images derived from a single modality. However, SIFT and SURF struggle to register images from different modalities because the features tend to manifest rather differently and at sometimes very different length-scales. The most successful methods that have been developed to register multi-modal data use information-theoretic approaches. These methods play a key part in nondestructive evaluation scenarios where data that is collected by sensors of different modalities must be registered to be fused. In this paper, automated registration based on normalized mutual information is applied to align data derived from ultrasonic and radiographic inspections of (i) additively manufactured titanium alloy test coupons, and (ii) thin, lithium metal pouch-cell batteries. The quality of the registration is quantified in terms of computational resources and spatial accuracy. In the first case the X-ray computed tomography (XCT) data is captured on a region corresponding to a small subset of the ultrasonic data, while in the case of the lithium batteries the digital radiography (DR) captures a larger region of interest than the ultrasonic data. In both cases the radiographic data resolution is much higher than for ultrasound, but interestingly, in both cases the accuracy of the registration is approximately equal to two-to-three-pixel lengths in the ultrasonic images.

Nondestructive Evaluation↗

NASA NTRS1991

The Space Station Freedom Flight Telerobotic Servicer: the design and evolution of a dexterous space robot

The Flight Telerobotic Servicer (FTS) Project at the Goddard Space Flight Center is developing an advanced telerobotic system to assist in and reduce crew extravehicular activity (EVA) for Space Station) Freedom (SSF). The FTS will provide a telerobotic capability to the Freedom Station in the early assembly phases of the program and will be employed for assembly, maintenance, and inspection applications throughout the lifetime of the space station. Appropriately configured elements of the FTS will also be employed for robotic manipulation in remote satellite servicing applications and possibly the Lunar/Mars Program. In mid-1989, the FTS entered the flight system design and implementation phase (Phase C/D) of development with the signing of the FTS prime contract with Martin Marietta Astronautics Group in Denver, Colorado. The basic FTS design is now established and can be reported on in some detail. This paper will describe the FTS flight system design and the rationale for the specific design approaches and component selections. The current state of space technology and the nature of the FTS task dictate that the FTS be designed with sophisticated teleoperation capabilities for its initial primary operating mode. However, there are technologies, such as advanced computer vision and autonomous planning techniques currently in research and advanced development phases which would greatly enhance the FTS capabilities to perform autonomously in less structured work environments. Therefore, a specific requirement on the initial FTS design is that it has the capability to evolve as new technology becomes available. This paper will describe the FTS design approach for evolution to more autonomous capabilities. Some specific task applications of the FTS and partial automation approaches of these tasks will also be discussed in this paper.

Task Performance and Analysis↗

NASA NTRS1992

Model based estimation of image depth and displacement

Passive depth and displacement map determinations have become an important part of computer vision processing. Applications that make use of this type of information include autonomous navigation, robotic assembly, image sequence compression, structure identification, and 3-D motion estimation. With the reliance of such systems on visual image characteristics, a need to overcome image degradations, such as random image-capture noise, motion, and quantization effects, is clearly necessary. Many depth and displacement estimation algorithms also introduce additional distortions due to the gradient operations performed on the noisy intensity images. These degradations can limit the accuracy and reliability of the displacement or depth information extracted from such sequences. Recognizing the previously stated conditions, a new method to model and estimate a restored depth or displacement field is presented. Once a model has been established, the field can be filtered using currently established multidimensional algorithms. In particular, the reduced order model Kalman filter (ROMKF), which has been shown to be an effective tool in the reduction of image intensity distortions, was applied to the computed displacement fields. Results of the application of this model show significant improvements on the restored field. Previous attempts at restoring the depth or displacement fields assumed homogeneous characteristics which resulted in the smoothing of discontinuities. In these situations, edges were lost. An adaptive model parameter selection method is provided that maintains sharp edge boundaries in the restored field. This has been successfully applied to images representative of robotic scenarios. In order to accommodate image sequences, the standard 2-D ROMKF model is extended into 3-D by the incorporation of a deterministic component based on previously restored fields. The inclusion of past depth and displacement fields allows a means of incorporating the temporal information into the restoration process. A summary on the conditions that indicate which type of filtering should be applied to a field is provided.

Damour, Kevin T.↗

DOE OSTI2024

Open Call LDRD: Physically Informed Autoencoders for Galactic Redshift Regression

Physical constraints have been suggested to make neural network models more generalizable, act scientifically plausible, and be more data-efficient over unconstrained baselines. In this report, we present preliminary work on evaluating the effects of adding soft physical constraints to computer vision neural networks trained to estimate the conditional density of redshift on input galaxy images for the Sloan Digital Sky Survey. We introduce physically motivated soft constraint terms that are not implemented with differential or integral operators. We frame this work as a simple ablation study where the effect of including soft physical constraints is compared to an unconstrained baseline. We compare networks using standard point estimate metrics for photometric redshift estimation, as well as metrics to evaluate how faithful our conditional density estimate represents the probability over the ensemble of our test dataset. We find no evidence that the implemented soft physical constraints are more effective regularizers than augmentation.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting

Abstract Drug response prediction (DRP) methods tackle the complex task of associating the effectiveness of small molecules with the specific genetic makeup of the patient. Anti-cancer DRP is a particularly challenging task requiring costly experiments as underlying pathogenic mechanisms are broad and associated with multiple genomic pathways. The scientific community has exerted significant efforts to generate public drug screening datasets, giving a path to various machine learning models that attempt to reason over complex data space of small compounds and biological characteristics of tumors. However, the data depth is still lacking compared to application domains like computer vision or natural language processing domains, limiting current learning capabilities. To combat this issue and improves the generalizability of the DRP models, we are exploring strategies that explicitly address the imbalance in the DRP datasets. We reframe the problem as a multi-objective optimization across multiple drugs to maximize deep learning model performance. We implement this approach by constructing Multi-Objective Optimization Regularized by Loss Entropy loss function and plugging it into a Deep Learning model. We demonstrate the utility of proposed drug discovery methods and make suggestions for further potential application of the work to achieve desirable outcomes in the healthcare field.

Biochemistry & Molecular Biology↗

DOE OSTI2025

Automated Image Segmentation and Processing Pipeline Applied to X–Ray Computed Tomography Studies of Pitting Corrosion in Aluminum Wires

Understanding pitting corrosion is critical, yet its kinetics and morphology remain challenging to study from X-ray computed tomography (XCT) due to manual segmentation barriers. To address this, an automated pipeline leveraging deep learning for efficient large-scale XCT analysis is developed, revealing new corrosion insights. The pipeline enables pit segmentation, 3D reconstruction, statistical characterization, and a topological transformation for visualization. Here, the pipeline is applied to 87 648 XCT images capturing commercial purity aluminum (1100 Al) wire exposed to sodium chloride (NaCl) salt particles over a period of 122 h. The pipeline achieves complete feature extraction and statistical quantification across the entire XCT dataset, leveraging distributed computing environment for high efficiency. Global growth kinetics such as high-level stepwise sigmoidal volume loss patterns and granular individual pit developments are both captured for 36 detected pits. By combining automation, computer vision, and extensive XCT datasets, this research accelerates precise corrosion assessment to enable materials science discoveries at scale.

36 MATERIALS SCIENCE↗

DOE OSTI2025

ORBITaL-Net: A labeled training library for large-scale building feature extraction

Over the course of several years, nearly 1.5 million building outlines have been created from approximately 128,000 training tiles covering roughly 7,000 km 2 of very high-resolution multispectral overhead imagery, primarily dated between 2010 and 2020. This dataset, dubbed the Oak Ridge Building Image and TrAining Label Net (ORBITaL-Net), is designed for machine learning applications and is global in scope, with samples drawn from 72 countries across North America, South America, Africa, Europe, and Asia. ORBITaL-Net captures a great diversity in geographic setting, structural characteristics, land use (urban and rural), terrain, and imagery conditions. While the labeled building outlines are themselves valuable, the dataset’s true strength lies in the pairing of these labels with corresponding reference imagery, which is being released for open source use. Similar to SpaceNet and Replicable AI For Microplanning (ramp), this building outline dataset will allow the larger computer vision community from academia, government, and industry the opportunity to develop robust, scalable, and generalizable geospatial machine learning techniques. Unlike SpaceNet and ramp, which offer high resolution labels and imagery primarily for large urban cities, ORBITaL-Net is not focused on training samples from heavily populated areas but instead aims to capture the innate variability of conditions present in both the physical environment and imagery collections.

Geography↗

NASA NTRS2018

Overview of the Digitization Workflow Post Image Acquisition of Apollo Lunar and Antarctic Meteorite Samples Using Agisoft Photoscan for the NASA 3D Astromaterials Virtual Samples Collection

The 3D Virtual Astromaterials Samples (3DVAS) collection is a multi-year funded project to create a digital database of sixty Apollo Lunar and Antarctic Meteorite samples following non-destructive documentation conservation protocols. After initial image processing, the photos are evaluated and processed using unique structure-from-motion photogrammetric techniques in a high performance modelling software designed to create a 3D model from 2D images: Agisoft Photoscan Pro. Agisoft Photoscan Pro uses image processing algorithms and techniques originating in computer vision to resolve 3D models for accurate and detailed visualization of a subject. The software provides a stepwise process that is tailored per model based on spatial and specular reflectance properties, for example. The process includes: photo alignment, creation of a dense point cloud, mesh, and finally texture. Photo alignment is dependent on model properties. The 3DVAS process requires a special rotation platform with calibrated photogrammetric targets, specific distance rotation protocols, and a contrasting background for alignment and scale accuracy. As a result of the photographic process, alignment will complete with two mirrored hemispheres that, in a sense, represent the 2D images overlapping to create a 3D model. Each dense point cloud is analyzed with provided statistical measures in a gradual selection process to eliminate outliers. The point cloud is reduced to include only data valuable to the final model. When a precise dense point cloud is achieved, a mesh and texture are applied. Each model is scaled with scale bar accuracies within 100 microns. Each sample has its own intimate process for modelling; there is no standard for the parameters required in the final creation of a high resolution model. By processing multiple samples, a skill is gained in practice to allow a close definition of the original sample and will result in the most detailed version of the sample shell. This process completes one-fifth of the 3DVAS protocol for providing accurate digital documentation. Each model shell is merged with X-ray Computed Tomography data to create a full volumetric sample. All 3DVAS data will be served on NASA's Astromaterials Acquisition and Curation website with an early subset of data available in 2019 and the 3D Virtual Astromaterials Samples Collection launch in 2020.

Thomas, Andi B.↗

DOE OSTI2026

Illuminating the Material World: Autonomous Microscopy to Understand Order, Disorder, and Everything In Between

Artificial intelligence (AI) holds immense promise for revolutionizing microscopy, yet its widespread adoption has been hindered by challenges ranging from user inexperience to limited model transferability and difficulties in operationalizing machine learning. This presentation showcases our approach to developing practical autonomy for materials discovery, aiming to accelerate the integration of AI into everyday microscopy workflows. As shown in Fig. 1, I will focus on three key areas: understanding order-disorder transitions, quantifying point defects, and achieving truly device-scale microscopy. First, I will demonstrate the power of multi-modal knowledge graphs for integrating diverse microscopy data. By combining imaging, spectroscopy, and diffraction data, these graphs provide a holistic view of material behavior, capturing the intricate relationships between different modalities [1,2]. I will present a case study on how these models illuminate the structural and chemical changes associated with irradiation in oxide thin films, revealing critical insights for designing materials for extreme environments like spaceflight and nuclear energy. Specifically, I will show how multi-modal analysis clarifies the evolution of order-disorder transitions under irradiation, a key factor influencing material performance in these applications. Next, I will address the challenge of quantifying point defects in 2D materials. We demonstrate the application of computer vision and transfer learning to accurately identify and classify various defect types, such as vacancies and substitutional atoms, and to quantify their concentrations. This information is crucial for understanding and tailoring the properties of 2D materials for applications in electronics, optoelectronics, and catalysis. For example, I will show how our models can characterize the topological distribution of point defects in MXene transition metal carbides, providing valuable insights for optimizing their performance in energy storage and separation science. Finally, I will discuss our progress toward autonomous device-scale microscopy [3,4]. We are fundamentally redesigning electron microscopes around the principles of machine reasoning, enabling automation beyond basic tasks like sample navigation and data acquisition to include sophisticated experimental design. This approach paves the way for truly reproducible and massively scaled analysis campaigns. I will emphasize the importance of autonomous microscopy platforms for high-throughput materials discovery and characterization, facilitating the rapid screening of materials for a broad range of applications and accelerating the development of next-generation technologies.

36 MATERIALS SCIENCE↗

DOE OSTI2024

HDBind: encoding of molecular structure with hyperdimensional binary representations

Traditional methods for identifying “hit” molecules from a large collection of potential drug-like candidates rely on biophysical theory to compute approximations to the Gibbs free energy of the binding interaction between the drug and its protein target. These approaches have a significant limitation in that they require exceptional computing capabilities for even relatively small collections of molecules. Increasingly large and complex state-of-the-art deep learning approaches have gained popularity with the promise to improve the productivity of drug design, notorious for its numerous failures. However, as deep learning models increase in their size and complexity, their acceleration at the hardware level becomes more challenging. Hyperdimensional Computing (HDC) has recently gained attention in the computer hardware community due to its algorithmic simplicity relative to deep learning approaches. The HDC learning paradigm, which represents data with high-dimension binary vectors, allows the use of low-precision binary vector arithmetic to create models of the data that can be learned without the need for the gradient-based optimization required in many conventional machine learning and deep learning methods. This algorithmic simplicity allows for acceleration in hardware that has been previously demonstrated in a range of application areas (computer vision, bioinformatics, mass spectrometery, remote sensing, edge devices, etc.). To the best of our knowledge, our work is the first to consider HDC for the task of fast and efficient screening of modern drug-like compound libraries. We also propose the first HDC graph-based encoding methods for molecular data, demonstrating consistent and substantial improvement over previous work. We compare our approaches to alternative approaches on the well-studied MoleculeNet dataset and the recently proposed LIT-PCBA dataset derived from high quality PubChem assays. We demonstrate our methods on multiple target hardware platforms, including Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), showing at least an order of magnitude improvement in energy efficiency versus even our smallest neural network baseline model with a single hidden layer. Our work thus motivates further investigation into molecular representation learning to develop ultra-efficient pre-screening tools. We make our code publicly available at https://github.com/LLNL/hdbind.

59 BASIC BIOLOGICAL SCIENCES↗