SEARCH · Search NASA

Results for “machine visions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Spinning metasurface stack for spectro-polarimetric thermal imaging

Spectro-polarimetric imaging in the long-wave infrared (LWIR) region plays a crucial role in applications from night vision and machine perception to trace gas sensing and thermography. However, the current generation of spectro-polarimetric LWIR imagers suffers from limitations in size, spectral resolution, and field of view (FOV). While meta-optics-based strategies for spectro-polarimetric imaging have been explored in the visible spectrum, their potential for thermal imaging remains largely unexplored. In this work, we introduce an approach for spectro-polarimetric decomposition by combining large-area stacked meta-optical devices with advanced computational imaging algorithms. The co-design of a stack of spinning dispersive metasurfaces along with compressive sensing and dictionary learning algorithms allows simultaneous spectral and polarimetric resolution without the need for bulky filter wheels or interferometers. Our spinning-metasurface-based spectro-polarimetric stack is compact (<10×10×10cm) and robust, and it offers a wide field of view (20.5°). We show that the spectral resolving power of our system substantially enhances performance in machine learning tasks such as material classification, a challenge for conventional panchromatic thermal cameras. Our approach represents a significant advance in the field of thermal imaging for a wide range of applications including heat-assisted detection and ranging (HADAR).

Optics↗

DOE OSTI2024

From Chaos to Clarity: Autonomous Materials Discovery for Extreme Environments [Slides]

The pursuit of advanced functional materials for energy applications demands an understanding of their behavior under the most challenging conditions. Extreme environments, characterized by intense radiation, high temperatures, and corrosive chemistries, push materials to their limits, often revealing unexpected behaviors and degradation pathways. Traditional materials research approaches, relying on trial-and-error experimentation, are often slow and resource-intensive, ill-suited to the complexities of extreme environments. This talk will explore the transformative potential of autonomous materials science in revolutionizing our understanding of materials synthesis and degradation in extreme environments. By integrating advanced microscopy techniques, artificial intelligence, and robotic experimentation, we can accelerate the discovery and design of resilient materials for a sustainable future. The presentation will highlight recent breakthroughs in autonomous microscopy, computer vision, and machine learning, showcasing their ability to unravel complex material transformations at the atomic scale. The talk will also delve into the challenges and opportunities associated with deploying autonomous systems to probe extreme environments, emphasizing the importance of robust algorithms, real-time data analysis, and adaptive experimentation. The ultimate goal is to empower scientists with unprecedented capabilities to explore, understand, and engineer materials that can withstand the harshest conditions, paving the way for innovations in energy, aerospace, and beyond.

14 SOLAR ENERGY↗

DOE OSTI2025

Unveiling the Hidden Evolution of Crystal Defects and Disorder in Energy Materials

Control of point defects and disorder in functional thin films and 2D materials is critical to realizing their full potential in applications ranging from energy storage to advanced electronics. However, these phenomena are often poorly understood, difficult to characterize, and challenging to direct with precision. This presentation explores emerging multi-modal computer vision to decipher and predict order in materials across multiple length scales in the electron microscope, from the atomic to the nanoscale. By fusing data from diverse sources, these powerful models provide unprecedented insights into materials' lifecycles, enabling the control of defects and their associated properties at a fundamental level. This capability promises to transform materials design and accelerate the development of next-generation technologies.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

AI-Driven Accelerated Inclusion Analysis for Energy Efficient Steelmaking (Final CRADA Report)

This was a collaborative effort between Lawrence Livermore National Security, LLC (LLNS) as manager and operator of Lawrence Livermore National Laboratory (LLNL) and ArcelorMittal USA Research LLC (“ArcelorMittal” as the Participant), to use scanning electron microscopy (SEM) images, computer vision and machine learning methods, and high-performance computing to accelerate the inclusion analysis process of liquid steel so that new methods can be used for near-real time process control on the shop floor.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Vegetation Detection Using Deep Learning and Conventional Methods

Land cover classification with the focus on chlorophyll-rich vegetation detection plays an important role in urban growth monitoring and planning, autonomous navigation, drone mapping, biodiversity conservation, etc. Conventional approaches usually apply the normalized difference vegetation index (NDVI) for vegetation detection. In this paper, we investigate the performance of deep learning and conventional methods for vegetation detection. Two deep learning methods, DeepLabV3+ and our customized convolutional neural network (CNN) were evaluated with respect to their detection performance when training and testing datasets originated from different geographical sites with different image resolutions. A novel object-based vegetation detection approach, which utilizes NDVI, computer vision, and machine learning (ML) techniques, is also proposed. The vegetation detection methods were applied to high-resolution airborne color images which consist of RGB and near-infrared (NIR) bands. RGB color images alone were also used with the two deep learning methods to examine their detection performances without the NIR band. The detection performances of the deep learning methods with respect to the object-based detection approach are discussed and sample images from the datasets are used for demonstrations.

58 GEOSCIENCES↗

DOE OSTI2023

Vehicle Localization in 3D World Coordinates Using Single Camera at Traffic Intersection

Optimizing traffic control systems at traffic intersections can reduce the network-wide fuel consumption, as well as emissions of conventional fuel-powered vehicles. While traffic signals have been controlled based on predetermined schedules, various adaptive signal control systems have recently been developed using advanced sensors such as cameras, radars, and LiDARs. Among these sensors, cameras can provide a cost-effective way to determine the number, location, type, and speed of the vehicles for better-informed decision-making at traffic intersections. In this research, a new approach for accurately determining vehicle locations near traffic intersections using a single camera is presented. For that purpose, a well-known object detection algorithm called YOLO is used to determine vehicle locations in video images captured by a traffic camera. YOLO draws a bounding box around each detected vehicle, and the vehicle location in the image coordinates is converted to the world coordinates using camera calibration data. During this process, a significant error between the center of a vehicle’s bounding box and the real center of the vehicle in the world coordinates is generated due to the angled view of the vehicles by a camera installed on a traffic light pole. As a means of mitigating this vehicle localization error, two different types of regression models are trained and applied to the centers of the bounding boxes of the camera-detected vehicles. The accuracy of the proposed approach is validated using both static camera images and live-streamed traffic video. Based on the improved vehicle localization, it is expected that more accurate traffic signal control can be made to improve the overall network-wide energy efficiency and traffic flow at traffic intersections.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

YOU-ONLY-LOOK-ONCE (YOLO) FOR RADIO FREQUENCY SIGNAL CLASSIFICATION

We propose a method that uses deep learning (DL) to identify and frame various signals that are present in an environment. This DL framework is based on the You Only Look Once (YOLO) object detection pipeline Our work demonstrates a specific application of high performance computing and computer vision to the field of telecommunications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

From Chaos to Clarity: Autonomous Materials Discovery for Extreme Environments

The pursuit of advanced functional materials for energy applications demands an understanding of their behavior under the most challenging conditions. Extreme environments, characterized by intense radiation, high temperatures, and corrosive chemistries, push materials to their limits, often revealing unexpected behaviors and degradation pathways. Traditional materials research approaches, relying on trial-and-error experimentation, are often slow and resource-intensive, ill-suited to the complexities of extreme environments. This talk will explore the transformative potential of autonomous materials science in revolutionizing our understanding of materials synthesis and degradation in extreme environments. By integrating advanced microscopy techniques, artificial intelligence, and robotic experimentation, we can accelerate the discovery and design of resilient materials for a sustainable future. The presentation will highlight recent breakthroughs in autonomous microscopy, computer vision, and machine learning, showcasing their ability to unravel complex material transformations at the atomic scale. The talk will also delve into the challenges and opportunities associated with deploying autonomous systems to probe extreme environments, emphasizing the importance of robust algorithms, real-time data analysis, and adaptive experimentation. Our ultimate goal is to empower scientists with unprecedented capabilities to explore, understand, and engineer materials that can withstand the harshest conditions, paving the way for innovations in energy, aerospace, and beyond.

artificial intelligence↗

DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2022

BFSVBF (BatFinder Smart Video BioFilter) [SWR-22-87] and Multi-class BatFinder Smart Video BioFilter Keras

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The code has gone through five rounds of development with images used to train the models. This advancement allows for automated real-time data collection, identification, and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. We will discuss the two open source and publicly available machine learning models developed within this scope of this work: 1) a binary model with a 97.5% accuracy in identifying the difference between an object and an empty scene, including wind turbine and clouds; and 2) a multiple classification model with the capability of identifying the type of object detected: bats (90% accuracy), birds (83% accuracy), insects (69% accuracy) and non-biological (99% accuracy).

Yarbrough, John↗

DOE OSTI2020

Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterizations

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward by breakthroughs in deep convolutional neural networks. These state-of-the-art methods are poised to become widely used in environmental monitoring applications as a result of the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data) that can be analyzed to observe, model, and understand environmental conditions. Computer vision and pattern recognition (CVPR) tools advance our ability to use imagery and camera-based tools in cost-effective ways for environmental monitoring. Although these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for model training and validation and optimized hardware and software to ensure that the models can be trained effectively and in a reasonable amount of time. Here, we have overcome many of these rate-limiting challenges by using a diverse image library built across multiple projects coupled with staff expertise and onsite computing resources. We are working toward a fully automated SPI processing system and also are moving to develop CVPR analytical tools for other imaging platforms and data sets.

02 PETROLEUM↗

DOE OSTI2023

Trustworthy Physics-Informed Deep Learning for Predictive Scientific Computing

This project has developed powerful trustworthy physics-informed deep learning (TPiDL) models and methods to fundamentally enhance the scale and power of computational modeling in the scientific and engineering domains. Deep learning (DL) has radically advanced the state-of-the-art in machine learning, computer vision, natural language processing, and also scientific computing. Nevertheless, progress has been driven almost entirely by empirical observations, hacks, and tricks. Under the support of this project, the graph operator learning tools and advanced trustworthy physical informed neural networks have been developed. In addition, stochastic gradient replica-exchange Markov Chain Monte Carlo (MCMC) sampling algorithms have been designed to quantify the uncertainties and speed up the training of large-scale neural networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Chapter 18: Learning and Tracking Ad Hoc Fiducial Markers in Spatial Augmented Reality

We describe a spatial augmented reality system with a tangible user interface used to control computer simulations of complex systems. In spatial augmented reality, the user’s physical space is augmented with projected imagery, blending real objects with projected information, and a tangible user interface enables users to manipulate physical objects as controllers for interactive visualizations. Our system learns ad hoc objects in the user’s environment as fiducial markers (i.e., objects that are visually recognized and tracked). When combined with simulation and visualization tools, these interfaces allow the user to control simulations or ensembles of simulations via physical objects using apt metaphors. While other research has leveraged the use of depth cameras, our system enables the use of standard cameras in readily available smartphones and webcams and has an implementation that runs completely in JavaScript in the web browser. We discuss the prerequisite object-recognition requirements for such tangible user interfaces and describe computer-vision and machine-learning algorithms meeting those requirements. We conclude by presenting example applications, which are also available online.

energy distribution↗

DOE OSTI2024

Hyperparameter Studies for Vision Transformers Trained on High-Fidelity Simulations

This library is a collection of python modules that define, train, and analyze vision-transformer (ViT) machine learning models. The code implements, with mild modifications, ViT models that have been made publicly available through publication and GitHub code. The training data for these models is hydrodynamic simulation output in the form of numpy arrays. This library contains code to train these ViT models on the hydrodynamic simulation output with a variety of hyperparameters, and to compare the results of such models. Furthermore, the library contains definitions of simple convolutional neural network (CNN) machine learning architectures which can be trained on the same hydrodynamic simulation output. These are included as a reference point to compare the ViT models to. Additionally, the library includes trained ViT and CNN models and example input data for demonstration purposes. The code is based on the PyTorch python library.

Callis, Skylar↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Fourier-based three-dimensional multistage transformer for aberration correction in multicellular specimens

High-resolution tissue imaging is often compromised by sample-induced optical aberrations that degrade resolution and contrast. Although wavefront sensor-based adaptive optics (AO) can measure these aberrations, such hardware solutions are typically complex, expensive to implement and slow when serially mapping spatially varying aberrations across large fields of view. Here we introduce AOViFT (adaptive optical vision Fourier transformer)—a machine learning-based aberration sensing framework built around a three-dimensional multistage vision transformer that operates on Fourier domain embeddings. AOViFT infers aberrations and restores diffraction-limited performance in puncta-labeled specimens with substantially reduced computational cost, training time and memory footprint compared to conventional architectures or real-space networks. We validated AOViFT on live gene-edited zebrafish embryos, demonstrating its ability to correct spatially varying aberrations using either a deformable mirror or postacquisition deconvolution. By eliminating the need for the guide star and wavefront sensing hardware and simplifying the experimental workflow, AOViFT lowers technical barriers for high-resolution volumetric microscopy across diverse biological samples.

Alshaabi, Thayer [Howard Hughes Medical Institute,↗

DOE OSTI2026

Challenges and Vision for Standardization of Biopolymer Data Sets for Machine Learning

Machine learning (ML) is transforming materials research, yet potential for biopolymer discovery remains constrained by fragmented data and nonstandardized reporting. Biopolymers differ significantly from synthetic polymers, requiring specialized approaches to represent their biosynthetic origins, hierarchical structures, and application-specific metrics. In this Perspective, we identify three core challenges limiting biopolymer representation: information encoding, data quality, and data sharing. We describe the most pressing issues and propose commensurate approaches to address each key challenge. Recommendations include the design and adoption of biopolymer-specific fingerprinting and representation frameworks, development of hybrid human-large language model (LLM) data extraction strategies, and expanding Findable, Accessible, Interoperable, Reusable (FAIR)-compliant repositories. We propose a robust foundation to define interoperable, high-quality data sets that capture the full context of biopolymer materials. Standardized metadata, shared ontologies, and community-driven infrastructure would enable scalable, reproducible workflows and accelerate the ML-driven development of biopolymers.

36 MATERIALS SCIENCE↗