SEARCH · Search NASA

Results for “Object detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Active Learning Meets Foundation Models: Fast Remote Sensing Data Annotation for Object Detection

Object detection in remote sensing demands extensive, high-quality annotations—a process that is both labor-intensive and time-consuming. In this work, we introduce a real-time active learning and semi-automated labeling framework that leverages foundation models to streamline dataset annotation for object detection in remote sensing imagery. For example, by integrating a Segment Anything Model (SAM), our approach generates mask-based bounding boxes that serve as the basis for dual sampling: (a) uncertainty estimation to pinpoint challenging samples, and (b) diversity assessment to ensure broad data coverage. Furthermore, our Dynamic Box Switching Module (DBS) addresses the well-known cold start problem for object detection models by replacing its suboptimal initial predictions with SAM-derived masks, thereby enhancing early-stage localization accuracy. Extensive evaluations on multiple remote sensing datasets plus a real-world user study, demonstrate that our framework not only reduces annotation effort, but also significantly boosts detection performance compared to traditional active learning sampling methods. The code for training and the user interface will be made available.

Burges, Marvin [ORNL] (ORCID:0000000312690769)↗

DOE OSTI2021

Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images

Known for their simplistic design and continuous detonation, rotating detonation engines (RDEs) constitute a majority of current pressure gain combustion (PGC) research efforts. Experimental RDE operation times have been continuously extended through the use of rig cooling techniques. As the window of observable behavior is expanded, and as the technology matures toward eventual integration within gas turbines, monitoring techniques must evolve to better match industrial diagnostics. High-speed image analysis techniques prove useful to capture and evaluate the unsteady detonation behavior within the RDE. Traditional image analysis techniques, however, require extensive processing times which prohibit simultaneous monitoring. To better address this problem, a computer vision object detection methodology is proposed to quickly detect individual detonation waves within a single down-axis image. Detonation waves are detected in individual images by the implemented computer vision method You Only Look Once (YOLO) object detection network. In order to detect detonation waves, the network must first be trained using RDE images of interest, for which each required phase of network development is outlined. Detection of waves is improved through proper treatment of the collected image set, variation of Intersection over Union (IoU) and confidence thresholding, and through a parametric study of annotation dimensions. Each detected wave is described by its location and rotational direction, and locations are tracked to calculate wave velocity across each frame, leading to a timestep resolution of 20 µs. Wave velocities are also calculated through a series of frames, leading to a suitable average velocity estimation using as few as 10 frames. Uncertainty analysis accounting for variation in camera framerate, pixel width and annotation centroid locations estimates a total uncertainty of ±4.3% for velocity calculations, using the smallest annotation boxes. This method offers great reductions in processing times, as a step toward real-time monitoring of detonation waves within an RDE. Improving on previous studies, this technique is impartial to wave modes not included in the original training set and calculates wave velocities independent of high-speed pressure data. The ability to isolate waves within predicted bounding boxes will likely facilitate analysis of pixel intensity variation as an estimation of wave strength in future work.

Johnson, Kristyn↗

DOE OSTI2020

Dynamical Classification of Trans-Neptunian Objects Detected by the Dark Energy Survey

The outer solar system contains a large number of small bodies (known as trans-Neptunian objects or TNOs) that exhibit diverse types of dynamical behavior. The classification of bodies in this distant region into dynamical classes—subpopulations that experience similar orbital evolution—aids in our understanding of the structure and formation of the solar system. In this work, we propose an updated dynamical classification scheme for the outer solar system. This approach includes the construction of a new (automated) method for identifying mean motion resonances. We apply this algorithm to the current data set of TNOs observed by the Dark Energy Survey (DES) and present a working classification for all of the DES TNOs detected to date. Our classification scheme yields 1 inner centaur, 19 outer centaurs, 21 scattering disk objects, 47 detached TNOs, 48 securely resonant objects, 7 resonant candidates, and 97 classical belt objects. Among the scattering and detached objects, we detect 8 TNOs with semimajor axes greater than 150 au.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

UIR-Net: Object Detection in Infrared Imaging of Thermomechanical Processes in Automotive Manufacturing

Thermomechanical processes (TMPs) such as resistance spot welding (RSW) and hot stamping are widely used in automotive manufacturing. Recent advancement in sensing technology has led to an increasing adoption of thermographic cameras to capture the infrared (IR) radiation of a metal part (or component of a part) during its thermomechanical processing or immediately after the process when the part is still hot. Detecting the object(s) of interest from raw IR images is an essential step in analyzing these data. Deep learning (DL) has been a recent success for object detection (OD), but the application of DL-based OD for industrial IR images in manufacturing is largely lagging behind. The major contribution of this work, which is also the distinction from previous OD studies, is the capability of building the OD model with unlabeled IR images, i.e., imaging data without accurate information indicating the object position. Here, the architecture of Unsupervised IR Image Net (UIR-Net) is designed to accommodate the unique characteristics of IR images from TMPs in manufacturing. This study presents a novel method for OD in unlabeled IR images from TMPs. The proposed method, called UIR-Net, consists of two components: label generation and DL model construction. Two case studies from automotive manufacturing, RSW and hot stamping, are reported to demonstrate the feasibility and effectiveness of the proposed method.

42 ENGINEERING↗

DOE OSTI2022

Lightweight jet reconstruction and identification as an object detection task

We apply object detection techniques based on deep convolutional blocks to end-to-end jet identification and reconstruction tasks encountered at the CERN large hadron collider (LHC). Collision events produced at the LHC and represented as an image composed of calorimeter and tracker cells are given as an input to a Single Shot Detection network. The algorithm, named PFJet-SSD performs simultaneous localization, classification and regression tasks to cluster jets and reconstruct their features. This all-in-one single feed-forward pass gives advantages in terms of execution time and an improved accuracy w.r.t. traditional rule-based methods. A further gain is obtained from network slimming, homogeneous quantization, and optimized runtime for meeting memory and latency constraints of a typical real-time processing environment. We experiment with 8-bit and ternary quantization, benchmarking their accuracy and inference latency against a single-precision floating-point. We show that the ternary network closely matches the performance of its full-precision equivalent and outperforms the state-of-the-art rule-based algorithm. Finally, we report the inference latency on different hardware platforms and discuss future applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Wave Detection and Tracking Within a Rotating Detonation Engine Through Object Detection

As the operational time window of experimental rotating detonation engines (RDEs) is expanded and the technology matures toward integration within gas turbines, monitoring techniques must evolve to offer computationally efficient and highly time-resolved diagnostics. In this study, computer vision object detection methodology that seeks to reduce data processing time and calculate wave velocity within drastically reduced time intervals as compared to traditional high-frame-rate RDE images analysis techniques is proposed. The adapted you-only-look-once object detection network is trained to detect individual detonation waves within single down-axis RDE images. The wave location and rotational direction detected within a frame are tracked through a series of high-speed images to calculate the frame-to-frame wave velocity with the time-step resolution of $\mathrm{20 μs}$ across a series of frames. The analysis of the annotation box size and image linearization effects is presented, demonstrating the lowest frame-to-frame velocity total uncertainty of $\mathrm{±3.8\%}$ and the highest classification speed of 9.5 frames per second using linearized images. Linearized images “unwrap” the RDE annulus pixel region to a reduced image size. Here, this new method offers great reductions in data processing times and unsteady detonation behavior insight at intervals more comparable to the timescales of detonation wave interactions via the application of machine learning to experimental RDE data.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2025

Interactive Rotated Object Detection for Novel Class Detection in Remotely Sensed Imagery

In this paper we propose IRTR-DETR an Interactive and Real-Time Rotated DEtection TRansformer that extends IRTDETR to predict rotated bounding boxes. IRTR-DETR maintains the Human-In-The-Loop (HIL) workflow of IRTDETR but introduces rotation-aware heads for improved detection of objects with arbitrary orientations. Similarly to IRTDETR IRTR-DETR can be trained with a small labeled sample set in an interactive setting but we show that it can also be pretrained on related but not identical data--such as a building damage dataset--before being applied to tasks like identifying buildings under construction. We demonstrate the efficacy of our approach on the publicly available Tiny-DOTA and xBD dataset as well as two study-cases on proprietary datasets of greenhouses and houses under construction ("waffle homes"). Detecting greenhouses is highly relevant in the context of damage assessment while "waffle homes" aid understanding typical floorplans and building codes in different areas both thereby supporting population modeling emergency response and policy planning. Our method outperforms the state of the art in interactive rotated object detection on the Tiny-DOTA dataset by 5.7 percent and improves upon the non interactive RTDETR by 7.85 to 19.39 percent (depending on the number of provided samples) while maintaining its real-time efficiency.

Burges, Marvin [ORNL] (ORCID:0000000312690769)↗

DOE OSTI2020

Improving radiograph analysis throughput through transfer learning and object detection

SIGN Fracture Care International partners with surgeons in low-resource hospitals worldwide to provide access to effective orthopedic care by donating educational materials and innovatively designed surgical implants. Over two decades, SIGN’s Online Surgical Database (SOSD) has grown to contain over 500,000 medical images, with radiographs holding the majority share. One challenge in working with hospitals worldwide is that both the radiographs uploaded to the SOSD and the data entry accompanying the uploads vary in quality. To improve the accuracy of data in the SOSD, we trained a model to detect surgical implants in radiographs. We first developed a tool to automatically detect radiographs, then trained an object detection model to determine the number and placement of surgical implants visible in the radiograph. Active learning was used to generate a training set containing 2,510 radiographs with screws, nails, and plates labeled by bounding boxes. Training a model to simultaneously recognize all three classes of implants gave a low average precision (AP) for the plate class, likely due to the low number of plate instances in our training set and the large variety of surgical plates used by SIGN-partnered surgeons. Applying standard image augmentation techniques to increase the plate count in our training set did not appreciably increase the AP of plate detection. To improve plate detection, we redrew the bounding boxes to account for correlations between the screw and plate classes. Training one model to detect nails and screws and a separate model to detect plates increased the AP of plate detection by 78.8 percentage points. The AP of each class was 80.7% for screws, 93.6% for nails, and 92.6% for plates; meanwhile, the sensitivity was 92% for screws, 86% for nails, and 81% for plates. We show that object detection methods can be used to detect surgical implants in radiographs of varying quality; however, the detection ability is dependent on the type of implant, and some implants, in our case plates, must be treated differently than others. Such tools can improve the throughput of radiograph analysis, assisting physicians and surgeons with the treatment of bone fractures.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images

Johnson, Kristyn↗

DOE OSTI2020

Mitigating Shear-dependent Object Detection Biases with Metacalibration

M ETACALIBRATION is a new technique for measuring weak gravitational lensing shear that is unbiased for isolated galaxy images. Here we test M ETACALIBRATION with overlapping, or "blended" galaxy images. Using standard M ETACALIBRATION , we find a few percent shear measurement bias for galaxy densities relevant for current surveys, and that this bias increases with increasing galaxy number density. We show that this bias is not due to blending itself, but rather to shear-dependent object detection. If object detection is shear-independent, no deblending of images is needed, in principle. We demonstrate that detection biases are accurately removed when including object detection in the M ETACALIBRATION process, a technique we call M ETADETECTION . This process involves applying an artificial shear to images of small regions of sky and performing detection on the sheared images, as well as measurements that are used to calculate a shear response. We demonstrate that the method can accurately recover weak shear signals even in highly blended scenes. In the M ETACALIBRATION process, the space between objects is sheared coherently, which does not perfectly match the real universe in which some, but not all, galaxy images are sheared coherently. We find that even for the worst case scenario, in which the space between objects is completely unsheared, the resulting shear bias is at most a few tenths of a percent for future surveys. We discuss additional technical challenges that must be met in order to implement M ETADETECTION for real surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

Performance Comparison of Object Detection Networks for Shrapnel Identification in Ultrasound Images

Ultrasound imaging is a critical tool for triaging and diagnosing subjects but only if images can be properly interpreted. Unfortunately, in remote or military medicine situations, the expertise to interpret images can be lacking. Machine-learning image interpretation models that are explainable to the end user and deployable in real time with ultrasound equipment have the potential to solve this problem. We have previously shown how a YOLOv3 (You Only Look Once) object detection algorithm can be used for tracking shrapnel, artery, vein, and nerve fiber bundle features in a tissue phantom. However, real-time implementation of an object detection model requires optimizing model inference time. Here, we compare the performance of five different object detection deep-learning models with varying architectures and trainable parameters to determine which model is most suitable for this shrapnel-tracking ultrasound image application. We used a dataset of more than 16,000 ultrasound images from gelatin tissue phantoms containing artery, vein, nerve fiber, and shrapnel features for training and evaluating each model. Every object detection model surpassed 0.85 mean average precision except for the detection transformer model. Overall, the YOLOv7tiny model had the higher mean average precision and quickest inference time, making it the obvious model choice for this ultrasound imaging application. Other object detection models were overfitting the data as was determined by lower testing performance compared with higher training performance. In summary, the YOLOv7tiny object detection model had the best mean average precision and inference time and was selected as optimal for this application. Next steps will implement this object detection algorithm for real-time applications, an important next step in translating AI models for emergency and military medicine.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Glass Refraction Distortion Object Detection via Abstract Features

Glass reflection and refraction lead to missing and distorted object feature data, affecting the accuracy of object detection. In order to solve the above problems, this paper proposed a glass refraction distortion object detection via abstract features. The number of parameters of the algorithm is reduced by introducing skip connections and expansion modules with different expansion rates. The abstract feature information of the object is extracted by binary cross-entropy loss. Meanwhile, the abstract feature distance between the object domain and source domain is reduced by a loss function, which improves the accuracy of object detection under glass interference. To verify the effectiveness of the algorithm in this paper, the GRI dataset is produced and made public on GitHub. The algorithm of this paper is compared with the current state-of-the-art Deep Face, VGG Face, TBE-CNN, DA-GAN, PEN-3D, LMZMPM, and the average detection accuracy of our algorithm is 92.57% at the highest, and the number of parameters is only 5.13 M.

Cai, Lei↗

DOE OSTI2022

An Application of Faster R-CNNs in Object Detection Efforts Applied to the Critical Infrastructure Domain

Object detection and segmentation approaches have been successfully applied to multiple interdisciplinary domains ranging from medical imaging to autonomous vehicles. The following study details efforts to apply segmentation methods to the critical infrastructure domain. Specifically, the identification of critical sub-components of four facilities: airports, hospitals, hydroelectric dams, and water treatment plants. Data for this study was extracted from the U.S. Department of Agriculture’s National Agriculture Imagery Program data set. Each facility class was considered independently of the other facilities, producing four different segmentation models during the testing and training process. Ultimately, the study team concluded there are significant challenges associated with object detection modeling when applied to facility analysis within the critical infrastructure domain. The study team was able to identify possible sources of these challenges that could be potentially addressed in future work.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Concept Study of Robotic Camera-Based Foreign Object Detection for EV Wireless Charging

Wireless charging of an electric vehicle (EV) is an emerging charging technology promising convenient, autonomous, and highly efficient EV charging without requiring heavy gauge cables. However, due to the strong electromagnetic field created by this process that surrounds the wireless charger, the presence of foreign objects can detrimentally interact with it, thus affecting wireless power transfer (WPT) performance or leading to harmful and unwanted safety risks. This paper presents the results for a concept study on a robotic camera-based foreign object detection (FOD) system, as a supplement to the industry-existing overlapped FOD coil array method, for EV wireless charging. A Raspberry PI 4 control board and compatible Raspberry PI Camera Module 2 are used to implement camera-based object detection. The FOD program was developed using a state-of-the-art deep learning object detection model with the OpenCV and Pytorch library and is compatible with camera module hardware. A dry-run test with Raspberry PI and a camera module was conducted and the preliminary FOD function was verified. The feasibility assessment is also validated by comparing the performance of five existing state-of-the-art deep learning object detection models for vehicles, animals, persons, and metals subsets, respectively. Satisfactory performance on the benchmark datasets is observed by the tests, but further improvements are needed in future work when detecting small-sized metallic objects. A programable robotic car is also under development as ongoing work for carrying the Raspberry PI and camera module while moving for the maintenance process.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Event-to-Video Conversion for Overhead Object Detection

Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of overlap between the event representations and the pre-training data that the object detectors were initially trained on through a number of experiments. Then, apply an off-the-shelf event-to-video conversion tool that converts event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection techniques on our overhead target task. These results suggest that better aligning event representations with existing large pre-trained models may result in greater short-term performance gains compared to end-to-end event-specific architectural improvements.

machine learning (ML), computer vision, Neuromorph↗

DOE OSTI2024

AEPF: Attention-Enabled Point Fusion for 3D Object Detection

Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class’s easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.

Chemistry↗

DOE OSTI2024

Exploring Object Detection and Image Classification Tasks for Niche Use Case in Naturalistic Driving Studies

Naturalistic driving studies consist of drivers using their personal vehicles and provide valuable real-world data, but privacy issues must be handled very carefully. Drivers sign a consent form when they elect to participate, but passengers do not for a variety of practical reasons. However, their privacy must still be protected. One large study includes a blurred image of the entire cabin which allows reviewers to find passengers in the vehicle; this protects the privacy but still allows a means of answering questions regarding the impact of passengers on driver behavior. A method for automatically counting the passengers would have scientific value for transportation researchers. We investigated different image analysis methods for automatically locating and counting the non-drivers including simple face detection and fine-tuned methods for image classification and a published object detection method. We also compared the image classification using convolutional neural network and vision transformer backbones. Our studies show the image classification method appears to work the best in terms of absolute performance, although we note the closed nature of our dataset and nature of the imagery makes the application somewhat niche and object detection methods also have advantages. We perform some analysis to support our conclusion.

Peruski, Ryan↗

DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗