Improving Object Detectors by Exploiting Bounding Boxes for Augmentation Design
Not Available
97 MATHEMATICS AND COMPUTING↗
SEARCH · Search NASA
Results for “Bounding boxes”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Centimeter human skeleton pose estimation
A method, apparatus and system for human skeleton pose estimation includes synchronously capturing images of a human moving through an area from a plurality of different points of view, for each of the plurality of captured images, determining a bounding box that bounds the human in the captured image and identifying pixel locations of the bounding box in the image, for each of the plurality of captured images, determining 2D and single-view 3D skeletons from the pixel locations of the bounding box, determining a first, multi-view 3D skeleton using a combination of the 2D and single-view 3D skeletons, and optimizing the first, multi-view 3D skeleton to determine a final 3D skeleton pose for the human. The method, apparatus and system can further include illuminating the area with structured light during the capturing of the images of the human moving through the area.
Brookshire, Jonathan D.↗
Advancing Tassel Detection and Counting: Annotation and Algorithms
Tassel counts provide valuable information related to flowering and yield prediction in maize, but are expensive and time-consuming to acquire via traditional manual approaches. High-resolution RGB imagery acquired by unmanned aerial vehicles (UAVs), coupled with advanced machine learning approaches, including deep learning (DL), provides a new capability for monitoring flowering. In this article, three state-of-the-art DL techniques, CenterNet based on point annotation, task-aware spatial disentanglement (TSD), and detecting objects with recursive feature pyramids and switchable atrous convolution (DetectoRS) based on bounding box annotation, are modified to improve their performance for this application and evaluated for tassel detection relative to Tasselnetv2+. The dataset for the experiments is comprised of RGB images of maize tassels from plant breeding experiments, which vary in size, complexity, and overlap. Results show that the point annotations are more accurate and simpler to acquire than the bounding boxes, and bounding box-based approaches are more sensitive to the size of the bounding boxes and background than point-based approaches. Overall, CenterNet has high accuracy in comparison to the other techniques, but DetectoRS can better detect early-stage tassels. The results for these experiments were more robust than Tasselnetv2+, which is sensitive to the number of tassels in the image.
54 ENVIRONMENTAL SCIENCES↗
Vehicle Localization in 3D World Coordinates Using Single Camera at Traffic Intersection
Optimizing traffic control systems at traffic intersections can reduce the network-wide fuel consumption, as well as emissions of conventional fuel-powered vehicles. While traffic signals have been controlled based on predetermined schedules, various adaptive signal control systems have recently been developed using advanced sensors such as cameras, radars, and LiDARs. Among these sensors, cameras can provide a cost-effective way to determine the number, location, type, and speed of the vehicles for better-informed decision-making at traffic intersections. In this research, a new approach for accurately determining vehicle locations near traffic intersections using a single camera is presented. For that purpose, a well-known object detection algorithm called YOLO is used to determine vehicle locations in video images captured by a traffic camera. YOLO draws a bounding box around each detected vehicle, and the vehicle location in the image coordinates is converted to the world coordinates using camera calibration data. During this process, a significant error between the center of a vehicle’s bounding box and the real center of the vehicle in the world coordinates is generated due to the angled view of the vehicles by a camera installed on a traffic light pole. As a means of mitigating this vehicle localization error, two different types of regression models are trained and applied to the centers of the bounding boxes of the camera-detected vehicles. The accuracy of the proposed approach is validated using both static camera images and live-streamed traffic video. Based on the improved vehicle localization, it is expected that more accurate traffic signal control can be made to improve the overall network-wide energy efficiency and traffic flow at traffic intersections.
47 OTHER INSTRUMENTATION↗
Estimating vehicle fuel economy from overhead camera imagery and application for traffic control
In this work, we explore the ability to estimate vehicle fuel consumption using imagery from overhead fisheye lens cameras deployed as traffic sensors. We utilize this information to simulate vision-based control of a traffic intersection, with a goal of improving fuel economy with minimal impact to mobility. We introduce the ORNL Overhead Vehicle Dataset (OOVD), consisting of a data set of paired, labeled vehicle images from a ground-based camera and an overhead fisheye lens traffic camera. The data set includes segmentation masks based on Gaussian mixture models for vehicle detection. We show the dataset utility through three applications: the estimate of fuel consumption based on segmentation bounding boxes, vehicle discrimination for those vehicles with largest bounding boxes, and a fine-grained classification on a limited number of vehicle makes and models using a pre-trained set of convolutional neural network models. We compare these results with estimates based on a large open-source data set based on web-scraped imagery. Finally, we show the utility of the approach using reinforcement learning in a traffic simulator using the open source Simulation of Urban Mobility (SUMO) package. Our results show the feasibility of the approach for controlling traffic lights for better fuel efficiency based solely on visual vehicle estimates from commercial, fisheye lens cameras.
Karnowski, Thomas↗
YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy
Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this paper a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.
3D instance segmentation↗
A method for bounding high-order finite element functions: Applications to mesh validity and bounds-preserving limiters
We introduce a novel method for bounding high-order multi-dimensional polynomials in finite element approximations. The method involves precomputing optimal piecewise-linear bounding boxes for polynomial basis functions, which can then be used to locally bound any combination of these basis functions. This approach can be applied to any element/basis type at any approximation order, can provide local (i.e., subcell) extremum bounds to a desired level of accuracy, and can be evaluated efficiently on-the-fly in simulations. Furthermore, we show that this approach generally yields more accurate bounds in comparison to traditional methods based on convex hull properties (e.g., Bernstein polynomials). Furthermore, the efficacy of this technique is shown in applications such as mesh validity checks and optimization for high-order curved meshes, where positivity of the element Jacobian determinant can be ensured throughout the entire element, and continuously bounds-preserving limiters for hyperbolic systems, which can enforce maximum principle bounds across the entire solution polynomial.
Bounding box↗
RG-CAT: Detection pipeline and catalogue of radio galaxies in the EMU pilot survey
Abstract We present source detection and catalogue construction pipelines to build the first catalogue of radio galaxies from the 270$\rm deg^2$pilot survey of the Evolutionary Map of the Universe (EMU-PS) conducted with the Australian Square Kilometre Array Pathfinder (ASKAP) telescope. The detection pipeline uses Gal-DINO computer vision networks (Gupta et al. 2024, PASA, 41, e001) to predict the categories of radio morphology and bounding boxes for radio sources, as well as their potential infrared host positions. The Gal-DINO network is trained and evaluated on approximately 5 000 visually inspected radio galaxies and their infrared hosts, encompassing both compact and extended radio morphologies. We find that the Intersection over Union (IoU) for the predicted and ground-truth bounding boxes is larger than 0.5 for 99% of the radio sources, and 98% of predicted host positions are within$3^{\prime \prime}$of the ground-truth infrared host in the evaluation set. The catalogue construction pipeline uses the predictions of the trained network on the radio and infrared image cutouts based on the catalogue of radio components identified using theSelavysource finder algorithm. Confidence scores of the predictions are then used to prioritiseSelavycomponents with higher scores and incorporate them first into the catalogue. This results in identifications for a total of 211 625 radio sources, with 201 211 classified as compact and unresolved. The remaining 10 414 are categorised as extended radio morphologies, including 582 FR-I, 5 602 FR-II, 1 494 FR-x (uncertain whether FR-I or FR-II), 2 375 R (single-peak resolved) radio galaxies, and 361 with peculiar and other rare morphologies. Each source in the catalogue includes a confidence score. We cross-match the radio sources in the catalogue with the infrared and optical catalogues, finding infrared cross-matches for 73% and photometric redshifts for 36% of the radio galaxies. The EMU-PS catalogue and the detection pipelines presented here will be used towards constructing catalogues for the main EMU survey covering the full southern sky.
Astronomy & Astrophysics↗
Improving radiograph analysis throughput through transfer learning and object detection
SIGN Fracture Care International partners with surgeons in low-resource hospitals worldwide to provide access to effective orthopedic care by donating educational materials and innovatively designed surgical implants. Over two decades, SIGN’s Online Surgical Database (SOSD) has grown to contain over 500,000 medical images, with radiographs holding the majority share. One challenge in working with hospitals worldwide is that both the radiographs uploaded to the SOSD and the data entry accompanying the uploads vary in quality. To improve the accuracy of data in the SOSD, we trained a model to detect surgical implants in radiographs. We first developed a tool to automatically detect radiographs, then trained an object detection model to determine the number and placement of surgical implants visible in the radiograph. Active learning was used to generate a training set containing 2,510 radiographs with screws, nails, and plates labeled by bounding boxes. Training a model to simultaneously recognize all three classes of implants gave a low average precision (AP) for the plate class, likely due to the low number of plate instances in our training set and the large variety of surgical plates used by SIGN-partnered surgeons. Applying standard image augmentation techniques to increase the plate count in our training set did not appreciably increase the AP of plate detection. To improve plate detection, we redrew the bounding boxes to account for correlations between the screw and plate classes. Training one model to detect nails and screws and a separate model to detect plates increased the AP of plate detection by 78.8 percentage points. The AP of each class was 80.7% for screws, 93.6% for nails, and 92.6% for plates; meanwhile, the sensitivity was 92% for screws, 86% for nails, and 81% for plates. We show that object detection methods can be used to detect surgical implants in radiographs of varying quality; however, the detection ability is dependent on the type of implant, and some implants, in our case plates, must be treated differently than others. Such tools can improve the throughput of radiograph analysis, assisting physicians and surgeons with the treatment of bone fractures.
60 APPLIED LIFE SCIENCES↗
Multi-resolution Arctic Shrub Cover Dataset Derived from UAS and Airborne SfM and LiDAR (2013-2025)
We synthesized 177 unoccupied aerial system flights and 77 airborne flights across the Arctic and created a multi-resolution benchmark data of low-to-tall shrub fractional cover leveraging Structure-from-Motion and Light Detection and Ranging. The resulting dataset covered a total of 1899 km2 across Alaska, Western Canada, Sweden, and Siberian Arctic, including key sites from the Oro Arctic to the High Arctic. The dataset is organized into 6 primary data collection directories (“Abisko,” “AWI,” “ERE,” “Fairbanks,” “NGEE,” “Toolik”), each containing site and flight subdirectories. Flight directories include shrub cover rasters (*.tifs) at 1 m, 5 m, and 30 m resolution, the canopy height model at 1 m resolution (*.tifs), and a bounding box *.kml file. For the AWI, Abisko, NGEE, and Fairbanks collections, we also include the GCC raster at 1 m resolution (*.tif). Files are organized by Collection > Site > Flight Name > Data Files. Flight rasters are in the local UTM zone and the .kml files are in the geographic coordinate system EPSG 4326. We also include a .csv file that details the source datasets for every flight. The Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) project is a research effort to reduce uncertainty in the Department of Energy’s Energy Exascale Earth System Model (E3SM) by developing a predictive understanding of Arctic tundra ecosystems underlain by permafrost and to quantify feedbacks from the Arctic tundra to the Earth system. NGEE Arctic is supported by the Department of Energy's Office of Biological and Environmental Research. Over Phases 1–3, observations made by the NGEE Arctic team across a gradient of permafrost landscapes in Arctic Alaska improved the representation of tundra processes in the land surface component of E3SM (the E3SM Land Model, ELM). Model improvements emphasized unique aspects of permafrost environments and explored reductions in model complexity while retaining predictive power. The Arctic-informed ELM developed by NGEE Arctic has been used to make novel predictions on processes ranging from permafrost thaw to soil biogeochemical cycling to Earth system feedbacks associated with the unique characteristics of tundra plants. In Phase 4, the NGEE Arctic team is evaluating our new predictive understanding under novel conditions across the Arctic domain. In collaboration with partners at long-term pan-Arctic research sites we are examining whether an Arctic-informed ELM can faithfully simulate interactions among surface and subsurface processes at site, regional, and pan-Arctic scales. In turn, we are using variety of tools to dynamically extend and evaluate ELM inference, with an emphasis on data synthesis and pan-Arctic model evaluation, reintegration of code with an evolving E3SM, scaling across heterogeneous Arctic landscapes, and the appropriate representation of the impacts of increasingly frequent Arctic disturbances.
canopy height model↗
General field evaluation in high-order meshes on GPUs
Robust and scalable function evaluation at any arbitrary point in the finite/spectral element mesh is required for querying the partial differential equation solution at points of interest, comparison of solution between different meshes, and Lagrangian particle tracking. This is a challenging problem, particularly for high-order unstructured meshes partitioned in parallel with MPI, as it requires identifying the element that overlaps a given point and computing the corresponding reference space coordinates. Here, we present a robust and efficient technique for general field evaluation in large-scale high-order meshes with quadrilaterals and hexahedra. In the proposed method, a combination of globally partitioned and processor-local maps are used to first determine a list of candidate MPI ranks, and then locally candidate elements that could contain a given point. Next, element-wise bounding boxes further reduce the list of candidate elements. Finally, Newton’s method with trust region is used to determine the overlapping element and corresponding reference space coordinates. Since GPU-based architectures have become popular for accelerating computational analyses using meshes with tensor-product elements, specialized kernels have been developed to utilize the proposed methodology on GPUs. The method is also extended to enable general field evaluation on surface meshes. The paper concludes by demonstrating the use of the proposed method in various applications ranging from mesh-to-mesh transfer during r-adaptivity to Lagrangian particle tracking.
97 MATHEMATICS AND COMPUTING↗
Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images
Known for their simplistic design and continuous detonation, rotating detonation engines (RDEs) constitute a majority of current pressure gain combustion (PGC) research efforts. Experimental RDE operation times have been continuously extended through the use of rig cooling techniques. As the window of observable behavior is expanded, and as the technology matures toward eventual integration within gas turbines, monitoring techniques must evolve to better match industrial diagnostics. High-speed image analysis techniques prove useful to capture and evaluate the unsteady detonation behavior within the RDE. Traditional image analysis techniques, however, require extensive processing times which prohibit simultaneous monitoring. To better address this problem, a computer vision object detection methodology is proposed to quickly detect individual detonation waves within a single down-axis image. Detonation waves are detected in individual images by the implemented computer vision method You Only Look Once (YOLO) object detection network. In order to detect detonation waves, the network must first be trained using RDE images of interest, for which each required phase of network development is outlined. Detection of waves is improved through proper treatment of the collected image set, variation of Intersection over Union (IoU) and confidence thresholding, and through a parametric study of annotation dimensions. Each detected wave is described by its location and rotational direction, and locations are tracked to calculate wave velocity across each frame, leading to a timestep resolution of 20 µs. Wave velocities are also calculated through a series of frames, leading to a suitable average velocity estimation using as few as 10 frames. Uncertainty analysis accounting for variation in camera framerate, pixel width and annotation centroid locations estimates a total uncertainty of ±4.3% for velocity calculations, using the smallest annotation boxes. This method offers great reductions in processing times, as a step toward real-time monitoring of detonation waves within an RDE. Improving on previous studies, this technique is impartial to wave modes not included in the original training set and calculates wave velocities independent of high-speed pressure data. The ability to isolate waves within predicted bounding boxes will likely facilitate analysis of pixel intensity variation as an estimation of wave strength in future work.
Johnson, Kristyn↗
Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images
Known for their simplistic design and continuous detonation, rotating detonation engines (RDEs) constitute a majority of current pressure gain combustion (PGC) research efforts. Experimental RDE operation times have been continuously extended through the use of rig cooling techniques. As the window of observable behavior is expanded, and as the technology matures toward eventual integration within gas turbines, monitoring techniques must evolve to better match industrial diagnostics. High-speed image analysis techniques prove useful to capture and evaluate the unsteady detonation behavior within the RDE. Traditional image analysis techniques, however, require extensive processing times which prohibit simultaneous monitoring. To better address this problem, a computer vision object detection methodology is proposed to quickly detect individual detonation waves within a single down-axis image. Detonation waves are detected in individual images by the implemented computer vision method You Only Look Once (YOLO) object detection network. In order to detect detonation waves, the network must first be trained using RDE images of interest, for which each required phase of network development is outlined. Detection of waves is improved through proper treatment of the collected image set, variation of Intersection over Union (IoU) and confidence thresholding, and through a parametric study of annotation dimensions. Each detected wave is described by its location and rotational direction, and locations are tracked to calculate wave velocity across each frame, leading to a timestep resolution of 20 µs. Wave velocities are also calculated through a series of frames, leading to a suitable average velocity estimation using as few as 10 frames. Uncertainty analysis accounting for variation in camera framerate, pixel width and annotation centroid locations estimates a total uncertainty of ±4.3% for velocity calculations, using the smallest annotation boxes. This method offers great reductions in processing times, as a step toward real-time monitoring of detonation waves within an RDE. Improving on previous studies, this technique is impartial to wave modes not included in the original training set and calculates wave velocities independent of high-speed pressure data. The ability to isolate waves within predicted bounding boxes will likely facilitate analysis of pixel intensity variation as an estimation of wave strength in future work.
Johnson, Kristyn↗
Explicit model predictive control through robust optimization
A strategy that calculates an explicit state feedback policy to regulate constrained uncertain discrete‐time uncertain linear systems is presented. We consider uncertain processes, affected by box‐bounded multiplicative uncertainty as well as bounded additive uncertainty with linear state and inputs constraints. The proposed method includes (i) the calculation of a terminal set constraint and (ii) the robust reformulation of state constraints in the prediction horizon. These features allow the derivation of the desired policy by solving a single multiparametric quadratic programming problem that guarantees feasible operation in the presence of uncertainty. Additionally, we employ variable and constraint elimination approaches to enhance the computational performance of the strategy. We demonstrate the steps and benefits of these developments with a numerical example and a chemical engineering case study.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Neural-network model for force prediction in multi-principal-element alloys
Atomistic simulations can provide useful insights into the physical properties of multi-principal-element alloys. However, classical potentials mostly fail to capture key quantum (electronic-structure) effects. Here we present a deep 3D convolutional neural network (3D CNN) based framework combined with a voxelization technique to design interatomic potentials for chemically complex alloys. We highlight the performance of the 3D CNN model and its efficacy in computing potentials using the medium-entropy alloy TaNbMo. In order to provide insights into the effect of voxel resolution, we implemented two approaches based on the inner and outer bounding boxes. An efficient 3D CNN model, which is as accurate as the density-functional theory (DFT) approach, for calculating potentials will provide a promising schema for accurate atomistic simulations of structure and dynamics of general multi-principle element alloys.
36 MATERIALS SCIENCE↗
RadioGalaxyNET: Dataset and novel computer vision algorithms for the detection of extended radio galaxies and infrared hosts
Abstract Creating radio galaxy catalogues from next-generation deep surveys requires automated identification of associated components of extended sources and their corresponding infrared hosts. In this paper, we introduce RadioGalaxyNET, a multimodal dataset, and a suite of novel computer vision algorithms designed to automate the detection and localization of multi-component extended radio galaxies and their corresponding infrared hosts. The dataset comprises 4 155 instances of galaxies in 2 800 images with both radio and infrared channels. Each instance provides information about the extended radio galaxy class, its corresponding bounding box encompassing all components, the pixel-level segmentation mask, and the keypoint position of its corresponding infrared host galaxy. RadioGalaxyNET is the first dataset to include images from the highly sensitive Australian Square Kilometre Array Pathfinder (ASKAP) radio telescope, corresponding infrared images, and instance-level annotations for galaxy detection. We benchmark several object detection algorithms on the dataset and propose a novel multimodal approach to simultaneously detect radio galaxies and the positions of infrared hosts.
Astronomy & Astrophysics↗
Crack Identification and Characterization in Deformed Nb3Sn Rutherford Cable Stacks Using Machine Learning
An investigation of instance segmentation of cracks in Nb3Sn 4-stack 40-strand Rutherford cables using machine learning is presented. Three samples were uniaxially and biaxially loaded before metallographic inspections were performed. The Mask R-CNN model was used in the Detectron2 framework with pre-trained weights but fine-tuned to detect and segment cracks. The model detected cracks with bounding box and mask average precisions (AP) of 42.8 and 27.9, respectively, and was used for instance segmentation of all cracks in the three samples. More cracks were found in the sample pre-loaded along the z-axis (i.e., along the cable length). Pre-loading along the x-axis (i.e., on the cables edges) reduced the number of cracks and changed the crack orientation distribution, away from being highly aligned with the y-axis (i.e., normal to the cables broad faces), i.e., the direction with the highest applied load. Fine-tuning of the Segment Anything Model (SAM) was also studied but performed poorly without human-provided prompts. However, the zero-shot capability of SAM showed high promises to accelerate the image annotation process for applications beyond this study.
Croteau, Jean-Francois↗
Uncertainty quantification of fireball features extracted from nuclear test films using computer vision
Films from the US’s historic nuclear testing era comprise the only extensive collection of imagery depicting high-yield detonations. These films offer unique insights into the characteristics of flows occurring on scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models used to describe nuclear detonations. In recent work, we implemented modern computer vision and machine learning techniques to extract features of the fireball following nuclear detonation. With a training dataset of fireball films, we fine-tuned a You Only Look Once 11 (YOLO11) model to detect and track the fireball. Applied to a video, the outer bounding box produced in each frame by YOLO11 is used as an input prompt to Meta’s Segment Anything Model 2 (SAM2), which is shown to accurately predict the boundary of the fireball over time with high resolution. These state-of-the-art computer vision foundation models exhibit impressive visual accuracy in their results but lack an output of values that robustly quantify uncertainty in scientific applications. In this paper, we develop procedures for uncertainty quantification of extracted fireball features. We outline the application of a parallel attention mechanism to calculate uncertainty ranges that complement and better pose model validation data. This higher quality fireball validation data may serve to improve prognostic models describing nuclear detonations in support of nuclear forensic and emergency response activities.
Khristy, Joel [ORNL] (ORCID:0000000209963060)↗