SEARCH · Search NASA

Results for “vehicle detection and tracking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Multi-INT Signature Collection and Exploitation for Security Los Alamos LDRD Report

This project demonstrates the feasibility of a 2-D passive tracking capability for ground-based and airborne moving sources using multi-INT signal processing of acoustic and seismic signals. We leveraged the recent DDSTE-supported investment in rapid deployable sensors (the Los Alamos Rapid-deployable Sensor Array—LARSA), and used uncrewed aerial vehicles (UAV) as a data source. Moving source types are characterized by their variability with time (exemplified but not limited to the Doppler shift of a passing vehicle) and the signatures used for detection should have similar characteristics. We have demonstrated that time-varying sensor signatures (TVS) combined with time-stable sensor signatures provide a more complete understanding of moving source behavior and drive towards path prediction. Importantly, we have collected a new, relevant LANL dataset of moving sources (small UAV flights and vehicle movement), developed new signatures, and demonstrated a capability to identify, track and forecast the position of TVS.

42 ENGINEERING↗

DOE OSTI2025

Harnessing the Power of AI: Status and Expansion of Current Domestic Transport Security Through Flexible Embedded Hardware

As applications of Artificial Intelligence (AI) continue to expand, there are increasing opportunities to leverage applied AI methodologies with mobile transportation focused embedded systems. Current applications of AI in transportation focus on a variety of areas, including fuel efficiency, safety, security, and other broad fields of optimization or detection. To leverage these AI workflows and methodologies in the field, teams must utilize complex embedded systems capable of implementing these AI-enabled algorithms in real-time. In this paper, we will investigate how these algorithms can be integrated into existing technologies leveraging vehicle data - such as the Controller Area Network Transport Security Tracking and Reporting Unit (C-STAR). The C-STAR technology is an embedded platform with onboard computation capable of running next generation algorithms in vehicle systems AI, such as preventative maintenance, driver authentication, and transport security. As deployed in the field, the C-STAR has a limited AI functionality –this paper will directly discuss how a device like C-STAR can be utilized and the advantages of integrating these new technologies. We will open with relevant background information and transportation projects that leverage AI, focusing specifically on those around transport security such as vehicle identification, anomaly detection, and deterrence. We will then extend this into potential opportunities and scaling for AI methodologies using platforms like the C-STAR. Finally, we will speak directly to the challenges of deploying AI-powered workflows, such as computing power needs, bandwidth, hallucinations, and other regulatory considerations.

Cook, Adian [ORNL] (ORCID:0000000160825395)↗

DOE OSTI2023

Moving-horizon false data injection attack design against cyber–physical systems

Systematic attack design is essential to understanding the vulnerabilities of cyber–physical systems (CPSs), to better design for resiliency. In particular, false data injection attacks (FDIAs) are well-known and have been shown to be capable of bypassing bad data detection (BDD) while causing targeted biases in resulting state estimates. However, their effectiveness against moving horizon estimators (MHE) is not well understood. In fact, this paper shows that conventional FDIAs are generally ineffective against MHE. One of the main reasons is that the moving window renders the static FDIA recursively infeasible. Here, this paper proposes a new attack methodology, moving-horizon FDIA (MH-FDIA), by considering both the performance of historical attacks and the current system’s status. Theoretical guarantees for successful attack generation and recursive feasibility are given. Numerical simulations on the IEEE-14 bus system further validate the theoretical claims and show that the proposed MH-FDIA outperforms state-of-the-art counterparts in both stealthiness and effectiveness. In addition, an experiment on a path-tracking control system of an autonomous vehicle shows the feasibility of the MH-FDIA in real-world nonlinear systems.

42 ENGINEERING↗

DOE OSTI2022

Recommendations for Secure Transport: Material Conveyance Physical Protection Technology

Nuclear material is at higher risk of theft and sabotage during transport than during any other phase of the nuclear fuel lifecycle. Nuclear materials are transported in the public domain where adversaries have an upper hand by taking advantage of the time and location of the theft or sabotage attempt. As such, even modest threat profiles for transport of nuclear and radioactive material can require substantial detection and delay measures to support timely response. Conveyance tracking augmented with technology that improves on-the-scene situational awareness at a remote monitoring center has been adopted as a de-facto standard approach for transportation security. At present, the extended tracking and situational awareness capabilities needed for a nuclear material shipment, as is provided by the purpose designed, Transportation – Security, Tracking and Reporting (T-STAR) System, do not exist in a single commercial off-the-shelf (COTS) solution. Typically, the COTS systems that excel in one area are deficient in other areas, presenting challenges to designing well-rounded, robust systems. Still, COTS solutions can offer the basic set of tracking and situational awareness capabilities by indicating last update time, current location, and route taken. By incorporating vehicle and driver performance measures via the vehicle’s controller area network (CAN bus) and video alongside other data streams allows telemetry and other shipment information to be assessed in novel ways.This paper describes the operation, capabilities and features of various conveyance protection systems and approaches, assesses emerging technologies and how they could be used through a unified interface, and enumerates additional ways to provide early detection using vehicle, onboard technologies, effective delay technologies and approaches and simple equipment to improve protection during transport.

Shannon, Michael↗

DOE OSTI2026

Automatic Calibration and Health Monitoring of Infrastructure Sensors

Smart transportation infrastructure relies on networks of heterogeneous sensors - cameras, radars, and lidars - continuously monitoring traffic conditions. However, executing the initial spatial calibration of multiple sensors and the subsequent health monitoring presents significant operational challenges. Environmental factors, mechanical vibrations, and gradual drift cause spatial misalignment, degrading fusion performance and tracking accuracy. Traditional calibration approaches require manual intervention with specialized targets or survey equipment, resulting in service interruptions and high maintenance costs. This work presents an automated framework for initial calibration and continuous health monitoring without human intervention or service disruption. Our approach addresses two critical problems: (1) detecting when sensors become miscalibrated during operation, and (2) automatically re-establishing spatial alignment using only operational traffic data. The health monitoring component analyzes measurement innovations - differences between sensor observations and predicted object states - to detect systematic biases indicative of calibration drift. By computing bias magnitude, directional consistency, and rejection rates, the system identifies miscalibrations as small as 0.5 meters. Unlike traditional methods requiring known calibration targets, our diagnostic operates continuously on live traffic observations, enabling early detection before fusion quality degrades. The automatic recalibration algorithm leverages overlapping sensor fields-of-view and temporal correlation of vehicle observations. Using graph-based optimization, the system automatically discovers which sensor pairs observe common regions, estimates pairwise spatial transformations using RANSAC-based robust estimation, and jointly optimizes all sensor poses through bundle adjustment. The framework handles practical deployment challenges, including different sensor sampling rates (1-10 Hz), varying installation positions, unknown orientations, and limited overlap regions (>10%). When approximate sensor positions are available from installation surveys (+/-1m accuracy), the algorithm additionally estimates sensor orientations, refining both position and rotation to sub-meter and sub-degree accuracy. We validate the framework on multi-hour traffic datasets from six heterogeneous sensors with sampling rates ranging from 1 Hz to 10 Hz. Results demonstrate successful calibration even with sparse overlap (<20%) and automatic detection of miscalibrations exceeding 0.8 meters. This work enables a "deploy-and-forget" sensor infrastructure that maintains calibration autonomously, reducing maintenance costs while improving tracking accuracy. The techniques generalize beyond transportation to any multi-sensor monitoring application requiring robust spatial alignment, including smart cities, industrial monitoring, and surveillance systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2020

Data-based and secure switched cyber–physical systems

In this work, we develop a completely model-free moving target defense framework for the detection and mitigation of sensor and/or actuator attacks in cyber–physical systems with dynamics that evolve in discrete-time. We incorporate an intrusion detection mechanism based on an approximate dynamic programming technique that learns the policies for optimal regulation and optimal tracking while simultaneously defending against actuator and sensor attacks in a model-free fashion. Switching rules are leveraged to force proactive and reactive defense mechanisms as well as, guarantee the stability of the equilibrium point. Finally, as a case study, we apply the proposed moving target defense framework to a DC–DC converter that is used in electric vehicles.

42 ENGINEERING↗

DOE OSTI2025

The Sensor Dilemma in Intelligent Transportation Systems

Intelligent Transportation Systems (ITS) are at the forefront in advancing the way we interact and perceive with the transportation network. This revolution is fueled by the significant advancement in sensor perception technologies such as Radar, LiDAR and Video Imaging which are the most popular modalities for ITS. Real-time perception data from these sensors allows intelligent infrastructure side decision making to improve the energy, efficiency and safety at traffic intersections. As traffic departments across the United States are transitioning from traditional loop detectors / emulators and embracing newer technologies, they are often left with a dilemma in choosing a sensor technology for infrastructure-based perception which is reliable, inexpensive, easy to setup and has robust performance in varying weather conditions. However, choosing a sensor which checks all boxes is not straightforward as every sensor type has unique benefits and drawbacks. Radar is excellent at detecting long range vehicles and weather resistance but lacks high resolution. LiDAR is expensive and weather-sensitive, while cameras provide rich visual data at a low cost but are constrained by lighting and visibility. This study examines Radar, LiDAR and camera sensors capabilities to ascertain whether any of these qualifies as the "best" sensor for ITS perception. Through this evaluation, we hope to draw attention to the necessity of National Renewable Energy Laboratory's (NREL) Infrastructure Perception and Control (IPC) framework which presents a multi-sensor track data fusion engine to assimilate multiple data streams in order to provide robust and reliable perception. While no single sensor can meet all the demands of ITS, a hybrid approach combining multiple sensor modalities like Radar, LiDAR and cameras, offers the most robust solution for enhancing the safety and efficiency in intelligent transportation systems.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2024

Evaluating the Effectiveness of a Detection and Deterrent System in Reducing Golden Eagle Fatalities at Operational Wind Facilities

The Renewable Energy Wildlife Institute (REWI) was appointed as the prime awardee of DOE award number DE-EE0007883 to lead a team of scientists, wind developers, and technology manufacturers toward the overarching goal of evaluating the effectiveness of the current DTBird system in minimizing the risk of golden eagles (Aquila chrysaetos) and other large soaring raptors from approaching the rotor-swept zone (RSZ) of operating wind turbines. As part of this goal, the team set out to 1) quantify the expected reduction in collision risk for golden eagles from operation of the detection and deterrence modules in a manner that supports the approach used by the U.S. Fish and Wildlife Service (USFWS) to assess and credit facility operators for their efforts to minimize predicted collision fatalities and 2) provide information to help improve the technology to maximize its effectiveness. DTBird is an automated detection and audio deterrent system created by the Spanish company Liquen, designed to discourage birds from entering the RSZ of spinning wind turbines. The system uses cameras to automatically detect airborne targets of interest, records each such event in an online database, and triggers a warning signal (loud sound) if the tracked object has moved close to the turbine. If the object moves even closer to the RSZ, a more aggressive dissuasion signal is broadcast. To meet our objectives, the team conducted a two-year experiment at the Goodnoe Hills wind facility in Washington state, in which 14 turbines were outfitted with DTBird units. Daily, each DTBird-equipped turbine was randomly assigned to a control or treatment group. Treatment turbines operated with DTBird running as intended—broadcasting warning or deterrent signals when DTBird detected a target within range. On control turbines, no sound signals were broadcast if a moving target triggered the DTBird system. The team also flew unmanned aerial vehicles (UAVs) designed to coarsely mimic the general size, weight, and coloration of golden eagles in programmed flight transects across DTBird detection ranges to quantify DTBird’s ability to detect intended targets and to evaluate factors that influence the probability of detection and DTBird’s response distances. Additionally, the team evaluated the behavioral responses of in situ eagles exposed to spinning turbines alone (visual and sound influences) versus spinning turbines plus broadcasted DTBird audio deterrents, to estimate the effectiveness of deterrence by the DTBird system. The data and results from these investigations were combined with those from a pilot study conducted at the Manzana Wind Power Project in California to better evaluate DTBird’s effectiveness across different landscapes.

17 WIND ENERGY↗

DOE OSTI2026

IPC-Fusion (Infrastructure Perception and Control (IPC): Multisensor Data Fusion Software) [SWR-25-153]

As part of the National Laboratory of the Rockies' (NLR’s) Infrastructure Perception and Control Laboratory, the IPC-Fusion toolkit provides a probabilistic, scalable, multi-sensor fusion framework that integrates (late-stage fusion) heterogeneous object detection data from traffic sensors to enable robust, real-time tracking of roadway occupants. The algorithmic design of the toolkit is motivated by the need for creating a digital twin of traffic at the edge in a scalable and affordable manner. The software operates by combining object-level measurements (such as position and velocity) from a suite of sensors (such as radar, lidar, camera) using Kalman filtering and probabilistic data association techniques to overcome individual sensor limitations and achieve superior tracking performance in complex traffic zones. The framework addresses key challenges including heterogeneous measurement uncertainties, asynchronous data streams, varying spatiotemporal data resolutions, robust data association, and adaptive object lifecycle management. Validated on real-world traffic intersection data including vehicles and pedestrians, IPC-Fusion demonstrates enhanced tracking reliability across scenarios involving occlusions, sensor failures, and varying traffic densities, supporting the broader IPC initiative's goal of transforming transportation infrastructure through advanced perception capabilities for intelligent transportation systems, traffic safety applications, and autonomous vehicle support.

Sandhu, Rimple [National Laboratory of the Rockies↗

DOE OSTI2024

Live-Cell Quantum Dot-Based Tracking of Plant & Microbial Extracellular Vesicles

All cellular organisms examined to date release various types of small vesicles called extracellular vesicles (EVs). They were originally thought to be unimportant, but research into EVs as carriers of key messages for communication within large, multicellular organisms or between interacting organisms has gained interest in many different scientific areas of study. Recently, it was discovered that both plants and their pathogenic fungi generate EVs and they are important vehicles for sending plant signals to thwart invasion by fungal pathogens, and in response, fungi use EVs to help evade plant defenses. This project aimed to develop new ways to visualize EVs and their potential nucleic acid cargo isolated from the crop plant, sorghum. We adapted a click chemistry-based method for making quantum dot-based RNA sensors, which will be a useful tool for others studying small RNAs. These sensors were not used detect RNA in sorghum EVs because unlike EVs from Arabidopsis, sorghum EVs were not enriched for specific small RNAs. This interesting difference shifted the project in an exciting direction, which focused on comparing EV proteomes. Our study discovered that greater than 60% of Arabidopsis EV proteins were also found in the sorghum EV proteome, revealing a partial conservation between a eudicot and monocot plant. These studies have raised new, fundamental questions on the conserved function of EVs in plants.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

The Sensor Dilemma in Intelligent Transportation Systems: Evaluating Radar, Lidar and Camera: Preprint

Intelligent transportation systems (ITS) are at the forefront in advancing the way we interact with and perceive the transportation network. This revolution is fueled by the significant advancement in sensor perception technologies such as radar, lidar, and video imaging, which are the most popular modalities for ITS. Real-time perception data from these sensors allow intelligent infrastructure-side decision-making to improve the energy, efficiency, and safety at traffic intersections. As traffic departments across the United States transition from traditional loop detectors and emulators and embrace newer technologies, they are often left with a dilemma in choosing a sensor technology for infrastructure-based perception that is reliable, inexpensive, and easy to set up and that has robust performance in varying weather conditions. However, choosing a sensor that checks all these boxes is not straightforward, as every sensor type has unique benefits and drawbacks. Radar is excellent at detecting long-range vehicles and weather resistance but lacks high resolution. Lidar is expensive and weather-sensitive, while cameras provide rich visual data at a low cost but are constrained by lighting and visibility. This study examines radar, lidar, and camera sensor capabilities to ascertain whether any of these qualifies as the "best" sensor for ITS perception. While no single sensor can meet all the demands of ITS, a hybrid approach combining multiple sensor modalities like radar, lidar, and cameras offers the most robust solution for enhancing the safety and efficiency of ITS. Through this evaluation, we hope to draw attention to the necessity of the National Renewable Energy Laboratory's infrastructure perception and control framework, which presents a multisensor track data fusion engine to assimilate multiple data streams in order to provide robust and reliable perception.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2025

High-Resolution Sampling of a River Plume Front with Uncrewed Underwater and Aerial Vehicles

Sampling fast-propagating oceanic features is inherently challenging and demands versatile instrumentation and innovative strategies. This paper introduces a novel sampling strategy designed to capture such phenomena, exemplified by a river plume front. Our method revolves around modifying the preprogrammed pathway of an uncrewed underwater vehicle (UUV) to dynamically track and three-dimensionally sample the evolution of the front. To enable the UUV to follow the feature, we adapt the use of a drifting gateway buoy to be positioned and trapped at the front’s convergence zone, allowing underway navigation relative to the buoy. In our demonstration, we showcase the effectiveness of this strategy by successfully conducting over 30 crossings of a river plume front within a 6-h window. The UUV sensors allowed a comprehensive assessment of key front characteristics, including density, velocity, and turbulence. Supplemental drone footage contributed to the overall picture and facilitated the transformation of the dataset into a front-following reference frame. This article provides an in-depth description of the deployment strategy and required postcollection data processing, including frontal crossing detection, the assessment of the frontal orientation from drone footage, and defining the plume bottom boundaries using backscatter intensity contours.

autonomous observations↗

DOE OSTI2024

Infrastructure-Based Cooperative Perception at a Traffic Intersection: Overview and Challenges: Preprint

Recent advancement in autonomous driving vehicles and V2X communication has attracted increasing attention towards Intelligent Transportation Systems to build a safe and reliable traffic intersection. However, most of the systems are still at the initial stages and require significant progress to become a reality. This paper presents an overview of NREL Infrastructure Perception and Control (IPC) framework which is an open-source track-data fusion engine which takes input from infrastructure-based perception sensors and cooperatively shared messages from Connected Autonomous Vehicles (CAVs) and Connected Vehicle (CVs) and the challenges associated with deploying such cooperative perception framework at a four-way traffic intersection in the city of Colorado Springs, CO, USA. The sensor data is collected by deploying two radars and two LiDAR sensors on the IPC mobile lab and two radars on diagonally opposite traffic poles at the proposed intersection. The sensor output results imply the need for rapid sensor calibration to bring the collective perception to a common coordinate frame, the importance of time synchronization between the sensors in order to capture accurate spatial and temporal alignment of the objects, and the need for a health monitoring system with fail safe closed-loop detection model for real-time deployment.

camera↗

DOE OSTI2025

Vertical column dual-comb spectroscopy to a TBS

Open-path dual-frequency-comb spectroscopy (DCS) is a broadband, high spectral resolution, and high precision method for measuring gas concentrations over kilometer-scale paths. It has been used for detection and quantification of emissions of pollutants, hazardous gasses, and greenhouse gasses (GHGs). DCS has been shown to measure trace-gas mixing ratios with 0.14%-0.4% agreement between instruments. To achieve high signal-to-noise ratios (SNRs) with DCS, comb light is targeted onto a retroreflector at the end of the measurement path, which returns the signal to a detector co-located with the launch signal. Installing the retroreflector on a mobile platform such as a balloon or unmanned aerial vehicle (UAV) extends the capabilities of DCS by enabling variable path lengths, greater mobility, and access to higher altitudes. Mobile-target DCS has many uses in plume and leak detection, emissions modeling, and planetary boundary layer (PBL) studies. It is a promising method for observing vertical distributions of GHGs and mixing processes in the PBL, which are difficult to measure but important for pollution and climate monitoring as well as for understanding transport of gasses through the atmosphere. Tethered balloons are an intriguing platform because they enable longer flight durations and higher altitudes than easily obtainable with a UAV and thus allow for column measurements up to and above the PBL. New measurements completed in October/November 2024 reach the highest altitudes above ground level yet achieved by mobile-target DCS. For these measurements, the retroreflector was mounted on a 7 m diameter tethered helium balloon. An actively tracking gimbal on the ground holds the DCS launch telescope and keeps the 5 cm beam pointed onto the retroreflector while the balloon is lifted, lowered, and moved by wind and turbulence.

atmosphere↗

DOE OSTI2021

Study of Hydrogen Fuel Cell Technology for Freight Rail Propulsion and Review of Relevant Industry Standards.

Alternatives to conventional diesel electric propulsion are currently of interest to rail operators. In the U.S., smaller railroads have implemented natural gas and other railroads are exploring hydrogen technology as a cleaner alternative to diesel. Diesel, battery, hydrogen fuel cell, or track electrification all have trade-offs for operations, economics, safety, and public acceptability. A framework to compare different technologies for specific applications is useful to optimize the desired results. Standards from the Association of American Railroads (AAR) and other industry best practices were reviewed for applicability with hydrogen fuel cell technology. Some technical gaps relate to the physical properties of hydrogen, such as embrittlement of metals, invisible flames, and low liquid temperatures. A reassessment of material selection, leak/flame detection, and thermal insulation methods is required. Hydrogen is less dense and diffuses more easily than natural gas, and liquid hydrogen is colder than liquefied natural gas. Different densities between natural gas and hydrogen require modifications to tank designs and flow rates. Leaked hydrogen will rise rather than pool on the ground like diesel, requiring a modification to the location of hydrogen tanks on rolling stock. Finally, the vibration and shock experienced in the rail environment is higher than light-duty vehicles and stationary applications for which current fuel cell technology has been developed, requiring a modification in tank design requirements and testing.

08 HYDROGEN↗

DOE OSTI2024

Automated Inspection of Criticality Control Overpacks for Surplus Plutonium Disposition: Qualification Update – 25313

In an effort to reduce the amount of nuclear waste in South Carolina, the Department of Energy (DOE) tasked the Savannah River Site (SRS) with diluting and disposing of the amount of plutonium in the state. This process involves the movement and shipment of over 100,000 criticality control overpacks (CCOs) throughout the project lifespan, lending itself to the use of automation to reduce worker radiation exposure and more efficiently utilize human capital. Due to the large scope, this overarching process was broken down into several different “automation projects” to be developed. The first opportunity pursued was the receipt and inspection of empty CCO drums coming into SRS, identified as Automation Project 1 (AP1), and is the focus of this paper. AP1 was developed to unpack incoming CCOs and inspect them for unwanted foreign objects and any damage to the drum or its contents. This process is accomplished by the combination of an automated guided vehicle (AGV) that delivers CCOs to a robotic arm which uses a suite of custom tools to disassemble a CCO, inspect the inside and outside of the CCO and its inner criticality control container (CCC), reassemble the CCC and CCO, and apply a tamper indicating device (TID) to the inspected drum. In past years, the robotic work cell had been developed in a small-scale testing facility for proof-of-concept. This year, major improvements were made to the robotic work cell to perform the process, including integration into the final facility where CCOs will be inspected. Other technical improvements include the implementation of sensor feedback and safety relays into the control system to allow the state of the work cell to be better tracked, and additional development of the TID application process to complete the robotic inspection. Further enhancements were made to the robotic vision processes and robot pathing, as well as development on a computer vision inspection process to detect inspection criteria anomalies in CCOs. In addition to developmental improvements, the work cell underwent a six-month testing period to ensure the project requirements were met. Results of this testing period demonstrate the work cell’s capability to meet project throughput goals at an acceptable level, successfully document the status of each CCO inspected, and reduce the toll on technical operations’ human power by two thirds. At the time of this paper, the work cell is capable of autonomously handling up to eight CCOs with an AGV, delivering CCOs to and from the robot work cell, and having a robotic arm perform a full receipt and inspection procedure on each CCO. Moving forward, repeatability will be improved so that these CCOs can be run back-to-back seamlessly, as well as improving the system to handle more significant edge cases and failure modes.

Spivey, Nicholas↗

DOE OSTI2021

Using Machine Learning to Track Objects Across Cameras

Video surveillance is one of the most important technologies used by the International Atomic Energy Agency in international safeguards. At large, complicated facilities, multiple surveillance cameras are deployed to monitor the transfer of safeguards-relevant objects across the site. During inspections, all surveillance videos are reviewed to ensure the objects are not manipulated or diverted during transfer, a laborious, time-consuming task. This work describes using deep machine learning algorithms to track objects automatically across multiple cameras, greatly improving the efficiency of the review process. The fundamental problem in this object tracking task across multiple cameras is how to associate the same object, which may show extreme intra-class variations, such as viewpoints, occlusions, and various scales, in different and even non-overlapped cameras. Object re-identification (Re-ID) in nuclear facility video surveillance is even more challenging than classic person or vehicle Re-ID problems because different instances in the same category may display an identical appearance. One observation from nuclear facility surveillance videos is that all objects must be carted (e.g., via forklift) to move. Therefore, the spatial context information of an object, which provides the feature from the carrier, is critical for the object Re-ID task. This work proposes a two-stream convolutional neural networks model that takes features of objects and their surrounding regions into account. Moreover, the custom videos usually are gleaned from different scenes from the training data, which may have extreme variations in illumination changes and/or cluttered backgrounds. Directly applying the trained model to custom videos will dramatically decrease the performance. To tackle this problem, an advanced domain adaptation technique is proposed to mitigate the gap between the data taken from different scenes. The proposed framework will track objects of interest across a nuclear complex. The resulting tracks can be used in further analyses, such as event/activity recognition, anomaly detection, etc.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2022

Profile Images and Annotations for Vehicle Re-identification Algorithms (PRIMAVERA)

This dataset contains 636,246 profile images of vehicles representing 13,963 unique vehicles. The data was collected by a set of roadside sensors over the course of three years. Each time a vehicle passed by one of the sensors, a series of images was collected. The images were processed to detect and localize each vehicle, and a license plate reader collocated with the sensor was used to provide a unique ID for the vehicle. Actual license plate numbers have been obfuscated by replacing with an arbitrary numerical ID for each vehicle. After localizing the vehicle in each image, the original RGB image was rotated, scaled, and shifted to produce a new RGB image of size 234x234 pixels such that the outermost two wheels are located at predetermined pixel locations in the image. In this way, all vehicle images are aligned to one another. This registration process occasionally results in a portion of certain vehicles being cutoff at the edges of the image. The dataset has been partitioned into two sets called training and validation. The two partitions no common vehicles, i.e., a vehicle present in one partition is guaranteed not to be present in the other. In this way, an algorithm can be validated against a set of new vehicles that were not seen during the training process. The training set contains 543,926 images from 64,440 vehicle passes representing 11,918 unique vehicles, while the validation set contains 92,320 images from 10,991 vehicle passes representing 2,045 unique vehicles. Vehicle images are organized by directories corresponding to unique vehicles. The file naming scheme is as follows: veh_{vehID}_tr_{passID}_{frameID}_{elevation}_{timeofday}.jpg where {vehID} is the vehicle ID (unique across the entire dataset), {passID} is an identifier for each tracked vehicle pass (unique across the entire dataset), {frameID} is the index of the frame within the given vehicle pass starting at 0, {elevation} is a two-letter string indicating whether the sensor was elevated (el) or at ground-level (gl), and {timeofday} is a two-letter string indicating whether the image was captured during daytime (dt) or nighttime (nt).

image↗