Engineering topics
Publications and source records attributed to Matzner, Shari.
This dataset contains ThermalTracker-3D flying fauna trajectories saved in json files, along with camera orientation and GPS location data. The 3D trajectories are in x,y,z camera space coordinates. They can be converted to geo-referenced coordinates using the orientation and GPS data.
Explore the source record for details and available documents.
This dataset contains ThermalTracker-3D flying fauna trajectories saved in json files, along with camera orientation and GPS location data. The 3D trajectories are in x,y,z camera space coordinates. They can be converted to geo-referenced coordinates using the orientation and GPS data.
Planning is underway for placement of infrastructure needed to begin offshore wind (OSW) energy generation along the West Coast of the United States and elsewhere in the Pacific Ocean. In contrast to the primarily nearshore windfarms currently in the North Atlantic, the seabird communities inhabiting Pacific Wind Energy Areas (WEAs) include significant populations of species that fly by dynamic soaring, a behavior dependent on wind and in which flight height increases steeply with wind speed. Therefore, a more precise and detailed assessment of their 3D airspace use is needed to better understand the potential collision risks that OSW turbines may present to these seabirds. Toward this end, a novel technology called the ThermalTracker-3D (TT3D), which uses thermal imaging and stereo vision, was developed to render high-resolution (on average within ±5 m) flight tracks and related behavior of seabirds. The technology was developed and deployed on a wind-profiling LiDAR buoy in the Humboldt WEA, located 34 to 57 km off California’s coast. During the at-sea deployment between 24 May and 13 August 2021, the TT3D successfully tracked birds moving between 10 and 500 m from the device, around the clock, and in all weather conditions; a total of 1407 detections and their corresponding 3D flight trajectories were recorded. Mean altitudes of detections ranged 6-295 m above sea level (asl). Considering the degree of overlap with anticipated rotor swept zones (RSZ), which extend 25-260 m asl, 79% of detected birds (per m 3 of airspace) moved below the RSZ, 21% moved at heights overlapping the RSZ, and another 0.04% occurred at heights exceeding the RSZ. The high-resolution tracks provided valuable insight into seabird space use, especially at heights that make them vulnerable to collision during various environmental conditions (e.g., darkness, strong winds). Observations made by the TT3D will be useful in filling critical knowledge gaps related to estimating collision and avoidance between seabirds and OSW facilities in the Pacific and elsewhere. Future research will focus on enhancing the TT3D’s identification capabilities to the lowest taxon through validation studies and artificial intelligence, further contributing to seabird conservation efforts associated with OSW.
ThermalTracker-3D is a stereo-vision solution for evaluating flight tracks of birds and bats around offshore wind turbines. Using a pair of thermal video cameras, the technology remotely senses movement of animals and objects, day and night, near critical assets. It generates motion tracks by collapsing a sequence of video frames from each camera into a single image that contains an entire flight track and then applies stereo-vision processing to transform the flight track into three dimensions. The approach allows tracking in near real time and automatically identifies moving objects based on features from the motion track and object size.
Described herein are sensing methods, sensor systems, and non-transitory, computer-readable, storage media having programs for long-duration, continuous monitoring of flying objects during the day or the night and regardless of weather conditions. The methods and systems are computationally efficient and can provide compact, three-dimensional representations of motion from the observed object. A 3D track of the flying object can be generated from a point-matched pair of stereo composite motion track images and not directly from the videos, wherein each composite motion track image is based on a composite of a plurality of video frames composited in part according to video frame numbers.
The “Intelligent Adaptable Monitoring Package” project was a four-year effort that demonstrated the feasibility of integrated sensing packages at tidal and wave energy sites. Such integration is generally required by the breadth of sensors required to understand environmental effects at marine energy sites and the operational difficulty of deploying, maintaining, and recovering such sensors. Over the course of the project, the Adaptable Monitoring Package (AMP) was deployed in multiple settings, each corresponding to a project budget period: - Budget Period 1: Demonstration of cabled deployment at Pacific Northwest National Laboratory’s Marine Science Laboratory. The deployment highlighted AMP hardware endurance over a 4-month deployment in a tidally-dominated environment and laid the groundwork for machine learning algorithms to detect and classify targets present in active sonar data. - Budget Period 2: Demonstration of an autonomous deployment at PacWave South off the coast of Newport, Oregon. The deployment highlighted the stability of AMP hardware and software, with the autonomous package collecting data on a duty cycle over a 1.5-month deployment. - Budget Period 3: Demonstration of an autonomous deployment powered by a wave energy converter at the U.S. Navy’s Wave Energy Test Site. The deployment highlighted the potential of wave energy to power ocean observatories and led to the development of machine learning algorithms to detect and classify targets in optical camera data. In aggregate, this project’s greatest success was demonstrating the AMP’s flexibility in a range of deployment scenarios. Each budget period represented a “first of a kind” demonstration of integrated instrumentation – cabled AMP, autonomous AMP, wave-energy powered AMP – and each deployment helped to identify and set goals for the next. Further, despite the exploratory nature of these deployments, each one achieved high system up-time and proved that flexible integration of multiple sensors in a single package represents a viable strategy for marine energy environmental monitoring. The key lessons learned from the project are: - Without continuous power, either from a shore cable or in situ source, many of the benefits of integration are lost (If continuous power is not available, the ability to detect rare events is lost, as is the ability to minimize the risk of behavioral changes through adaptive sensing. However, even on a duty cycle, there is still value in being able to acquire synchronous data from multiple sensors.); and - Observations from a moving platform present substantially greater data processing challenges than those from stationary platforms. Finally, these deployments also demonstrate an important truth: successful integration alone does not guarantee that relevant data are collected. To grow the knowledge base about environmental interactions with marine energy converters, integrated systems, like the AMP, need to include the right sensor mix and connect the data pipelines to effective processing algorithms. These deployments establish a strong foundation for future collaborations with the environmental research community: not only to understand the environmental effects of marine energy, but also to improve our general ability to study life in the sea.
Clean energy from oceans and rivers is becoming a reality with the development of new technologies like tidal and instream turbines that generate electricity from naturally flowing water. These new technologies are being monitored for effects on fish and other wildlife using underwater video. Methods for automated analysis of underwater video are needed to lower the costs of analysis and improve accuracy. We have trained a deep learning model, YOLO, to recognize fish in underwater video using three very different datasets recorded at real-world water power sites. Training and testing with examples from all three datasets resulted in a mean average precision (MAP) score of 0.5392. To test how well a model could generalize to new datasets, the model was trained using examples from only two of the datasets and then tested on examples from all three datasets. The resulting model could not recognize fish in the dataset that was not part of the training set. The MAP scores on the other two datasets that were included in the training set were higher than the scores achieved by the model trained on all three datasets. These results indicate that different methods are needed in order to produce a trained model that can generalize to new data sets such as those encountered in real world applications.
This code can be used to generate a damage map based on the presence of rubble. Rubble is often detectable in images with submeter resolution, and can indicate damage to structures from high winds or earthquakes. The process_images.py script outputs a mask indicating the pixels in the original image where rubble was detected. The mask can be used in a geospatial software application like ArcMap to generate a damage map based on the rubble density. The rubble detection mask will contain some false positives which are usually more isolated than true rubble, so a density map will provide a more accurate indication of affected areas than the indivdidual rubble detections will.
ThermalTracker tracks birds and bats in thermal video. It will process video in any standard format that was recorded with a thermal video camera. The output is images of the flight tracks of birds and bats in PNG format, and a comma-separated value (CSV) file listing the details of the extracted flight tracks.