An Anti-Occlusion Approach for Enhanced Unmanned Surface Vehicle Target Detection and Tracking with Multimodal Sensor Data

Abstract

Multimodal sensors are often employed by USVs (unmanned surface vehicles) to enhance situational awareness, and the fusion of LiDAR and monocular vision is widely used in near-field perception scenarios. However, this strategy of fusing data from LiDAR and monocular vision may lead to the incorrect matching of image targets and LiDAR point cloud targets when targets occlude one another. To address this issue, a target matching network with an attention module was developed to process occlusion information. Additionally, an image target occlusion detection branch was incorporated into YOLOv9 to extract the occlusion relationships of the image targets. The introduction of the attention module and the occlusion detection branch allows for the consideration of occlusion information in matching point cloud and image targets, thereby achieving more accurate target matching. Based on the target matching network, a method for water surface target detection and multi-target tracking was proposed. This method fuses LiDAR point cloud and image data while considering occlusion information. Its effectiveness was confirmed through experimental verification. The experimental results show that the proposed method improved the correct matching rate in complex scenarios by 13.83% compared to IoU-based target matching methods, with an MOTA metric of 0.879 and an average frame rate of 21.98. The results demonstrate that the method effectively reduces the mismatch rate between point cloud and image targets. The method’s frame rate meets real-time requirements, and the method itself offers a promising solution for unmanned surface vehicles (USVs) to perform water surface target detection and multi-target tracking.