broadbrained Space, news, venture, robots, technology
3D pedestrian detection is one of the most difficult tasks in autonomous driving. It typically uses two data acquisition sources: LiDAR and camera. Combining these two sensors should be beneficial; however, improper fusion methods lead to worse results than LiDAR-only detection.
A recent paper on arXiv.org proposes a method that is able to apply fusion at different levels. It encodes semantic features obtained from the image in voxels and fuses them with geometric features from the LiDAR point cloud. The fusion can be performed at different levels; for example, early fusion combines information in the input stage, and in the late fusion specific features from both inputs are learned and fused before the prediction. The suggested method outperformed current detection methods during experiments. Moreover, it was effective in challenging cases, for instance, when a pedestrian walks out suddenly in front of parked cars.
3D pedestrian detection is a challenging task in automated driving because pedestrians are relatively small, frequently occluded and easily confused with narrow vertical objects. LiDAR and camera are two commonly used sensor modalities for this task, which should provide complementary information. Unexpectedly, LiDAR-only detection methods tend to outperform multisensor fusion methods in public benchmarks. Recently, PointPainting has been presented to eliminate this performance drop by effectively fusing the output of a semantic segmentation network instead of the raw image information. In this paper, we propose a generalization of PointPainting to be able to apply fusion at different levels. After the semantic augmentation of the point cloud, we encode raw point data in pillars to get geometric features and semantic point data in voxels to get semantic features and fuse them in an effective way. Experimental results on the KITTI test set show that SemanticVoxels achieves state-of-the-art performance in both 3D and bird’s eye view pedestrian detection benchmarks. In particular, our approach demonstrates its strength in detecting challenging pedestrian cases and outperforms current state-of-the-art approaches.
Link: https://arxiv.org/abs/2009.12276
