The ability to semantically interpret 3D scenes is important for accurate 3D perception and scene understanding in tasks like robotic grasping, scene-level robot navigation, or autonomous driving. However, there is currently no large-scale photorealistic 3D point cloud dataset available for fine-grained semantic understanding of urban scenarios.
Photogrammetric point could datasets are important for tasks such as robotic grasping, scene-level robot navigation, or autonomous driving. Image credit: Pxhere, CC0 Public Domain
A recent paper published on arXiv.org builds a UAV photogrammetric point cloud dataset for urban-scale 3D semantic understanding.
The dataset covers 7.6 km2 of urban areas along with nearly 3 billion richly annotated 3D points. A comprehensive benchmark for semantic segmentation of urban-scale point clouds is provided together with experimental results of different state-of-the-art approaches.
The results reveal several challenges faced by existing neural pipelines. Therefore, the researchers provide an outlook of the future directions of 3D semantic learning.
With the recent availability and affordability of commercial depth sensors and 3D scanners, an increasing number of 3D (i.e., RGBD, point cloud) datasets have been publicized to facilitate research in 3D computer vision. However, existing datasets either cover relatively small areas or have limited semantic annotations. Fine-grained understanding of urban-scale 3D scenes is still in its infancy. In this paper, we introduce SensatUrban, an urban-scale UAV photogrammetry point cloud dataset consisting of nearly three billion points collected from three UK cities, covering 7.6 km^2. Each point in the dataset has been labelled with fine-grained semantic annotations, resulting in a dataset that is three times the size of the previous existing largest photogrammetric point cloud dataset. In addition to the more commonly encountered categories such as road and vegetation, urban-level categories including rail, bridge, and river are also included in our dataset. Based on this dataset, we further build a benchmark to evaluate the performance of state-of-the-art segmentation algorithms. In particular, we provide a comprehensive analysis and identify several key challenges limiting urban-scale point cloud understanding. The dataset is available at this http URL.
Research paper: Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., and Markham, A., “SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds”, 2022. Link: https://arxiv.org/abs/2201.04494