The overall pipeline of geometric flow network (GFNet).
Point cloud semantic segmentation from projected views, such as range-view (RV) and bird's-eye-view (BEV), has been intensively investigated. Different views capture different information of point clouds and thus are complementary to each other. However, recent projection-based methods for point cloud semantic segmentation usually utilize a vanilla late fusion strategy for the predictions of different views, failing to explore the complementary information from a geometric perspective during the representation learning.
In this paper, we introduce a geometric flow network (GFNet) to explore the geometric correspondence between different views in an align-before-fuse manner. Specifically, we devise a novel geometric flow module (GFM) to bidirectionally align and propagate the complementary information across different views according to geometric relationships under the end-to-end learning scheme. We perform extensive experiments on two widely used benchmark datasets, SemanticKITTI and nuScenes, to demonstrate the effectiveness of our GFNet for project-based point cloud semantic segmentation. Concretely, GFNet not only significantly boosts the performance of each individual view but also achieves state-of-the-art results over all existing projection-based models.
Here we provide some visualizations. First, we show a gif of the driving scene with RV maps; then the information flowing between RV and BEV leads to mutual help; finally the comparisons between predictions and gt are given.
BEV helps to handle the distant occluded points caused by RV projection.
The view with the cyan contour helps the red one.
@article{qiu2022gfnet,
title={{GFN}et: Geometric Flow Network for 3D Point Cloud Semantic Segmentation},
author={Haibo Qiu and Baosheng Yu and Dacheng Tao},
journal={Transactions on Machine Learning Research},
year={2022},
url={https://openreview.net/forum?id=LSAAlS7Yts},
}