MSPNet: A Multiscale Pyramid Network for Semantic Segmentation of Urban-Scale Photogrammetric Point Clouds

Photogrammetric point clouds have emerged as a cost-effective solution to urban environment perception and scene understanding, significantly advancing recent research in urban-scale 3-D semantic segmentation. In contrast to well-explored indoor and road scenes, urban-scale photogrammetric point clouds exhibit more complex spatial geometries, richer texture details, and pronounced multiscale disparities between background environments and foreground objects. These challenges demand neural networks with enhanced spatial, low-level, and multiscale learning capabilities. To address these issues, we propose the multiscale pyramid network (MSPNet), a conceptually streamlined, end-to-end network for semantic segmentation of urban scenes. The proposed MSPNet comprises three key components: 1) the ellipsoid spherical position embedding (ESPE) module, which leverages learnable ellipsoid queries and spherical harmonics (SHs), rather than using a fixed ball query and a linear combination of spatial coordinates, to explicitly model position embeddings and effectively capture intricate geometric relationships; 2) the top-down information retrospection (TDIR) module, which establishes hierarchical semantic guidance through retrospective pathways, enhancing structural consistency across features at different levels; and 3) the low-rank adaptation (LoRA) fusion module, which applies matrix rank constraints during multiscale feature fusion to decompose high-level semantics into low-rank subspaces, alleviating multiscale semantic confusion. We have conducted extensive experiments on several established urban-scale photogrammetric point cloud segmentation benchmarks, including HRHD-HK, SensatUrban, and UrbanBIS. The experimental results demonstrate that MSPNet achieves significant improvements across these benchmarks, highlighting the superior ability to accurately segment urban scenes and the potential to advance urban environment perception and scene understanding.