Towards Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups.

Abstract

Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring target-domain retraining, fine-tuning, or dataset-specific metric parameter tuning for new environments. We identify three critical limitations: (a) fixed user-defined parameters (e.g., voxel size, search radius) that fail to generalize across varying scales, (b) learned keypoint detectors exhibit poor cross-domain transferability, and (c) absolute coordinates amplify scale mismatches between datasets. To address these three issues, we present BUFFER-X, a registration framework whose descriptor model is trained once and then applied without parameter tuning or target-domain retraining, which achieves zero-shot generalization through: (a) geometric bootstrapping for automatic hyperparameter estimation, (b) distribution-aware farthest point sampling to replace learned detectors, and (c) patch-level coordinate normalization to ensure scale consistency. Our approach employs hierarchical multi-scale matching to extract correspondences across local, middle, and global receptive fields, enabling robust registration in diverse environments. For efficiency-critical applications, we present BUFFER-X-Lite from a holistic system perspective, integrating the BUFFER-X front-end and adaptive early exit with the fast and robust pose solver to reduce total computation time by 43% relative to BUFFER-X while preserving comparable accuracy. We evaluate on a comprehensive benchmark comprising 12 datasets spanning object-scale, indoor, and outdoor scenes, including cross-sensor registration between heterogeneous LiDAR configurations. Results demonstrate that our approach generalizes effectively across diverse scales, scenes, and sensor setups. Code: https://github.com/MIT-SPARK/BUFFER-X.