Interactive 3D scene graph alignment for zero-shot object navigation.

Abstract

Navigating in unknown, dynamic, and complex 3D environments using purely visual inputs remains a significant challenge for embodied agents. To address this, we propose I3DSG (Interactive 3D Scene Graph), a novel perception framework that constructs a Bird's Eye View (BEV)-based scene graph enriched with geometric layout, semantic relationships, and actionable affordances. By integrating open-vocabulary object detection and large language models, I3DSG aligns prior commonsense knowledge with real-time scene observations, enabling robust reasoning about missing triples and feasible interactions in unseen environments. Action selection is further optimized through graph-based grid decision scoring, which leverages global spatial context for consistent path planning. Extensive experiments on standard benchmarks demonstrate superior performance in success rate and SPL compared to existing methods. Critically, we validate our approach not only in simulation but also in real-world deployments using a Scout ROS2 robot equipped with multi-view RGBD cameras, confirming that I3DSG generalizes effectively to physical environments despite sensor noise and illumination variations. Code and datasets can be found in https://github.com/zhoukang12321/3DSG4Nav.