Learning Modular Policy for Multi-Floor Object Navigation:A Factorized Framework for Diagnostic Study

Summary

Object-goal navigation (ObjectNav) in multi-floor scenarios presents a challenge due to sparse rewards caused by long-horizon decision-making.

Abstract Summary

Object-goal navigation (ObjectNav) in multi-floor scenarios presents a challenge due to sparse rewards caused by long-horizon decision-making. In this paper, we propose a diagnostic study based on a modular framework with an effective learnable policy to analyze failure factors in multi-floor scenarios. To achieve an effective policy for diagnosis, we design the hierarchical factorization policy that deconstructs a single global policy into an intra-floor exploration policy and an inter-floor switching policy. To providing an effective initialization for Reinforcement Learning (RL), the lightweight intra-floor policy is learned by distilling the exploration logic of Visual Language Models (VLMs). Under idealized assumptions, we show that the factorized policy is theoretically equivalent to a single global policy at the policy-representation level. Experiment results indicate that perception performance and stair climbing stability are the primary bottlenecks in multi-floor navigation.

Key Points

Object-goal navigation (ObjectNav) in multi-floor scenarios presents a challenge due to sparse rewards caused by long-horizon decision-making.
In this paper, we propose a diagnostic study based on a modular framework with an effective learnable policy to analyze failure factors in multi-floor scenarios.
To achieve an effective policy for diagnosis, we design the hierarchical factorization policy that deconstructs a single global policy into an intra-floor exploration policy and an inter-floor switching policy.
To providing an effective initialization for Reinforcement Learning (RL), the lightweight intra-floor policy is learned by distilling the exploration logic of Visual Language Models (VLMs).
Under idealized assumptions, we show that the factorized policy is theoretically equivalent to a single global policy at the policy-representation level.

Task setup

ObjectNav requires an agent to reach an object goal in an unseen environment. At each episode start, the agent is randomly initialized and assigned a goal category. At time $t$, the agent receives $O_{t}=(V_{t},P_{t})$, where $V_{t}$ is the RGBD image and $P_{t}$ is its pose, and selects from six actions: MoveForward, TurnLeft, TurnRight, LookUp, LookDown, and Stop.

An episode succeeds if the agent stops within distance $d_{\rm succ}$ of the goal and is limited to $T_{\max}$ steps.

Task setup — Learning Modular Policy for Multi-Floor Object Navigation:A Factorized Framework for Diagnostic Study — Fig. 3: The trends of episode rewards for the first round of intra-floor policy and the inter-floor policy training.

Methodology

Fig. 2 illustrates our framework, which uses a diagnostic paradigm to identify the main bottlenecks in multi-floor navigation. It employs a hierarchical factorization policy composed of inter-floor and intra-floor policies, followed by the low-level navigation policy described in Sec. IV-D . First, we validate policy effectiveness through multiple rounds of alternating intra-floor and inter-floor training.

The policy is considered effective if each round improves performance and our method outperforms the zero-shot method.

Methodology — Learning Modular Policy for Multi-Floor Object Navigation:A Factorized Framework for Diagnostic Study — Fig. 4: Failure cause statistics of our method.

Experiments

We conduct experiments on HM3D [ 17 ] and MP3D [ 18 ] . For the intra-floor policy, IL uses 80 HM3D v0.1 scenes and RL fine-tuning uses 65 other HM3D v0.2 scenes. For the inter-floor policy, IL and RL use 25 and 23 multi-floor HM3D v0.1 scenes, respectively. Diagnostics use the 20-scene HM3D v0.1 and 11-scene MP3D validation splits.

We use D-FINE [ 20 ] , Grounding DINO [ 21 ] , and MobileSAM [ 22 ] for perception, and BLIP-2 [ 23 ] as the VLM.

Experiments — Learning Modular Policy for Multi-Floor Object Navigation:A Factorized Framework for Diagnostic Study — Fig. 5: A real-world multi-floor demonstration using the legged robot, where the object goal is a chair. After exploring the initial floor for some time, the robot decides to move to the lower floor, where it successfully locates the goal and completes the task.

Sources and Demonstrations

Figures are reproduced from Zhai et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.