Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Goal-reaching in unknown environments is one of the essential tasks in robot applications. Large-scale perception and long-horizon decision-making are the keys to solving this task as the operation scope expands or complexity rises. Exist- ing navigation methods may suffer from degraded performance in complicated environments induced by scalability-limited map representation or greedy decision strategy. We propose the path- extended graph as a compact map representation providing suf- ficient structural information within a reasonable receptive field and incorporate it into a hierarchical policy for higher efficiency and generalizability. The path-extended graph contains the concise topology of environment structure and frontier layout for large-scale perception, avoiding the impact of redundant information. The hierarchical policy solves long-horizon non- myopic decision-making through a high-level frontier selection policy using deep reinforcement learning (DRL) and a low- level motion controller that handles path planning and collision avoidance. Simulation and real-world experiments demonstrate that our method outperforms other competitive approaches in avoiding redundant movement and achieves efficient goal- reaching, especially in complex environments.