Communication-Efficient Multi-Robot Exploration Using Distributed Coverage-Biased Q-Learning

Frontier exploration and reinforcement learning have historically been used to solve the problem of enabling many mobile robots to autonomously and cooperatively explore complex surroundings. These methods need to keep an internal global map for navigation, but they do not consider the high costs of communication and information sharing between robots. This study offers CQLite; a novel distributed Q- learning technique designed to minimize data communication overhead between robots while achieving rapid convergence and thorough coverage in multi-robot exploration. The proposed CQLite method uses ad hoc map merging, and selectively shares updated Q-values at recently identified frontiers to sig- nificantly reduce communication costs. The theoretical analysis of CQLite’s convergence and efficiency and extensive numerical verification on simulated indoor maps utilizing several robots demonstrate the method’s novelty. With over 2x reductions in computation and communication alongside improved mapping performance, CQLite outperformed cutting-edge multi-robot exploration techniques like Rapidly Exploring Random Trees and Deep Reinforcement Learning.