DQN-Based On-Line Path Planning Method for Automatic Navigation of Miniature Robots

Untethered magnetic microrobots with control- lable locomotion property and multiple functions have attracted lots of attention in recent years. Owing to the small scale, micro- robots with automatic navigation possess a promising perspec- tive for biomedical applications including precise delivery and targeted therapy in confined and narrow space, especially for in-vivo scenario. However, the practical working environment for microrobots can be various, dynamic, and complicated, and path planning algorithm applicable for both dynamic obstacle avoidance and planning in maze-like environments still remains a challenge. Furthermore, considering the sizes, different types of microrobots may occupy different proportions of the field of vision. The safe distance between the waypoints and the obstacles needs to be taken into thoughts. In this work, we proposed a reinforcement learning-based strategy capable of real-time path planning for microrobots in different scales. The reference moving direction at each control period is provided by a deep Q network (DQN) according to the local surrounding environment, and the corresponding control magnetic field is generated via a 3-axis Helmholtz coil system. A distur- bance observer (DOB) is responsible for the locomotion state observation and direction error compensation. Experiments demonstrate the effectiveness of our proposed strategy using microrobots with different locomotion mechanisms and scales, in both virtual dynamic obstacle environments and channel-like environments.