Reinforcement Learning for Laser Welding Speed Control Minimizing Bead Width Error

In this paper, we propose a method for rein- forcement learning-based laser welding control. Conventional methods apply standard reinforcement learning formulations to welding tasks, but we show that this formulation can minimize bead width or penetration depth errors only when the welding speed is constant. Therefore, conventional methods are suboptimal for training control parameters including the welding speed. The proposed method discounts future rewards with respect to the welding length instead of time steps to solve this issue. This is easily implemented by (1) modifying the discount factor used for Q-function updates in existing reinforcement learning algorithms and (2) using an appropriate reward function. Experimental results using simulators show that the proposed method achieves performance that is superior to conventional methods.