Predicting human behavior using knowledge information in jig operation and robot collaborative action generation

In human-robot collaborative tasks, learning- based models that can deal with behavior beyond the scope of human description are progressing rapidly. Deep learning is e↵ective in capturing complex nonlinear relationships, making it valuable in scenarios with intricate interactions between the environment and tasks, such as collaborative tasks. Deep learning improves the performance by incorporating multiple information sources. Human knowledge, which is regarded as supplemental information obtained from the environment, has been shown to enhance the generalization ability of task execution when it is appropriately incorporated into the learn- ing process for robot motion generation. Among the various models, those that utilize action labels subjectively defined by humans for robot behavior enable the robot to compre- hend its own actions better, leading to higher generalization. This approach also suggests that estimating human actions contributes to predicting robot movements in human-robot collaboration (HRC). However, the performance of learning- based methods is significantly influenced by the quality of the training data. Therefore, capturing appropriate human information and integrating this information into the learning process are critical for improving the ability of the robot to learn collaborative tasks. In this study, we propose a learning model that not only provides a robot with action labels for its own behavior but also includes human action labels, encouraging the robot to respond to human actions. The optimal amount of human information to be used in learning is evaluated by adjusting the methods for defining human action labels and the quantity of human data utilized. Experiments were conducted with a task in which the robot handled the manipulation of jigs in an assembly operation involving both humans and robots. The results of the learning process suggest that estimating human behavior can assist in generating collaborative robot actions.