Beyond Success: Quantifying Demonstration Quality in Learning from Demonstration

Learning from Demonstration (LfD) empowers novice users to teach robots daily life tasks without writing sophisticated code, thereby promoting the democratization of robotics. However, novice users often provide sub-optimal demonstrations, which can potentially impact the robot’s ability to efficiently learn and execute the tasks. Prior research has assessed the quality of demonstrations by evaluating the robot’s task performance; however, the approach remains insufficient to qualify individual demonstrations, leaving the reason for classifying demonstrations as high- or low-quality unknown. Therefore, this simulation-based study aims to quantify the quality of individual demonstration at each step by incorporating motion-related quality features such as manipulability and joint-space jerk. To assess the efficacy of these features, we initially evaluated the given demonstrations—taking into account each quality feature—to rank them from high- to low-quality. Subsequently, we investigated the impact of demonstration’s quality on task performance and the quality of task execution. In this pursuit, we trained a series of LfD models for distinct manipulation tasks: cube lifting and pick-and-place of soda can. Our results illustrate a strong correlation between ranked demonstrations and the quality of task execution. Interestingly, we observed that the quality features have a significant impact on task performance, particularly when the provided demonstrations exhibit diversity in terms of quality. Overall, this analysis enables quantifying the quality of individual demonstrations based on motion-related quality features, thus improving learning from demonstration.