Constrained Bootstrapped Learning for Few-Shot Robot Skill Adaptation

In this paper, we propose a robot skill-learning method that facilitates fast adaption to new tasks online. Our method is based on a hybrid learning from demonstration and reinforcement learning approach, which seeds learning with a compact and structured skill model, leading to efficient and stable behaviours. To facilitate fast skill adaption, we propose a bootstrapped learning framework that learns a policy for adapting a skill model across a wide range of initial conditions in simulation. This policy is then used to bootstrap a refinement process that quickly adapts the learnt skill model to new initial conditions in a few learning iterations. Our refined skill model is designed to be deployable on hardware and can correct for discrepancies between the simulation and the real world. Furthermore, we propose a novel method for constraining policy exploration to promising trajectories, which is crucial for enabling manipulation in complex environments. We eval- uate our framework in simulation and hardware in multiple environments with varying task complexity. We showcase that compared to the state-of-the-art, which achieves an average success rate of only 56.6% across three different tasks of varying difficulty, our algorithm significantly outperforms it with an average success rate of 90%.