Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning

The use of human demonstrations in reinforce- ment learning has proven to significantly improve agent per- formance. However, any requirement for a human to manu- ally ’teach’ the model is somewhat antithetical to the goals of reinforcement learning. This paper attempts to minimize human involvement in the learning process while retaining the performance advantages by using a single human example collected through a simple-to-use virtual reality simulation to assist with RL training. Our method augments a single demon- stration to generate numerous human-like demonstrations that, when combined with Deep Deterministic Policy Gradients and Hindsight Experience Replay (DDPG + HER) significantly improve training time on simple tasks and allows the agent to solve a complex task (block stacking) that DDPG + HER alone cannot solve. The model achieves this significant training advantage using a single human example, requiring less than a minute of human input. Moreover, despite learning from a human example, the agent is not constrained to human- level performance, often learning a policy that is significantly different from the human demonstration.