Baseline Policy Adapting and Abstraction of Shared Autonomy for High-Level Robot Operations
Ehsan Yousefi, Mo Chen, Inna Sharf
AI summary
Problem
Designing a shared autonomy architecture for hierarchical robotic decision-making where complete domain knowledge is unavailable, human input is often noisy or non-cooperative, and safety-critical applications require interpretable, tunable autonomy levels.
Approach
The authors model human-robot interaction using hierarchical Markov decision processes and deep reinforcement learning, introducing a policy adapting method that continuously shapes the robot's autonomous policy based on task context, human internal states, and a pre-trained baseline.
Key results
- A mathematical model of shared autonomy policy integrating hierarchical MDPs and variational analysis of human behavior
- An end-to-end deep reinforcement learning algorithm for training a tunable baseline policy under adversarial conditions
- A pilot human-in-the-loop study demonstrating effective high-level pick-and-place task execution with varying human skill levels
- Identification of key design variables that govern autonomy levels and system performance
Why it matters
Provides a robust, interpretable foundation for deploying shared autonomy in safety-critical, high-level robotic applications like heavy machinery and assistive systems where human expertise remains essential.
Abstract
This article presents a novel shared autonomy and baseline policy adapting framework for human–robot interactions in high-level context-aware robotic tasks. With a unique methodol- ogy that leverages hierarchies in decision-making as well as varia- tional analysis of human policy, we propose a mathematical model of shared autonomy policy. The framework aims at interpretable high-level decision-making for efficient robot operation with hu- man in the loop. We modeled the decision-making process using hierarchical Markov decision processes in an algorithm we called policy adapting, where the autonomous system policy is adapted, and hence shaped by incorporating design variables contextual to the robot, human, task, and pretraining. By integrating deep reinforcement learning within a multiagent hierarchical context, we present an end-to-end algorithm to train a baseline policy designed for shared autonomy. We showcase the effectiveness of our framework, and particularly the interplay between different design elements and human’s skill level, in a pilot study with a human user in a simulated sequence of high-level pick-and-place tasks. The proposed framework advances the state of the art in shared autonomy for robotic tasks, but can also be applied to other domains of autonomous operation.