Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics
Britt Besch∗,, Tai Mai, Jeremias Thun, Markus Huff,, J ̈orn Vogel, Freek Stulp, Samuel Bustamante
AI summary
Problem
Shared-control assistive robots often fail due to misaligned mental models between the human and the robot, yet existing explainability methods cannot actively help users correct the robot's knowledge.
Approach
The authors propose a model reconciliation framework that uses an LLM workflow to predict and explain discrepancies between the robot’s internal models and user queries, followed by a VLM-based arbitration module that allows the human to collaboratively update the robot’s world model or guide recovery actions.
Key results
- A bi-directional model reconciliation framework for shared control
- An LLM pipeline identifying five specific mental model divergences without explicit human modeling
- A VLM-driven recovery module enabling natural language corrections and adaptive robot guidance
- Experimental validation on a real wheelchair-based mobile manipulator and digital twin in daily living tasks
Why it matters
Enables trustworthy and adaptable human-robot collaboration in assistive settings by bridging knowledge gaps through transparent explanations and direct user corrections.
Abstract
Whenever humans and robots work together, it is essential that unexpected robot behavior can be explained to the user. Especially in applications such as shared control — as the name may imply — the user and the robot must share the same model of the objects in the world, and the actions that can be performed on these objects. We achieve this with a model reconciliation framework. We use a Large Language Model (LLM) to predict and explain differences between the robot’s and human’s mental models, without requiring a formal model of the user. The framework also resolves model divergence by allowing the human to correct the robot after the explanation. We provide an implementation in assistive robotics and conduct experiments with a real wheelchair-based mobile manipulator and its digital twin.