TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations
Guy Freund, Tom Jurgenson, Matan Sudry, Erez Karpas
AI summary
Problem
Demonstration-free robotic knot-tying is hindered by sparse rewards, complex topology, and inefficient data collection, while prior methods are limited to restrictive single-step execution and goal-conditioned policies that fail to generalize.
Approach
The framework pairs a high-level topological planner with specialized low-level reinforcement learning agents that execute multi-step sequences conditioned on abstract topological moves rather than specific goal states.
Key results
- Solves previously unattainable complex knots like Figure-8 and Overhand
- Significantly increases task success rates compared to prior demonstration-free methods
- Reduces planning time through efficient multi-step reinforcement learning
- Establishes a new state-of-the-art for demonstration-free robotic knot-tying
Why it matters
Advances autonomous robotic manipulation for applications like surgical robotics, textile manufacturing, and industrial cable management by enabling reliable, demonstration-free knot-tying.
Abstract
Robotic knot-tying represents a fundamental chal- lenge in robotics due to the complex interactions between de- formable objects and strict topological constraints. We present TWISTED-RL, a framework that improves upon the previous state-of-the-art in demonstration-free knot-tying (TWISTED), which smartly decomposed a single knot-tying problem into manageable subproblems, each addressed by a specialized agent. Our approach replaces TWISTED’s single-step inverse model that was learned via supervised learning with a multi- step Reinforcement Learning policy conditioned on abstract topological actions rather than goal states. This change allows more delicate topological state transitions while avoiding costly and ineffective data collection protocols, thus enabling better generalization across diverse knot configurations. Experimen- tal results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand. Furthermore, the increase in success rates and drop in planning time establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.