Dynamic Robotic Cloth Folding with Efficient Koopman Operator-Based Model Predictive Control
Edoardo Caldarelli, Franco Coltraro, Adri`a Colom ́e, Lorenzo Rosasco, , and Carme Torras
AI summary
Problem
Dynamic cloth folding is hindered by complex nonlinear dynamics that cause severe simulation-to-reality gaps and make fast, accurate trajectory planning computationally expensive. Existing robotic methods struggle with speed, precision, or require complex multi-robot setups.
Approach
The authors train a data-driven linear surrogate model of cloth dynamics using Koopman operator regression on high-fidelity simulator data. This efficient model replaces costly nonlinear physics simulations within a model predictive controller to rapidly generate constrained, high-speed folding trajectories.
Key results
- Data-driven linear surrogate model via Nyström-Koopman regression
- Linear MPC framework with embedded smoothness and safety constraints
- Zero-shot sim-to-real transfer for folding trajectories under 1.5 seconds
- Successful single-manipulator folding to unseen target poses
Why it matters
Provides a computationally efficient pathway for high-speed robotic automation of deformable objects, bridging the sim-to-real gap for industrial textile handling and assistive robotics.
Abstract
Robotic cloth folding is a challenging task, par- ticularly when considering dynamic folding tasks, which aim at folding cloth by fast motions that leverage its dynamics. When subject to such fast motions, the complexity of cloth dynamics hinders both system identification and planning of folding trajectories, resulting in a difficult simulation-to-reality transfer when using physical models of cloth. Compared to the dexterity that humans exhibit when performing folding tasks, robotic approaches usually employ small garments with quite rigid dynamics, and are either too slow, or fast but imprecise, requiring several attempts to achieve a reasonably good fold. In this paper, we tackle these challenges by generating fast folding trajectories with a novel model predictive controller, integrating physics-based simulation of cloth dynamics and efficient, kernel-based Koopman operator regression. Koopman operator regression, an increasingly popular machine learning technique for nonlinear system identification, is used to obtain a linear model for the cloth being folded. Such a surrogate model, trained with data from a high-fidelity, physics-based cloth simulator, can then be employed within a suitable model predictive control algorithm, in place of the costly, nonlinear one, to efficiently generate folding trajectories to be executed by a robotic manipulator. Both in simulated and real-robot experiments, we show how the linearization supplied by the Koopman operator-based model can be employed to efficiently generate fast folding trajectories to unseen poses, without sacrificing folding accuracy.