Robust Online Residual Refinement Via Koopman-Guided Dynamics Modeling
Zhefei Gong, Shangke Lyu, Pengxiang Ding, Wei Xiao, Donglin Wang
AI summary
Problem
Existing residual policy learning methods rely on local corrections around a base policy's output, lacking global awareness of state evolution, which limits robustness and generalization to unseen or perturbed scenarios in long-horizon robotic tasks.
Approach
KORR lifts system states into a linear latent space using Koopman operator theory to predict future states, then conditions a residual policy on these globally informed imagined states to generate stable, corrective actions.
Key results
- Consistently outperforms baselines in success rate under perturbations
- Superior generalization to unseen initial conditions and disturbances
- Validates Koopman-guided state imagination for stable residual updates
- Bridges classical control theory with modern residual learning
Why it matters
Enables more robust and generalizable robotic manipulation in real-world, long-horizon tasks by combining the interpretability of classical control theory with modern residual learning.
Abstract
Imitation learning (IL) enables efficient skill ac- quisition from demonstrations but often struggles with long- horizon tasks and high-precision control due to compounding errors. Residual policy learning offers a promising, model- agnostic solution by refining a base policy through closed- loop corrections. However, existing approaches primarily fo- cus on local corrections to the base policy, lacking a global understanding of state evolution, which limits robustness and generalization to unseen scenarios. To address this, we propose incorporating global dynamics modeling to guide residual policy updates. Specifically, we leverage Koopman operator theory to impose linear time-invariant structure in a learned latent space, enabling reliable state transitions and improved extrapolation for long-horizon prediction and unseen environments. We introduce KORR (Koopman-guided Online Residual Refinement), a simple yet effective framework that conditions residual corrections on Koopman-predicted latent states, enabling globally informed and stable action refinement. We evaluate KORR on long-horizon, fine-grained robotic furniture assembly tasks under various per- turbations. Results demonstrate consistent gains in performance, robustness, and generalization over strong baselines. Our findings further highlight the potential of Koopman-based modeling to bridge modern learning methods with classical control theory.