FAR-Dex: Few-Shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation
Yushan Bai, Fulin Chen, Hongzheng Sun, Yuchuang Tong, En Li, zhengtao zhang
AI summary
Problem
Dexterous manipulation requires seamless arm-hand coordination but suffers from scarce high-quality demonstration data and high-dimensional action spaces, making policy training and real-world deployment difficult.
Approach
The framework uses a simulator to generate diverse, physically constrained trajectories from few demonstrations, then applies an adaptive residual module that dynamically weights multi-step trajectory and observation features to refine the base policy online.
Key results
- 13.4% improvement in synthetic data quality
- 7% increase in simulation task success rates
- Over 80% real-world task success rate
- Reduced inference latency via consistency model distillation
Why it matters
It bridges the sim-to-real gap for fine-grained robotic manipulation, enabling scalable deployment of dexterous arm-hand systems with limited human demonstrations.
Abstract
Achieving human-like dexterous manipulation through the collaboration of multi-fingered hands with robotic arms remains a longstanding challenge in robotics, primarily due to the scarcity of high-quality demonstrations and the complexity of high-dimensional action spaces. To address these challenges, we propose FAR-Dex, a hierarchical framework that integrates few-shot data augmentation with adaptive residual refinement to enable robust and precise arm–hand coordination in dexterous tasks. First, FAR-DexGen leverages the IsaacLab simulator to generate diverse and physically-constrained trajec- tories from a few demonstrations, providing a data foundation for policy training. Second, FAR-DexRes introduces an adaptive residual module that refines policies by combining multi- step trajectory segments with observation features, thereby enhancing accuracy and robustness in manipulation scenarios. Experiments in both simulation and real-world demonstrate that FAR-Dex improves data quality by 13.4% and task success rates by 7% over state-of-the-art methods. It further achieves over 80% success in real-world tasks, enabling fine-grained dexterous manipulation with strong positional generalization.