Efficient Hierarchical Reinforcement Learning with Dynamic Kolmogorov�Arnold Network for Long-Horizon Robotic Manipulation
Yuke Qu, Junkai Ren, Jiawei Luo, Yufeng Xie, Huimin Lu, Xin Xu, Yicong Ye
AI summary
Problem
Long-horizon robotic manipulation suffers from an imbalance dilemma in hierarchical reinforcement learning where simplifying skill learning overloads the upper-layer planner, expanding the solution space and computational burden.
Approach
The framework decomposes long-horizon tasks into two intersecting sub-chains to reduce optimization conflicts, while employing a Dynamic Kolmogorov-Arnold Network that adaptively expands its basis functions and uses low-rank updates for stable, efficient skill learning.
Key results
- Dual-chain architecture reduces optimization conflicts and planning burden
- DyKAN dynamically expands basis functions while preserving learned knowledge
- Per-layer update module using Dynamic Tanh and low-rank decomposition ensures stable, low-cost training
- Achieves 10.9% higher task success rate under high-noise conditions with improved efficiency and robustness
Why it matters
Enables reliable, scalable robotic manipulation for complex, long-horizon applications like autonomous laboratories where traditional hierarchical RL struggles with planning complexity and training instability.
Abstract
Long-horizon robotic manipulation remains a crit- ical challenge in robotics. Hierarchical reinforcement learning offers a promising solution, but often suffers from an imbalance dilemma: simplifying skill learning increases the complexity of planning, thereby expanding the solution space and computa- tional burden of planning. To tackle this challenge, we propose a Hierarchical Reinforcement Learning framework with Dynamic Kolmogorov-Arnold Network (DyKAN) based Actor Critic, named HIKER. Firstly, HIKER innovates with a dual-chain design that decomposes the complex task into two intersecting sub-chains, reducing the optimization conflict across subtasks and alleviating the burden on the planning model. Secondly, we develop DyKAN, a scalable neural network for both actor and critic in the skill model of HIKER. DyKAN adaptively adjusts grids and basis functions while preserving learned knowledge, enabling efficient learning of complex manipulation skills. Furthermore, to optimize DyKAN’s performance, we design a per-layer update module that uses Dynamic Tanh (DyT) and low-rank decomposition to ensure stable, low-cost updates during training. Finally, experiments on long-horizon robotic manipulation tasks demonstrate that HIKER significantly im- proves efficiency and robustness, yielding higher-quality skill models and achieving a 10.9% increase in task success rate under the high noise condition. Further insights are available on the website: https://sites.google.com/view/hikerdykan.