Iterative Shaping of Multi-Particle Aggregates Based on Action Trees and VLM
Hoi-Yin Lee, Peng Zhou, Anqing Duan, Chenguang Yang, David Navarro-Alarcon
AI summary
Problem
Single-tool manipulation of multi-particle aggregates struggles to maintain group cohesion during transport, leaving a gap in high-level task planning and adaptive trajectory execution for cohesive herding.
Approach
The system uses Vision Language Models to plan high-level herding actions while representing the particle group's contour with truncated Fourier series to generate adaptive waypoints via an iterative action tree, refined by model predictive control for obstacle avoidance.
Key results
- Contour-based shaping strategy using non-prehensile robotic actions
- Iterative action tree for cohesion-preserving path planning
- Quantitative cohesiveness metric for ensemble compactness
- LLM-based planner for adaptive symbolic task decomposition
Why it matters
Provides a scalable framework for autonomous handling of loose or granular ensembles in logistics, agriculture, and industrial automation where maintaining group integrity is essential.
Abstract
In this paper, we address the problem of ma- nipulating multi-particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping and pushing actions using robotically controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate’s contour using truncated Fourier series, providing efficient parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real- world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi-particle aggregates while maintaining high system cohesion.