Localized Graph-Based Neural Dynamics Models for Terrain Manipulation
Chaoqi Liu, Yunzhu Li, Kris Hauser
AI summary
Problem
Modeling fine-resolution terrain dynamics at large scales is computationally prohibitive for existing graph-based neural networks due to unbounded particle counts and memory constraints.
Approach
The method learns a 3D region of interest to isolate only the particles likely to move, applying graph neural dynamics exclusively to this subgraph while using geometric normal features to enforce realistic boundary resistance.
Key results
- Orders-of-magnitude reduction in computation time and GPU memory usage
- Higher prediction accuracy than full-graph GBND and CNN-based heightmap models
- Boundary-aware normal features enable robust generalization across unbounded terrain depths
- Successful sim-to-real transfer and effective MPPI planning for excavation and shaping tasks
Why it matters
Enables autonomous robots to efficiently plan and execute complex terrain manipulation in large-scale, unbounded environments like construction sites or planetary surfaces.
Abstract
Predictive models can be particularly helpful for robots to effectively manipulate terrains in construction sites and extraterrestrial surfaces. However, terrain state repre- sentations become extremely high-dimensional especially to capture fine-resolution details and when depth is unknown or unbounded. This paper introduces L-GBND, a learning-based approach for terrain dynamics modeling and manipulation, leveraging the Graph-based Neural Dynamics (GBND) frame- work to represent terrain deformation as motion of a graph of particles. Based on the principle that the moving portion of a terrain is usually localized, our approach builds a large terrain graph (potentially millions of particles) but only identifies a very small active subgraph (hundreds of particles) for predicting the outcomes of robot-terrain interaction. To minimize the size of the active subgraph we introduce a learning-based approach that identifies a small region of interest (RoI) based on the robot’s control inputs and the current scene. We also introduce a novel domain boundary feature encoding that allows GBND to perform accurate dynamics prediction in the RoI interior while avoiding particle penetration through RoI boundaries. Our proposed method is both orders of magnitude faster than naïve GBND and it achieves better overall prediction accuracy. We further evaluated our framework on excavation and shaping tasks on terrain with different granularity.