Symbolic Manipulation Planning with Discovered Object and Relational Predicates
Alper Ahmetoglu, Erhan Oztop, Emre Ugur
AI summary
Problem
Learning scalable symbolic representations and planning rules from a robot’s continuous, unsupervised exploration remains difficult, particularly when handling arbitrary numbers of objects and their dynamic relations. Prior methods often rely on fixed-size object sets, implicit relational modeling, or complex state partitioning, hindering direct translation to off-the-shelf planners.
Approach
The system uses a differentiable encoder-decoder network with self-attention to explicitly learn unary and relational predicates from continuous object features, then groups these symbols into order-invariant categories to derive abstract operators that are directly translated into PDDL for planning.
Key results
- Learns symbolic knowledge of pick-up, carry, and place operations across varying object configurations
- Translates discovered unary and relational predicates directly into PDDL operators without manual clustering
- Achieves superior effect prediction accuracy and planning performance compared to DeepSym and Attentive DeepSym baselines
- Handles arbitrary object counts and remains invariant to object ordering in tabletop stacking tasks
Why it matters
Enables robots to autonomously discover scalable symbolic world models from raw experience and leverage fast, off-the-shelf planners for complex multi-object manipulation.
Abstract
Discovering the symbols and rules that can be used in long-horizon planning from a robot’s unsupervised exploration of its environment and continuous sensorimotor experience is a challenging task. The previous studies proposed learning symbols from single or paired object interactions and planning with these symbols. In this work, we propose a system that learns rules with discovered object and relational symbols that encode an arbitrary number of objects and the relations between them, converts those rules to Planning Domain Description Language (PDDL), and generates plans that involve affordances of the arbitrary number of objects to achieve tasks. We validated our system with box- shaped objects in different sizes and showed that the system can develop a symbolic knowledge of pick-up, carry, and place operations, taking into account object compounds in different configurations, such as boxes would be carried together with a larger box that they are placed on. We also compared our method with other symbol learning methods and showed that planning with the operators defined over relational symbols gives better planning performance compared to the baselines.