ILCL: Inverse Logic-Constraint Learning from Temporally Constrained Demonstrations
Minwoo Cho, Jaehwi Jang, Daehyung Park
AI summary
Problem
Learning temporal logic constraints from demonstrations is hindered by a combinatorially large specification space, non-differentiable logic representations, and the ill-posed nature of non-Markovian constraints that often neglect simultaneous reward maximization.
Approach
The method frames constraint learning as a two-player zero-sum game between a genetic algorithm that mines free-form temporal logic syntax trees and a logic-constrained reinforcement learning agent that optimizes policies using a novel constraint redistribution scheme.
Key results
- Outperforms state-of-the-art baselines in learning and transferring temporal logic constraints across four simulated tasks
- Achieves the lowest constraint-violation scores while maintaining high task rewards comparable to expert demonstrations
- Successfully transfers learned constraints to a real-world peg-in-shallow-hole manipulation task
- Provides the first model-free constrained RL approach capable of generalizing to arbitrary temporal logic specifications
Why it matters
It enables robots to autonomously learn and generalize complex, interpretable temporal constraints from demonstrations, advancing safe and reliable autonomous manipulation and navigation.
Abstract
We aim to solve the problem of temporal-constraint learning from demonstrations to reproduce demonstration-like logic-constrained behaviors. Learning logic constraints is chal- lenging due to the combinatorially large space of possible specifi- cations and the ill-posed nature of non-Markovian constraints. To this end, we introduce inverse logic-constraint learning (ILCL), a novel temporal-constraint learning method formulated as a two-player zero-sum game between 1) a genetic algorithm-based temporal-logic mining (GA-TL-Mining) and 2) logic-constrained reinforcement learning (Logic-CRL). GA-TL-Mining efficiently constructs syntax trees for parameterized truncated linear tem- poral logic (TLTL) without predefined templates. Subsequently, Logic-CRL finds a policy that maximizes task rewards under the constructed TLTL constraints via a novel constraint redistri- bution scheme. Our evaluations show ILCL outperforms state- of-the-art baselines in learning and transferring TL constraints on four temporally constrained tasks. We also demonstrate successful transfer to real-world peg-in-shallow-hole tasks.