Previous Knowledge Utilization in Online Anytime Belief Space Planning
Michael Novitsky, Moran Barenboim, Vadim Indelman
AI summary
Problem
Online planning in continuous belief spaces typically discards information from previous sessions, causing high computational costs and inefficient decision-making under uncertainty.
Approach
The authors introduce IR-PFT, an anytime Monte Carlo Tree Search algorithm that efficiently reuses past trajectories by applying an incremental Multiple Importance Sampling estimator to update action-value estimates.
Key results
- Incremental update method for Multiple Importance Sampling estimators
- Experience-based value function estimation from prior trajectories
- IR-PFT algorithm for efficient historical data reuse
- Significantly reduced planning time with maintained performance
Why it matters
Enables faster, more responsive autonomous systems operating in uncertain, continuous environments by making online POMDP planning computationally feasible.
Abstract
Online planning under uncertainty remains a critical challenge in robotics and autonomous systems. While tree search techniques are commonly employed to construct partial future trajectories within computational constraints, most existing methods discard information from previous plan- ning sessions considering continuous spaces. This study presents a novel, computationally efficient approach that leverages historical planning data in current decision-making processes. We provide theoretical foundations for our information reuse strategy and introduce an algorithm based on Monte Carlo Tree Search (MCTS) that implements this approach. Experimental results demonstrate that our method significantly reduces com- putation time while maintaining high performance levels. Our findings suggest that integrating historical planning information can substantially improve the efficiency of online decision- making in uncertain environments, paving the way for more responsive and adaptive autonomous systems.