← Back ICRA 2026

Find the Fruit: Zero-Shot Sim2Real RL for Occlusion-Aware Plant Manipulation

Nitesh Subedi, Hsin-Jung Yang, Devesh K. Jha, Soumik Sarkar

PDF

AI summary

Key figure (auto-extracted from paper)

A zero-shot sim-to-real reinforcement learning policy reliably repositions deformable plant foliage to reveal hidden fruits without per-plant retuning or explicit 3D reconstruction.

Sim-to-real transfer Reinforcement learning Occlusion-aware manipulation Autonomous harvesting Deformable object control

Problem

Autonomous harvesting robots struggle to locate fruits hidden by dense, variable foliage due to significant occlusion and structural uncertainty, making traditional perception and manipulation pipelines brittle.

Approach

The authors train an end-to-end RL policy in simulation to plan kinematic movements for occlusion removal, decoupling it from a compliant low-level controller to enable zero-shot transfer to real-world plants with varying stiffness and morphology.

Key results

Achieved up to 86.7% success rate in zero-shot real-world fruit exposure trials
Demonstrated robust generalization across plants with different stiffness and morphologies
Successfully extended to sequential multi-fruit discovery without architectural changes
Eliminated need for explicit 3D reconstruction or per-plant model retuning

Why it matters

Enables reliable, scalable autonomous harvesting in unstructured agricultural environments by bridging the sim-to-real gap for deformable plant manipulation.

Abstract

model generalizes to experimental plants. Abstract— Autonomous harvesting in the open presents a complex manipulation problem. In most scenarios, an au- tonomous system has to deal with significant occlusion and require interaction in the presence of large structural uncer- tainties (every plant is different). Perceptual and modeling uncertainty make design of reliable manipulation controllers for harvesting challenging, resulting in poor performance during deployment. We present a sim2real reinforcement learning (RL) framework for occlusion-aware plant manipulation, where a policy is learned entirely in simulation to reposition stems and leaves to reveal target fruit(s). In our proposed approach, we de- couple high-level kinematic planning from low-level compliant control which simplifies the sim2real transfer. This decompo- sition allows the learned policy to generalize across multiple plants with different stiffness and morphology. In experiments with multiple real-world plant setups, our system achieves up to 86.7% success in exposing target fruits, demonstrating robustness to occlusion variation and structural uncertainty.

Index terms

Agricultural Automation Manipulation Planning Reinforcement Learning