A Flexible Field-Based Policy Learning Framework for Diverse Robotic Systems and Sensors

We present a cross-robot visuomotor learning framework that integrates diffusion policy–based control with 3D semantic scene representations from D3Fields to enable category-level generalization in manipulation. Its modular de- sign supports diverse robot–camera configurations, including UR5 arms with Microsoft Azure Kinect arrays and bimanual manipulators with Intel RealSense sensors, through a low- latency control stack and intuitive teleoperation. A unified configuration layer enables seamless switching between setups for flexible data collection, training, and evaluation. In a grasp- and-lift block task, the framework achieved an 80% success rate after only 100 demonstration episodes, demonstrating robust skill transfer between platforms and sensing modalities. This design paves the way for scalable real-world studies in cross- robotic generalization.