3D Localization of Objects Buried within Granular Material Using a Distributed 3-Axis Tactile Sensor

While visual sensing is often the predominant modality for a robot to localize objects in the environment, tactile and force sensing become crucial when objects are occluded, poorly visible, or buried. However, existing works on locating buried objects rely solely on force measurements at a single contact point on the robot end-effector, making 3D localization very challenging. This paper presents an alternative approach using a tactile sensor that measures both normal and shear forces (i.e. 3-axis) on distributed points; three Long Short- Term Memory (LSTM) models are trained with real-world data to perform real-time 3D localization (i.e. distance, direction and depth) of an object buried within a granular material. Our experimental results suggest that measuring both normal and shear forces (instead of just normal) on distributed contact points (instead of only one point) is essential for the accurate 3D localization of buried objects.