Learning to Estimate Incipient Slip with Tactile Sensing to Gently Grasp Objects

To gently grasp objects, robots need to balance generating enough friction yet avoiding too much force that could damage the object. In practice, the force regulation is challenging to implement since it requires knowledge of the friction coefficient, which can vary from object to object and even from grasp to grasp. Tactile sensing offers a window in the contact mechanics and provides information about friction. Notably touch can detect the precursor of the object slipping away from the grasp. To find this information, tactile sensors measure the deformation field of an artificial skin in both the normal and tangential direction. However, current approaches only react to slip and therefore react too late to perturbations. The object slips, inducing a failure of the grasp and damage. In this study, we introduce a method that uses machine-learning to anticipate slip by computing the so-called safety margin of the grasp. This safety margin represents the extra lateral force that maintains the contact away from the frictional limit. To find this value, we use a high-density camera-based tactile sensor to measure the 3D deformation of the surface via the movement of 82 colored markers. We trained a Convolutional Neural Network (CNN) to estimate the safety margin from the tactile images. Because it gives a distance to slip, the safety margin is a powerful metric for regulating grasp forces. As a testament of this effectiveness, we show that a simple proportional controller can robustly grasp a wide variety of objects. The results show that this control method outperforms slip detection methods, by reducing regrasp reaction times while decreasing grasping forces to 1-3 N. INTRODUCTION When dynamically manipulating objects with a robotic gripper, the contact with fingers constantly evolves. During the movement, the pressure and traction distributions change in response to the dynamics and as a function of the friction and material properties. Consequently, it can be difficult to estimate and predict how the object will move within the grasp and whether or not the grasp will be stable. This prediction is crucial for grasping since the forces at the contact determine if the object can rotate, pivot, slide, or stay in place. Without the information regarding the frictional resistance, a controller cannot optimally determine the force that would maintain a stable grasp. Therefore friction-agnostic approaches generally overestimate the grip force to avoid a catastrophic loss of grip [1], [2]. Large forces prevent dropping objects, but also restrain manipulation flexibility [3]. Tactile sensing offers a promising avenue for capturing the mechanical interaction at the interface between the en- vironment and the fingers. Robotic tactile sensors capture the deformation of an artificial skin from which they can infer high-order information, such as material properties (i.e. compliance, texture, curvature) or the contact state (i.e. distance to slip or effort). Tactile sensors discretize the mechanical interaction, represented by the pressure or deformation field, often using miniaturized high-resolution cameras pointing at the membrane [4], [5]. The pressure or deformation field can be processed to estimate contact shape and force [6], or to detect slip from physics-based models [7]. More complex mechanical interac- tions can be captured using machine learning approaches [8]– [10]. However, when tactile sensors are deployed for grasp- ing regulation they are used to detect slip which makes the reaction to perturbation too slow and they often fails to regain stability after slip [11]–[13]. At a mechanical level, the transition from stick to slip for a soft fingertip occurs gradually. When the tangential force increases from a fully stuck contact, the outer edge of the contact area begins slipping while the center remains stuck. The slip region grows until the entire contact area is in the slip state and the object fully slips [14], [15]. It is postulated that humans use the ratio between the stick and the slip region inferred from the skin deformation to estimate the safety margin [16]. This distance from the onset of slip is believed to be ultimately used to regulate their grip force [17], [18]. In this work, we measured the pattern of deformation of the artificial fingertip before the onset of slip with an iterated version of our ChromaTouch tactile sensor [19], [20]. We trained a convolutional neural network (CNN) to estimate the frictional strength using the safety margin. The model performance is evaluated against an unseen dataset, which showed an average prediction accuracy of 98.2% from the ground truth, when computing the MSE loss over the entire range of safety margin predictions. The 50 Hz refresh rate and the accuracy of the estimation make it suitable for real- time grasping applications on soft and complex objects such as fruits and vegetables, which is demonstrated on three fragile fruits. We aim to create a tactile-enabled gripper that maintains a squeezing force on an arbitrary object so that the safety margin remains constant (Fig. 1A). To do so, we designed an impedance control gripper (Fig. 1B) which regulates 2024 IEEE International Conference on Robotics and Automation (ICRA 2024) May 13-17, 2024. Yokohama, Japan 979-8-3503-8457-4/24/$31.00 ©2024 IEEE 16118