Robust Surgical Tool Tracking with Pixel-Based Probabilities for Projected Geometric Primitives

Controlling robotic manipulators via visual feed- back requires a known coordinate frame transformation be- tween the robot and the camera. Uncertainties in mechanical systems as well as camera calibration create errors in this coordinate frame transformation. These errors result in poor lo- calization of robotic manipulators and create a significant chal- lenge for applications that rely on precise interactions between manipulators and the environment. In this work, we estimate the camera-to-base transform and joint angle measurement errors for surgical robotic tools using an image based insertion- shaft detection algorithm and probabilistic models. We apply our proposed approach in both a structured environment as well as an unstructured environment and measure to demonstrate the efficacy of our methods.