← Back ICRA 2026

Zero-Shot Recognition of Test Tube Types by Automatically Collecting and Labeling RGB Data

Yu Tang, Weiwei Wan, Hao Chen, Masaki Matsushita, Jun Takahashi, Takeyuki Kotaka, Kensuke Harada

PDF

AI summary

Key figure (auto-extracted from paper)

A zero-shot pipeline automatically segments, clusters, and labels test tube caps on-the-fly, enabling robotic lab systems to recognize novel tube types without manual training data.

Zero-shot recognition Test tube detection Robotic lab automation Automatic clustering SAM2 segmentation Self-supervised learning

Problem

Current vision-based lab automation relies on supervised learning, requiring extensive manually labeled datasets that break down when new or non-standardized test tubes are introduced.

Approach

The system first predicts rack slot occupancy globally, then locally crops and segments cap images using SAM2, automatically clustering them with ViT features and HDBSCAN to incrementally build a recognition library without human supervision.

Key results

Zero-shot recognition of novel test tube types without prior training
Automatic on-the-fly clustering and library expansion via HDBSCAN
Robust slot occupancy estimation using simulated data and Depth Anything V2
Successful real-world validation on a robotic manipulator with diverse tube geometries

Why it matters

Empowers adaptive laboratory automation by eliminating the need for costly retraining, allowing robots to instantly handle new experimental apparatus.

Abstract

This work presents a method for automatically detecting and recognizing test tube types in a rack. It leverages automatic segmentation, clustering, and labeling processes to eliminate the need for explicitly preparing training data. These processes are addressed by using combined global prediction and local cropping, where global prediction estimates the slot occupation states of a rack, and local cropping extracts tube pictures in the local regions of each slot for clustering and labeling. With the help of the proposed method, the robotic tube manipulation system no longer needs tailored data and explicit training in the presence of new tubes, thus achieving flexibil- ity and efficiency. Experimental evaluations conducted with a RealSense D405 camera and the UFactory xArm Lite6 robot manipulator confirm the method’s effectiveness in accurately identifying novel test tube types under real-world conditions.

Index terms

Computer Vision for Automation Robotics and Automation in Life Sciences Grasping