SGPT: The Secondary Path Guides the Primary Path in Transformers for HOI Detection

HOI detection is essential for human-computer interaction, especially in behavior detection and robot ma- nipulation. Existing mainstream transformer methods of HOI detection are focused on single-stream detection only, e.g., image →HOI(P1), or image →HO →I(P2). Both paths have their own characteristics of concern, so we propose a novel method, using the Secondary path (P2) Guides the Primary path (P1) in Transformers (SGPT). SGPT contains two core modules: the Dual-Path Consistency (DPC) module and the Instance Interaction Attention (IIA) module. DPC keeps human, object and interaction consistent on the dual-path and lets P2 guide P1 to learn more meaningful features. IIA fuses human and object to enhance interaction in P2, which allows instance to constrain interaction. Our proposed dual- path are employed during training, and only the P1 path is used for inference. Hence, SGPT improves generalization without increasing model capacity in HICO-DET and V-COCO datasets compared to the state-of-the-arts. The code of this work is available at https://github.com/visualVk/sgpt.git.