Where and When Should the Teleoperated Avatar Look: Gaze Instruction Dataset for Enhanced Teleoperated Avatar Communication

Effective teleoperated avatar communication re- quires expressing social behaviors. Gaze behavior is one of the crucial social behaviors and includes reflexive reactions to the avatar’s surroundings and intentional responses to the operator’s speech and actions. Teleoperated avatars must have their gaze behavior controlled according to situational changes in both the avatar’s and operator’s contexts. However, it is not clear how to adjust the avatar’s gaze in response to changes in both situations. In this paper, we collect a dataset of gazing positions that the avatar is instructed to face, taking into account both avatar and operator situations, and annotation labels that represent both situations in detail. We then exploratorily analyze the ratio of gazing positions per situation through dynamic area-of-interest (AOI) analysis. Our analysis provides insights into determining the gaze behavior of teleoperated avatars.