HAIR: Head-mounted AR Intention Recognition

Communicating the mutual intentions in augmented reality human-robot interaction is necessary to complete a team task. Previous work either focused on helping humans to understand robot intentions or require bulky and expensive systems.

A recent paper on proposes a system that evaluates the intentions of the human in a scenario between a single human and an industrial manipulator.

Industrial robot. Image credit: jarmoluk via Pixabay (Free Pixabay licence)

The system is portable and requires a short set-up. A single human worker can interact with multiple robots one after another as the approach is robot agnostic. It takes as input the pose of a head-mounted display, the position of hand joints, and a set of possible spatial goals. A set of probabilities of the goal human wants to approach, and the action they wish to perform is provided as an output.

In a simulation of an automatic warehouse, the suggested system improved warehouse efficiency compared to a simplistic approach.

Human teams exhibit both implicit and explicit intention sharing. To further development of human-robot collaboration, intention recognition is crucial on both sides. Present approaches rely on a vast sensor suite on and around the robot to achieve intention recognition. This relegates intuitive human-robot collaboration purely to such bulky systems, which are inadequate for large-scale, real-world scenarios due to their complexity and cost. In this paper we propose an intention recognition system that is based purely on a portable head-mounted display. In addition robot intention visualisation is also supported. We present experiments to show the quality of our human goal estimation component and some basic interactions with an industrial robot. HAIR should raise the quality of interaction between robots and humans, instead of such interactions raising the hair on the necks of the human coworkers.

Research paper: Puljiz, D., Zhou, B., Ma, K., and Hein, B., “HAIR: Head-mounted AR Intention Recognition”, 2021. Link:


Notify of
Inline Feedbacks
View all comments
Would love your thoughts, please comment.x