Multi-touch Interface for Controlling Multiple Mobile Robots

'08-09 Robot NUI ACM CHI '09 Student Research Competition

Abstract

Existing user interfaces for controlling mobile robots in real-time are generally for only one robot. Controlling multiple robots simultaneously is difficult. We developed a multi-touch interface with a top-down view from a ceiling-mounted camera for simultaneously controlling multiple mobile robots. Users touch and move their hands on the screen to specify the vector field, and all the robots move along the vector field. The test users in our study successfully operated multiple robots simultaneously with our interface.

This was my graduation thesis work and won 1st place at CHI 2009 Student Research Competition.

Multi-touch Multi-robot

Multi-touch Vector Field Operation

With our multi-touch interface, users can navigate multiple robots simultaneously through three operation modes. Please note that the users do not need to switch between these modes explicitly.

Drag
When users touch and drag their hands along the panel, a virtual stream appears on the vector field and robots move in accordance with the stream.
Mix
When users drag their fingers on existing streams, the vector data near the streams are blended in proportion to their distances from the streams. Nearer streams affect the vectors more strongly.
Hold
Touching the panel without further motion will reset the vectors under that hand. Thus, we can stop a robot by touching and holding an area in front of the robot.
Three operation modes

Publication