Touch Projector

Mobile Interaction Through Video

(in collaboration LMU Munich)

In 1992, Tani et al. proposed remotely operating machines in a factory by manipulating a live video image on a computer screen. In this paper we revisit this metaphor and investigate its suitability for mobile use. We present Touch Projector, a system that enables users to interact with remote screens through a live video image on their mobile device. The handheld device tracks itself with respect to the surrounding displays. Touch on the video image is “projected” onto the target display in view, as if it had occurred there. This literal adaptation of Tani’s idea, however, fails because handheld video does not offer enough stability and control to enable precise manipulation. We address this with a series of improvements, including zooming and freezing the video image. In a user study, participants selected targets and dragged targets between displays using the literal and three improved versions. We found that participants achieved highest performance with automatic zooming and temporary image freezing.

Touch Projector allows users to manipulate content on distant displays that are unreachable, such as (a) displays outside a window, or (b) a tabletop system crowded with people. It allows users to manipulate devices that are incapable of touch interaction, such as (c) a wall projection or (d) a laptop. Users point the device at the respective display and manipulate its content by touching and dragging objects in live video. The device “projects” the touch input onto the target display, which acts as if it had occurred on itself.


Boring, S., Baur, D., Butz, A., Gustafson, S., and Baudisch, P. Touch Projector: Mobile Interaction Through Video. To appear in Proceedings of CHI 2010, 10 pages.
DOC (0.5M) | PPT+Video (143.9M) | WMV (46.3M)