Camera–Based Interfaces for Mobile Devices


Designing comfortable user interfaces for mobile phones is a challenging problem, given the limited amount of interaction hardware of the device. However, multiple built-in cameras and the small size of handhelds are under-exploited assets for creating novel applications that are ideal for pocket size devices, but may not make much sense with personal or laptop computers. Studies into alternatives to mobile user interaction have, therefore, become a very active research area in recent years.

A key advantage of using cameras as an input modality is that it enables recognizing the 3-D context in real-time, and at the same time provides for single-handed operations in which the users' actions are interpreted without touching the screen or keypad. For example, the user's position and gaze can be measured, in order to display true 3-D objects even on a typical 2-D screen.

Our unit is one of the pioneers in developing computer vision solutions for mobile phones. For example, we have introduced an efficient motion estimation framework for mobile phones. The software was included to Nokia Computer Vision Library in 2007. Another example of using the camera as a sensor is an interactive panorama image constructor for mobile phones. The image capturing process analyzes the displacements between video frames, measures the quality of the frames based on the motion blur and focus, and selects the suitable frames for mosaicing. The implementation includes a parallel pipeline that uses the mobile Graphical Processing Unit (GPU) integrated on the platforms, reducing the computation and power needs of the application.

Our research on camera-based mobile user interfaces focuses on two subject areas:

Interactive camera-based applications

* Virtual 3-D display based on real-time face tracking

* Image browsing with motion estimation

* Interactive document scanner

* Video-based panorama imaging

* Finger-tracking browser

* Multimodal user interfaces

Efficient mobile video analytics

* Mobile computer vision using GPU acceleration

* Energy efficient analysis

* Sensor-data integration


Selected publications:

Hannuksela J, Sangi P & Heikkilä J (2005) A vision-based approach for controlling user interfaces of mobile devices. Proc. IEEE CVPR 2005 Workshop on Vision for Human-Computer Interaction, San Diego, CA, USA, 6 p.

Hannuksela J, Sangi P & Heikkilä J (2007) Vision-based motion estimation for interaction with mobile devices. Computer Vision and Image Understanding: Special Issue on Vision for Human-Computer Interaction, 108(1-2):188-195.

Hannuksela J, Sangi P, Heikkilä J, Liu X & Doermann D (2007) Document image mosaicing with mobile phones. Proc. 14th International Conference on Image Analysis and Processing, Modena, Italy, 575-580.

Boutellier J, Bordallo López M, Silvén O, Tico M & Vehviläinen M (2007) Creating panoramas on mobile phones. Proc. SPIE Electronic Imaging 2007, San Jose, CA, USA, 6498(07).

Bordallo López M, Hannuksela J, Silvén O & Vehviläinen M (2009) Graphics hardware accelerated panorama builder for mobile phones. Proc. SPIE Multimedia on Mobile Devices 2009, San Jose, CA, USA, 7256(72560D).


CMV/Research/Camera-BasedInterfacesForMobileDevices (last edited 2011-11-19 15:09:27 by WebMaster)