Tuesday, October 09, 2012

Microsoft Presents Alternative Gesture Recognition System

BBC: Microsoft's Digits prototype is part of an effort to bring gesture-based control in a mobile device. A wrist-worn sensor uses IR laser, LEDs and camera to recognize the fingers movement, as shown in Microsoft's paper:

Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor.
David Kim (Newcastle University, Newcastle upon Tyne & Microsoft Research, Cambridge, UK)
Otmar Hilliges (Microsoft Research, Cambridge, UK)
Shahram Izadi (Microsoft Research, Cambridge, UK)
Alex D. Butler (Microsoft Research, Cambridge, UK)
Jiawen Chen (Microsoft Research, Cambridge, UK)
Iason Oikonomidis (Microsoft Research, Cambridge, UK & University of Crete, Heraklion, Greece)
Patrick Olivier (Newcastle University, Newcastle upon Tyne, UK)

Presented at the 25th annual ACM symposium on User interface software and technology, Oct. 7-10, 2012 in Cambridge, MA, USA.

While the prototype is quite big, the company would like to shrink it a size of wrist watch that can be worn all the time. An Youtube video shows Digits sensor in action. A longer version of this video from Microsoft site also offers some insight into the algorithms used for fingers recognition, talks about functions of laser and LEDs and more:

Get Microsoft Silverlight Microsoft's team acknowledged the current device was still some way from being ready for market. It currently needs to be attached to a PC to carry out the necessary computations, making it impractical for real-world use. It also struggles if two fingers are crossed, the hand is flattened or if the user is holding something while making the gestures. However, the researchers suggested all these issues could be overcome with further work.


  1. I see a large increase in carpal tunnel syndrome cases coming....

    1. Actually this will probably lessen the occurence of carpal tunnel by allowing the user to be in a more comfortable position as opposed to going to the mouse or keyboard and adapting to them.

  2. Wouldn't gloves with finger position sensors be simpler and more effective? Use them in any environment, and detect touch as well as position?

    Seems like this is an attempt to do something optically where other technologies already provide a superior and simpler solution.

    1. your hand is free and not instrumented.

    2. Compare this to the advantage optical mice have brought to the table. Gloves can get grimy awfully fast...

  3. Gloves come in many designs and are worn for all manner jobs, both rough (construction) and delicate (surgeons). Instrumented gloves need not be bulky or insensitive.

  4. For fun, some key creative market segments...

    ...In education, to detect when students are using smart phones during examinations. Note that release into a previously unrecognized market of such vast size will probably trigger a wave of technological innovation in areas of interface and control. Will we see development of the Whisper-Ma-Phone, and will the PTO be able to find any prior art?

    ...In public health, to detect when citizens in Singapore who chew gum take it out of their mouths and throw it on the ground. The same device can be used to alert authorities of inappropriate hand gestures made to passing American humorists.

    ...In sport, to keep cycling clean. Sacrebleu! Oh, never mind...


  5. Nothing is better than a simple mouse!


All comments are moderated to avoid spam and personal attacks.