Microsoft Hololens Challenges

EETimes quotes Marc Pollefeys, Microsoft director of science who runs a computer vision lab at ETH Zurich:

“Most of the energy is spent moving bits around … so it would seem natural that … the first layers of processing should happen in the sensor,” Pollefeys told EE Times in a brief interview... “I’m following the neuromorphic work that promises very power-efficient systems with layers of processing in the sensor — that’s a direction where we need a lot of innovation — it’s the only way to get a device that’s not heavier than glasses and can do tracking all day.”

Researchers are still working on ways to map a user’s hands accurately into an environment so that they can be used to control virtual objects. Occlusions, segmentation failures, and noisy data have hindered such efforts for years, Pollefeys said."

MS Hololens Block Diagram