Google has published a video on status of its Project Tango dealing with mobile 3D tracking and perception. The latest version of Tango smartphone includes PMD ToF camera (see PMD PR here), together with two other rear side cameras, see minute 31 in the video:
I don't know whether those schemes are actually competing. Both have their clear pros and cons and you'll certainly find applications in which you'd prefer the one or the other. Structured light or triangulation in general can be used to achieve depth maps of fairly high resolution. But you need a large triangulation base to achieve decent precision - difficult to do in a mobile device... A large triangulation base also makes life hard when you have to do phase unwrapping. This un-wrapping needs quite some computational effort - also difficult in mobile devices. ToF on the other hand is coaxial and doesn't need as much computational effort - this makes it predestined for ranging in handhelds. However, image resolution is typically much smaller and it's also not so easy to get decent real-time images at high ambient light/low reflection/long distance.
Concluding, I see ToF as the clear winner in handhelds and structured light as beneficial for e.g. broadcasting where you rely on high image resolution and where you can perform phase un-wrapping after filming the scene (at least until ToF imagers have higher resolution).
Regarding mobile application, is the power required for active illumination of ToF less than than the power required for computational effort of triangulation systems?
Is it that clear that ToF is more suitable for mobile device?
Lastly, you did not mention the accuracy. Can you say that ToF is less accurate than the triangulation systems?
Your first question is fair but not so easy to answer. I do not know the power consumption of the phase unwrapping by heart. Concerning the power consumption of the active illumination in ToF it also depends on your application - scales with range, desired precision, frame-rate... But don’t forget that also structured light uses an active illumination! Irrespective of the power consumption, you would want to have real-time capability for gesture recognition applications. I think that this should be fairly difficult to achieve with structured light on a mobile device.
Concerning accuracy - I pointed out that in triangulation good precision requires a sufficiently large triangulation base. In a handheld you have very little room for improvement as the triangulation base is limited by the dimension of the handheld device. Obviously this is easier achieved in a tablet than in e.g. a mobile phone. This was actually one reason why I claimed that ToF should be better suited for mobile applications as in ToF you don't need a triangulation base. You have a coaxial setup.
Those are though all qualitative claims. If you want to quantitatively express how much better or worse these schemes are for your application, you'd need to do a case study...
If I understand you properly, you seem to be searching for general claims on what is better or worse. I think it's not that easy though. You really have to know what you want to do to understand what is better. For instance, I surely think that there are applications where you can end up with a better precision/accuracy with a ToF systems – especially if you have to ensure a small packaging size. However, if you can built a system arbitrarily large, you can also achieve a very good precision with triangulation. Such general claims simply doesn't make sense.
If you want to get some insights in the different techniques, I recommend the chapter "Three-Dimensional Imaging Techniques" by Rudolf Schwarte in "Computer Vision and Applications" edited by Bernd Jähne and Horst Haußecker. Prof. Schwarte nicely explains the trade-offs in triangulation/interferometry and time-of-flight imaging. Also you find typical precision values for typical ranges. That should give you a feeling for the differences of these systems. But don't take those data too serious as these systems are continuously progressing and the book is not so recent anymore... ;-D
Thanks for the answer. I remember the uncertainty graph given in the chapter you have mentioned from Lange's PhD thesis. According to the graph, ToF's uncertainty is higher than the triangulation.
Apart from mobile usage, I am also evaluating whether ToF can be succesfully employed in an application with sub milimeter accuracy.
ToF has been hot topic in the last years and the major driving topic for it has been UI development. German automobile producers has failed and stopped working with ToF considering that ToF is highly affected by weather conditions and range is limited.
Plus, there is not much development in the industrial side where you usually require high precision.
I think the correct interpretation is not that triangulation is less precise than ToF, but that ToF is "typically" more suitable for a certain range and triangulation for another. In triangulation precision scales with z^2 - in scanning direct ToF depth precision can be fairly independent (meaning much, much less dependent - of course not entirely independent) on distance. In ToF you have multiple techniques that just grew after Prof. Schwarte wrote his chapter and after Dr. Lange wrote his dissertation. Their not all the same, so don't simply believe those plots from 15 years ago, which were highly simplified even back then...
Surely ToF can in principle achieve sub millimeter accuracy - if you can sacrifice e.g. frame rate, spatial resolution and/or power consumption, if you don't need to measure at high ambient levels and if your objects are highly reflective and if you don't care about e.g. eye-safety.
You say, you are evaluating. Who are you actually by the way?
It's not true that automobile producers stopped working on ToF. Most of the recent LIDAR developments are basically ToF. In automotive you have for instance velodyne or Bosch - the latter being German. Also PMDtec (German) is still advertising solutions for the automotive market - have a look at their website. Automotive is big. Apart from long distance ranging for e.g. crash prevention you can also benefit with ToF in applications as parking assistance etc. Also I would not join your statements on industrial imaging. PMD and MESA (both German again) are quite successful in the industrial market. MESA was just bought by Heptagon last year...
I did not mean "triangulation is less precise than ToF", I just meant the opposite. Or better said, if you want to build an accurate 3D camera for example for an industrial inspection application, you would not go for ToF. I was basically referring whether this would be correct statement.
Related to automative usage, I knew that Audi and BMW were evaluating ToF systems for park assistance and crash prevention but as far as I know they shut down those programs or at least I did not see any successful product. ToF has problems under direct sun light and under foggy weather or rainy. Mercedes is using stereo for their auto suspension adjustment based on road surface feature. I believe at least at current level ToF is not suitable for automative systems. UI (gesture control) is the hot topic for ToF also in automative.(I differentiate LIDAR scanners from 3D TOF).This is what I think and I would be more than happy if you disagree and help me understand how I am mistaken.
Mesa was Swiss, not German. All I the impression I get from PMDTec's website is that they are going in the mobile or consumer and UI direction.
Hi Onur, triangulation being less precise than ToF was a typo. But I think that could've been understood from the context. I would in general never agree on such general claims. You always have to specify what you want. I already stated several points which can justify a decision of ToF over triangulation – also w.r.t. accuracy. So no, I don't think your statement is correct. Not as general as you make it.
Concerning the rest, I share the opinion that current ToF technologies are not as mature as necessary for products for e.g. crash prevention. But I don't see any alternatives out there either. I think that in principle ToF is still a well-suited technology to fulfill the requirements for such applications in the future, but obviously some things need to be improved (in my opinion mainly performance under harsh environment conditions & framerate).
Concerning PMD, surely they're looking into mobile. That is obvious concerning their activities in the Google Tango project. But historically they've been doing a lot also in industrial imaging together with ifm (which by now holds 100 % of their shares).
As our discussion is quite extensive by now, I propose to take this offline. I think good opportunities to discuss this further would be for instance the IISW or the Harvest Imaging 3D Imaging ToF forum. Maybe there you could also involve more opinions than mine...
Why would anybody think of using tof for sub millimeter precision if it is obvious from literature that triangulation or interferometry are supposed to be superior? What is the motivation behind going for tof?
Regarding power consumption I am wondering as well if tof makes sense on a tango like app. We kow from tof cameras such as Mesa (which is Swiss) and Kinect that this tech requires a lot of optical power of some Watts in order to cope measurement ranges of some meters. Is that suitable for a mobile device?
Has ToF won the competition against structured light?
ReplyDeleteI don't know whether those schemes are actually competing. Both have their clear pros and cons and you'll certainly find applications in which you'd prefer the one or the other. Structured light or triangulation in general can be used to achieve depth maps of fairly high resolution. But you need a large triangulation base to achieve decent precision - difficult to do in a mobile device... A large triangulation base also makes life hard when you have to do phase unwrapping. This un-wrapping needs quite some computational effort - also difficult in mobile devices. ToF on the other hand is coaxial and doesn't need as much computational effort - this makes it predestined for ranging in handhelds. However, image resolution is typically much smaller and it's also not so easy to get decent real-time images at high ambient light/low reflection/long distance.
DeleteConcluding, I see ToF as the clear winner in handhelds and structured light as beneficial for e.g. broadcasting where you rely on high image resolution and where you can perform phase un-wrapping after filming the scene (at least until ToF imagers have higher resolution).
- Andreas Süss
Regarding mobile application, is the power required for active illumination of ToF less than than the power required for computational effort of triangulation systems?
DeleteIs it that clear that ToF is more suitable for mobile device?
Lastly, you did not mention the accuracy. Can you say that ToF is less accurate than the triangulation systems?
Your first question is fair but not so easy to answer. I do not know the power consumption of the phase unwrapping by heart. Concerning the power consumption of the active illumination in ToF it also depends on your application - scales with range, desired precision, frame-rate... But don’t forget that also structured light uses an active illumination! Irrespective of the power consumption, you would want to have real-time capability for gesture recognition applications. I think that this should be fairly difficult to achieve with structured light on a mobile device.
DeleteConcerning accuracy - I pointed out that in triangulation good precision requires a sufficiently large triangulation base. In a handheld you have very little room for improvement as the triangulation base is limited by the dimension of the handheld device. Obviously this is easier achieved in a tablet than in e.g. a mobile phone. This was actually one reason why I claimed that ToF should be better suited for mobile applications as in ToF you don't need a triangulation base. You have a coaxial setup.
Those are though all qualitative claims. If you want to quantitatively express how much better or worse these schemes are for your application, you'd need to do a case study...
If I understand you properly, you seem to be searching for general claims on what is better or worse. I think it's not that easy though. You really have to know what you want to do to understand what is better. For instance, I surely think that there are applications where you can end up with a better precision/accuracy with a ToF systems – especially if you have to ensure a small packaging size. However, if you can built a system arbitrarily large, you can also achieve a very good precision with triangulation. Such general claims simply doesn't make sense.
If you want to get some insights in the different techniques, I recommend the chapter "Three-Dimensional Imaging Techniques" by Rudolf Schwarte in "Computer Vision and Applications" edited by Bernd Jähne and Horst Haußecker. Prof. Schwarte nicely explains the trade-offs in triangulation/interferometry and time-of-flight imaging. Also you find typical precision values for typical ranges. That should give you a feeling for the differences of these systems. But don't take those data too serious as these systems are continuously progressing and the book is not so recent anymore... ;-D
-- Andreas Süss
Thanks for the answer. I remember the uncertainty graph given in the chapter you have mentioned from Lange's PhD thesis. According to the graph, ToF's uncertainty is higher than the triangulation.
DeleteApart from mobile usage, I am also evaluating whether ToF can be succesfully employed in an application with sub milimeter accuracy.
ToF has been hot topic in the last years and the major driving topic for it has been UI development. German automobile producers has failed and stopped working with ToF considering that ToF is highly affected by weather conditions and range is limited.
Plus, there is not much development in the industrial side where you usually require high precision.
I think the correct interpretation is not that triangulation is less precise than ToF, but that ToF is "typically" more suitable for a certain range and triangulation for another. In triangulation precision scales with z^2 - in scanning direct ToF depth precision can be fairly independent (meaning much, much less dependent - of course not entirely independent) on distance. In ToF you have multiple techniques that just grew after Prof. Schwarte wrote his chapter and after Dr. Lange wrote his dissertation. Their not all the same, so don't simply believe those plots from 15 years ago, which were highly simplified even back then...
DeleteSurely ToF can in principle achieve sub millimeter accuracy - if you can sacrifice e.g. frame rate, spatial resolution and/or power consumption, if you don't need to measure at high ambient levels and if your objects are highly reflective and if you don't care about e.g. eye-safety.
You say, you are evaluating. Who are you actually by the way?
It's not true that automobile producers stopped working on ToF. Most of the recent LIDAR developments are basically ToF. In automotive you have for instance velodyne or Bosch - the latter being German. Also PMDtec (German) is still advertising solutions for the automotive market - have a look at their website. Automotive is big. Apart from long distance ranging for e.g. crash prevention you can also benefit with ToF in applications as parking assistance etc. Also I would not join your statements on industrial imaging. PMD and MESA (both German again) are quite successful in the industrial market. MESA was just bought by Heptagon last year...
-- Andreas Süss
I did not mean "triangulation is less precise than ToF", I just meant the opposite. Or better said, if you want to build an accurate 3D camera for example for an industrial inspection application, you would not go for ToF. I was basically referring whether this would be correct statement.
DeleteRelated to automative usage, I knew that Audi and BMW were evaluating ToF systems for park assistance and crash prevention but as far as I know they shut down those programs or at least I did not see any successful product. ToF has problems under direct sun light and under foggy weather or rainy. Mercedes is using stereo for their auto suspension adjustment based on road surface feature. I believe at least at current level ToF is not suitable for automative systems. UI (gesture control) is the hot topic for ToF also in automative.(I differentiate LIDAR scanners from 3D TOF).This is what I think and I would be more than happy if you disagree and help me understand how I am mistaken.
Mesa was Swiss, not German. All I the impression I get from PMDTec's website is that they are going in the mobile or consumer and UI direction.
Onur Andic (This is who I am, by the way)
Hi Onur,
Deletetriangulation being less precise than ToF was a typo. But I think that could've been understood from the context. I would in general never agree on such general claims. You always have to specify what you want. I already stated several points which can justify a decision of ToF over triangulation – also w.r.t. accuracy. So no, I don't think your statement is correct. Not as general as you make it.
Concerning the rest, I share the opinion that current ToF technologies are not as mature as necessary for products for e.g. crash prevention. But I don't see any alternatives out there either. I think that in principle ToF is still a well-suited technology to fulfill the requirements for such applications in the future, but obviously some things need to be improved (in my opinion mainly performance under harsh environment conditions & framerate).
Concerning PMD, surely they're looking into mobile. That is obvious concerning their activities in the Google Tango project. But historically they've been doing a lot also in industrial imaging together with ifm (which by now holds 100 % of their shares).
As our discussion is quite extensive by now, I propose to take this offline. I think good opportunities to discuss this further would be for instance the IISW or the Harvest Imaging 3D Imaging ToF forum. Maybe there you could also involve more opinions than mine...
-- Andreas Süss
Why would anybody think of using tof for sub millimeter precision if it is obvious from literature that triangulation or interferometry are supposed to be superior? What is the motivation behind going for tof?
ReplyDeleteRegarding power consumption I am wondering as well if tof makes sense on a tango like app. We kow from tof cameras such as Mesa (which is Swiss) and Kinect that this tech requires a lot of optical power of some Watts in order to cope measurement ranges of some meters. Is that suitable for a mobile device?