Yahoo: The Frost & Sullivan '2009 European Real-Time 3D Imaging Enabling Technology Innovation Award' is presented to PMDTechnologies GmbH (PMDTec). "Photonic mixer device (PMD) technology developed by PMDTechnologies GmbH has opened up opportunities to develop three-dimensional imaging devices that are low cost, complex and robust," notes Frost & Sullivan.
IFM Electronics GmbH, specializing in the industrial automation sensing, has included in its product offerings the 3D camera developed in cooperation with PMDTechnologies and provides this solution to about 80,000 customers worldwide. In co-operation with Audi and Volkswagen, PMDTechnologies is already in the pre-fabrication stage of 3D sensors for automotive applications. Moreover, with other partners, the company is also developing solutions for medical, security & surveillance, and gaming application areas.
Currently, PMDTechnologies is working on 0.18-micron CMOS technology integration. Such an approach would lead to the creation of even smaller 3D imaging devices which could be available at lower cost. The company envisions such a camera for below $30 soon. Such performance at this price point has not been achieved using other 3D approaches, e.g. fringe projection based 3D vision systems, so far [Questionable claim - ISW].
what is the impact of this on the roadmap of companies like Canesta , Microsoft/3DV, PrimeSense , etc.. ?
ReplyDeletePMD is an old technology, known for more than 12 years. I doubt that the recent award can have any direct impact on its competitors.
ReplyDeletePMD is not old, but established. And PMD is the only 3D ToF suplier, that really has mass products in a steadily growing number of markets. This can be seen online, e.g. on their partners websites... so to give the award to these guys seems to be a logical descision from Frost to me.
ReplyDeletePMDTec is younger than Canesta, PrimeSense and 3DV... by the way 3DV was not bought from MS because of their technology, which was not feasible for real low cost 3D imaging.
ReplyDeleteFrom PMDTec site:
ReplyDeletehttp://www.pmdtec.com/company/history/
S-TEC Sensor GmbH, founded in 1996, began developing PMD products in 1997. This company was acquired by PMDTec in 2002... PMDTec was founded in 2002.
From Canesta site:
http://canesta.com/corporate
Canesta was founded in April 1999.
3DV was founded in 1997:
http://www.linkedin.com/companies/3dv-systems
From Primesense site:
http://www.primesense.com/background
Prime Sense was founded in late 2005.
So, in terms of technology, PMD, 3DV and Canesta are about 12 years old, even though PMDTec as a company is younger than that. Primesense is 4 years old.
PMDTechnologies is the worldwide leading 3D time-of-flight Technique. It is standard CMOS and a low noise approach! The high noise current mixing Belgian approach is an old technology.
ReplyDeleteThanks to Frost & Sullivan!
ReplyDeleteCanesta copies the European 3D approaches and will have problems with the European patents
Not to detract from PMD value, but I have heard a lot of good words about Canesta camera. Also, Primesense technology works quite well, we all have seen it in Microsoft Project Natal demos.
ReplyDeletePrimeSense might be "only" 4 years old, but their approach to use structured light as an active illumination is pretty old (only the signal processing intergration is "new"). In contrast to that, 3D ToF as a technology (i.e. using CMOS integrated receivers) is clearly younger. Although demos might be nice it is easy to make them impressive - PMD is the only company with mass products... and they are also in that game with a number of partners as shown at PMD[vision] Day 2008. I have seen a lot of approaches incl. nice demos, but nothing afterwards. And Canesta indeed seems to copy things straight forward. Anyway, 3D ToF is getting stonger and stronger due to some clear advantages compared to stereo or structured light approaches due to cost and performance features - no doubt.
ReplyDeleteI have nothing against PMD, really. If PMD succeeds with its technology, I'm really happy to hear this. Said all this, $30 for a 3D camera sounds too much for a mass market product that appears few years from now. I think PMDTec needs to bring price closer to $10 to be really competitive. For $30 it will remain a niche product, in my opinion.
ReplyDeleteStructured light and stereo approaches have an advantage of using cheap off-the-shelf sensors, thus exploiting the mass market economics. I agree that ToF has few advantages, but my question is if ToF camp, as a whole, can translate them into a market share.
On the technical side, what is the shutter efficiency of PMD sensor, as compared with Vrije University? Can anyone combine Belgian's shutter efficiency with high QE in a single device? This could be a killer product.
I think many of the mainstream sensor companies may be entering the 3D market before long. This will result in both major ASP erosion as market growth and penetration increases.
ReplyDeleteThis will be bad news for the PMDs, Canestas, etc. While they may have some patents that are relevant (and even that is not for sure) the small companies are also wide open for patent infringement counter claims. Does a company like PMD know for sure that some transistor or some conventional circuit structure is not patented by some big semiconductor company? The big companies have mutual licensing agreements that the little companies don't enjoy and hence they are highly vulnerable to counter suits.
On the other hand, for some of these companies, the big companies offer an exit strategy thru acquisition. So, it will be very interesting to see how this all plays out.
Personally, I am a big fan of stereo for 3D despite its many shortcomings. Never underestimate the (future) power of digital image processing!
I agree with Mr. Fossum, that large companies have the abilities to be compeditive to PMD very fast for all high volume market. The stereo aproaches of course profit from the progesses in digital image processing but in the same way ToF approaches profit from the progesses in LED development. If you have no contrast in the scene, you have no 3D information - even if you use the IBM roadrunner computer.
ReplyDeleteIn my opinition there will be application for the stereo approach as well as for the ToF approach and of course there are also SME ToF Supplier next to the large companies.
Three comments on the discussion before:
ReplyDelete1. The age of the technology cannot be interpreted based on the date when a company was founded. Apparently for all kinds of technologies, the way to bring the idea to a mass-market product generally seems to be quite long and is often very underestimated. So although the basic underlying technology can be old, from a product specific point of view the technology can be considered to be young. And I think that this is the case for the 3D time-of-flihgt products.
It is a pitty that during this blog's discussion only PMD and Canesta as 3D time-of-flight suppliers were mentioned. It makes the whole story on time-of-flihgt imaging quite incomplete, in particular if the discussion is about the technology history but also about the presence. There are the swiss guys from MESA Imaging whose technology shows a first patent filing already in 1994 (at this time the technology was still develeoped at CSEM) and who had the first commercial 3D camera worldwide. (Just see their webpage www.mesa-imaging.ch). Also in terms of performance I heard a lot that their technology is superior to the others.
2. We have to distinguish between the different 3D measurement principles in general: stereo, time-of-flight and interferometry. Their advantages and disadvantages are fundamental of course. Just considering the stereo and time-of-flihgt since they allow for measuring medium to long distances, I think that the gre3at benefits of time-of-flight are (a) it is a real distance measurement and not estimate, and (b) the accuracy of the measurement directly benefits from the ongoing process miniaturization. Thus, depending on the applications a stereo system or a time-of-flihgt system might be better to be used but fundamentally in terms of a measurement tool the time-of-flight provides the most reliable measurement data and its performance is getting even better due to the ongoing miniaturization. Although the stereo systems benefit from increase of computation power there are some fundamental limits which will never enable this 3D imaging technique to be used in some certain applications (miniaturization is in contradiction with accuracy and all measurements are requiring textures).
3. Concerning the statement on the low noise performance of the current mixing approach of the Belgium guys, this seems to me it needs to be confirmed somehow. Is there a fair comparison between the approaches? Would be interesting.
Talking about different 3D techniques, you forgot structured light one. Primesense in the most prominent supporter of it, I believe. Both ToF and structured light, being active techniques, benefit from the fast LED progress. Stereo approach benefits from the fast image processing progress.
ReplyDeleteI agree with you that many other companies are working in this field, other than PMD and Canesta. As a matter of fact, Mesa Imaging link has been in the left column of this blog for years, it's hardly forgotten. There are just too many companies to mention in these comments.
BTW, I'm not sure the Brussel University approach is less noisy. It just has very good shutter efficiency that other ToF approaches lack.
Nothing to negate to the comments that tof and stereo will fight a competation. The advantage for stereo is and will be that the costs for processing power decreases over time. Moore has predicted this. But the applications in which tof will win the competition at any time are measuremts over long distances. The distance between the two lences in a stereo system is in first-order proportional to the distance which should be measured. With a tof system the advantages are considerable.
ReplyDeleteI am also interested in how all this plays out.
I believe that 3D ToF will make it.. it is just 1 CMOS chip, 1 lens, easier 3D computation and processing and the illumination is needed in many applications for stereo cams as well, as they need contrast. And stereo is really an old technological approach... just look at the enormous progress of 3D ToF cameras during thre last 2 years - just amazing. So the award for PMD makes sense to me.
ReplyDeleteI should say that I am a big fan of stereo for applications that try to record the sense of 3D as seen by a person. So, all the shortcomings of stereo imaging for 3D are pretty much the same as for a human looking at the same scene.
ReplyDeletePlus, stereo requires 2 sensors per camera. How bad is that??
For any sort of quantitative 3D imaging in controlled environments, TOF and structured light have large advantages. Take them outside, and well, things deteriorate quickly. And, so far no one has linked strong IR illumination to eye problems but it may take some years for those future gamers to show any problems. TOF gets problematic at long ranges, mostly because resolution requirements vary sort of linearly with range, and reflected power varies as the inverse of the square of range, assuming a sort of point source. The PMD light source housing is basically a large heat sink, for example, easily seen if one happens to take one apart.
Interesting discussion. By the way, the range resolution of both stereo and structured light approaches is proportional to square of range, similar to the SNR-limited ToF.
ReplyDeleteIt's really is funny to read, that a 3D stereo camera is compared with a sophisticated human vision system develped over millions of years. Keep in mind, that ressources (e.g. time, money, processing power, experiments...) were not limited in natures evolution! This is complently different in a technical product development, where nearly everything is limited! I agree that reducing a technical solution to the minimum effort that is possible to realize a solution (e.g. 3D ToF with 1 chip, 1 lens, low 3D pre-processing) is a strong approach, and will find it's way in many applications. And to use 2 normal high-res images to calculate a low-res 3D image only makes sense, when ressources are not limited at all. And that might be the case in demos with a lot of calculating horse power, but not when it comes simply down to a funcionality to be fulfilled at low cost! Ilumination is an issue, but new PMD cameras are USB powered and this includes the active illumination!
ReplyDeleteClearly, stereo 3D technique is more computationally intensive. Still, stereo companies like TYZX claim to be able to compute 3D images with very high frame rates (200 fps for 512x480 images) and low power requirements (<1 watt, well within USB power limits):
ReplyDeletehttp://www.tyzx.com/PDFs/tyzx_wp.pdf
It always depends on the application. Even in natures evolution, TOF systems were developed, very robust and for hard environments and especially when calculating power was not "unlimited". So I agree to the guys above that say that the award to PMD really makes a lot sense. By the way, eye safety is not a problem for the PMD cameras that are already on the market. And also: illumination is needed for many applications, no matter if a passive approach is used - so this must not be an disadvantage for PMD. As PMDs ate only sensitive to the correlated signal, they are very robust in environments, where lighting situations change. And the enormous calculating power availabe today can be used by PMD cameras as well, not to extract 3D information (which is directly detected without calculation), but to solve the needed functionality ny processing. How bad is that?
ReplyDelete"Plus, stereo requires 2 sensors per camera. How bad is that??" It is bad if the market requires those 2 sensors to cost the same as 1.
ReplyDeleteIntesting Discussion:
ReplyDeleteIn my opinion stereo is not a real 3D approach because you need contrast in the scene. No contrast = No 3D information!
On the other side, even if you have contrast stereo has a bad accuracy in longer ranges or you increase the base of both cameras enormously.
ToF approaches like PMD have no range limits!
PMD measures more than 80m today in automotive application.
MESA Imaging uses the same approach then PMD. If they use the patent from 1994 they waste light!
ReplyDeleteInteresting discussion, indeed.
ReplyDeleteTrue, stereo approach needs some scene contrast to operate. But all technologies have some weak spots either here or there. For example, if in demo room few PMD systems are placed close to each other, do they interfere?
Talking about the range limits, active methods need to increase the IR source power to extend the range. Then, to remain eye-safe, the exit pupil of the IR source should be increased or more and farther distributed LEDs used (if no active protection is employed). I do not have the numbers in hand, but it looks to me that that just like in case of stereo, PMD system size would become bigger too. Am I right?
On the other hand, stereo system can extend its range either by increasing the sensor and optics resolution (and pay computing power price) or by extending the stereo base (and pay the system size price).
All the above assumes that we maintain the same field of view and range measurement accuracy. Otherwise the comparisons would not hold.
Hi, this is simply not right. A PMD-systems performance depends invers linear on the modulation frequency that is used. This defines the accuracy of the distance measurement. Of course, reflectivity for PMD and contrast for stereo have influence, but a PMD system can measure by principle independent from a targets distance with the same accuracy/standard deviation... stereo can't!!!
ReplyDeleteOK, PMD device compares in-phase and out-of-phase signals to get the range, right? This comparison requires a decent SNR. The SNR depends from the transmitted signal level, field of view, target reflectivity and the range. All things being equal, the bigger the range, the higher transmitted power should be to maintain this decent SNR. Am I correct?
ReplyDeleteRight, but comparing the SNR with the size of the system (more light = more volume) is not valid. Again, a stereo camera can't detect anything witout contrast (which is comparable to TOF's SNR issues). But in TOF systems like PMD there are, in practical applications, enough pixel in the scene delivering "more than enough" reflected signal, thus the SNR does not limit the resolution. Again, the stereo depth resolution depends on the distance by principle... you have to enhance the optical base length in order to measure with stable accuracy in higher distances. TOF does not have this effect, which is a big advantage against any triangulation principle (e.g. stereo). Am I right?
ReplyDeleteIn stereo 3D system one needs either to enhance the optical base or to increase the number of pixels to be able to detect smaller stereo differences. So, stereo approach benefits from the industry-wide pixel race.
ReplyDelete@ But in TOF systems like PMD there are, in practical applications, enough pixel in the scene delivering "more than enough" reflected signal, thus the SNR does not limit the resolution.
If not SNR, what else limits PMD resolution?
It is SNR, of course, but also the frequency can be exploited to counteract low SNR.
ReplyDeleteA big disadvantage of PMD systems are strong distance drifts due to system heating. Judst imagine, signal delays of just a few nanoseconds due to temperature drift cause already a few dozens of centimeters errors (if using some MHz of modulation). PMD usually try to compensate that by actively cooling the systems with fans, which is particularly required due to their powerful illumination modules acting as huge heaters.
Every measurement system is limited due to SNR. ToF hold no exception. It is also true that standard deviation due to temperature shifts are a problem. But this fight between the salesmen who are blogging here displaces the discussion to a different direction. The initial question is: What is the influence of SNR in stereo and what in ToF systems? Due to triangulation you need a high resolution imager (small pixel causes noise) to reach depth information in combination whith a specified base. So you always need two or more pixel to dissolve the depth. In comparison to ToF, I suggest that the die area which is covered through the number of stereo pixel may only be one ToF pixel. On the one hand a pixel whith increased sensitivity is reached. On the other hand this is necessary because of the detection of the reflected modulated light. The noise in stereo system is influenced through the contrast of the scene, in ToF it is influenced through non coharent light and modulation frequency. Has anyone written a paper (no promotion stuff) in which stereo is compared to ToF, are there conferences about this stuff?
ReplyDeleteI think it's hard to compare stereo with ToF in general terms. True, ToF can generally use less pixels for the same depth resolution. However, ToF pixels use IR light, thus have lower QE and higher crosstalk than commonly used stereo sensors. Also, most ToF sensor implementations are not very effective in terms of QE and switching of charge collection paths is far from ideal.
ReplyDeleteAlso, stereo approach relies on regular light source which does not have the power limitations of ToF illumination sources.
Talking about the price of whole system, many gaiming and human interface applications include web cam too. Stereo approach can deliver webcam functionality almost for free, while ToF needs a separate camera.
Also, both ToF and stereo potentially could compensate for their deficiencies. For example, stereo camp could add a structured light illuminator to deal with low-contrast scenes, obviously, for the additional system price.
So, it would be hard to compile a fair comparison between all the 3D imaging schemes.
One comment concerning PMD and MESA:
ReplyDeletePMD has 3 very successful products with CE Certification for about 800EUR!
PMD is an established serial product since 5 years with all quality checks.
Serious companies choose the serial approved PMD approach instead of the university swiss approach.
This is awarded by Frost & Sullivan !
The new Belgian Company has defined a PPI (Pixel Performance Indicator) for quality of their approach. The receive a PPI = 1,18!
ReplyDeleteIn comparison PMD receive a PPI = 12,235 due to a incredible high bandwidth and a enormously better noise performance which is caused by established Photogate Technology.
By the way: Photogates are transpartent for infrared light!