"Event-based vision is poised to take over from the frame-based approach used by traditional film, digital and mobile phone cameras in many machine-vision applications. The mode of operation of state-of-the-art image sensors is useful for exactly one thing: photography, i.e. for taking an image of a still scene.
An ´ideal´ image sensor samples parts of the scene that contain fast motion and changes at high sampling rates and slow changing parts at slow rates, all at the same time – with the sampling rate going to zero if nothing changes. Obviously, this will not work using one common single sampling rate, the frame rate, for all pixels of a sensor. Conversely, one wants to have as many sampling rates as there are pixel in the sensor – and let each pixel’s sampling rate adapt to the part of the scene it sees."
Prophesee has a marketing division and website with big self confidence. I dont see their point of domination soon. Their demos on Vision booth (and also the demos of the other 2 event based suppliers) indicate to me that their technology is still searching for applications - and that is also unclear to them what this applications areas will be where DVS will dominate.
ReplyDeleteI see 2 major problems: 1.: in classical PC based machine vision environments one big problem is real time once actorics come into play. None of the demos had a link to a servo controller. If the camera should take the role of an intelligent encoder to make a servo controler act according to image information, the algorithmics will be very specific, running in some kind of real time embedded environment.
For other things where you can react a bit later (e.g. inspection where it does not matter if you decide for good/bad in 1 or 10 ms in most cases) there could be interesting applications, nevertheless the algorithmics on events is a bit tricky, there is one central point - problem nr2: once scene comes to standstill its over with events. So you need a movement in any case. They have grey value integrated that can be used for camera calibration with static targets etc. but the gery level images cannot compete with a classical 2D sensor.
I think it will take many years until domination of DVS if the areas of applications are not really know today - if DVS domination will ever become reality. I think DVS will be a niche player for quite some time in machine vision.
Fully agree: interesting new concept for imaging industry, however likely limited real-world application use-cases... I bet they will take 10 years to identify and develop in a few niches where their technology provides real differentiating value. I believe they have spun-out too early... in less than 15 years, the market might be starting (if it starts) while most of their core IP patents will have fallen into public domain... sad :/
ReplyDeleteWhen reading such comments without seeing a signature, I feel like I am on a teenager forum.
DeleteTrolls are really everywhere on the damn web.
I quickly read the wikipedia article about 'troll' https://en.wikipedia.org/wiki/Internet_troll and the postings above. In my opinion the postings above dont fullfil the definition of troll postings.
DeleteIn our environment, posting anonymous can have the motivation that one is not allowed to disclose the topics he is working on. If the name is public, the company is public.
The prophesee would have been announced too early then? =)
ReplyDeleteEarly innovators have to provide the complete plug and play solution to a real problem, a technology brick like a sensor will not do. There has been a few successes down that path, think Mobileye, or Da Vinci robotics. I think bold ambitions when there is significant science and technology behind those claims are ok. Then some will always recall that there are also a lot of failed attempts to change the imaging world such as Foveon, Pelican Imaging, Invisage,… but who are we to make fun of the attempts? There are so many companies, and not small ones, involved in DVS right now, Sony and Samsung among others, that this approach is certainly not so ahead of times. The way machine vision has evolved so far is not well suited to automation, as we all know industrial machine vision is extremely segmented so there is definitely some space for DVS there. On top of that robotic vehicles and Level 4/5 ADAS are needing such technology right now. More globally we live in a world on the verge of a complete robotic transformation. This paradigm shift is powered by Artificial Intelligence which evolves from the transformation of traditional computing. There is definitely a move to be done away from Von Neumann architectures to fully get the benefit from DVS, this will be difficult, but this is also where the technology is going. I do not understand negative comments on such great advances. Is this a misunderstanding of the tech behind? A misunderstanding of the potential of such innovation? Yes our industry is still in its infancy and many innovators are still trying new things. Let’s not nail to the ground these great startups that we have, what would be social media or car safety without digital imaging? Our industry is changing the world, please do not disturb the ones that are making it this way Pierre Cambou – Yole Développement
ReplyDeleteSalut Pierre, I (the OP) am not negative about DVS, I'm more enthusiastic than negative. And I also tried and try to use it in context of robotics-machine vision (actorics acting depending on infos from camera image). We see the benefit and that you can reach areas that you cannot reach with frame based images. And I agree, there are paradigm shifts in many areas. Nevertheless I dont see a domination coming soon. DVS will add partly redundant sensory data as you said to ADAS, as LIDAR and RADAR do. It will help to solve some parts of the ADAS problem. But I doubt it will replace frame based cameras in machine vision/robotics or be disruptive in a way to kill this frame based approach. DVS will find its niches and I really hope it will be successful, simply because its such a cool technology. But I think especially around AIish topics - there is a lot of hype these days... After trying to really use it and apply DVS and other AIish topics to real world problems in real world environment for a while, I came to the point to think that some aspects are overhyped.
ReplyDeleteDVS is not up to replace frame based cameras although it can work alone sometime. Rather, DVS works together with frame based cameras, lidar, radar... to better handle cases where frame based cameras have problems with.
DeleteThe marketing style of Prophesee can be discussed, but from scientific point of view, an event based system is made to reduce the redundency in the raw data. This data redundency is not only application dependent but also context dependent. The sequential aspect of Von Neumann architecture is not a conceptual limitation but a fundamental causality in the nature. That is why parallel computing is always context dependent.
ReplyDeleteA clockless causality control can be even more difficult than a clocked control. In the history, data-driven computer, event-driven asynchronous computing failed to find large applications.
For most of the production oriented machine vision, I cannot see any benefice of event-driven sensor and computing. Just imaging how it is bothering to read a barcode with their event driven machine...
reading a barcode is not a DVS application - it is reading a foto, the barcode is static. This is an example for what I meant. I think DVS will not be disruptive to frame based imaging in a way CIS was to film cameras. It cannot solve the same tasks in a better way and these tasks will remain in machine vision also in the future. Nevertheless the combination of frame based and DVS is promising to me - solve some of motion problems (that are currently unsolveable by cameras) with DVS and the classical problems with frame based cameras. Time will tell if solving these problems is more economical using DVS than other approaches (e.g. simply wait longer or use another type of encoder). And the von neuman discussions.. in classical environments (if you for example want to update a servo drive with the position of an object tracked with DVS camera) - you end up in a frame analogy, since if you want to update the position with lets say 2kHz, you process the events that occur within 500us at a time and send this to the controller. Most of these have time slice type of architecture so something like a 'frame'... you can calculate the position within this frame better than with a 500us exposure since you have the timestamps. But on the other side its harder because you lose the reduncancy that helps to locate the object precisely. but again, I am not negative at all - but already using it ;-)
DeleteI think I was wrong, I think reading a barcode is quite similar in DVS than in frame based imaging. "draw" the events of 1ms into a 2D matrix/image - you directly get the edges of your barcode, the "on" events give you one side of the code, the "off" edges the other, depending on color and moving direction. reading barcode that are printed on fast moving objects could be easier for DVS than for frame based/synchronized pulsed light
Delete"I think DVS will not be disruptive to frame based imaging in a way CIS was to film cameras. It cannot solve the same tasks in a better way". We have a fundamental difference in understanding of what "disruptive" means. I do refer to Clayton Christensen concept, and I can tell you this is my every day filter of understanding, I do not use the word lightly. I believe DVS is disruptive therefore I also believe it will never “solve the same tasks in a better way”. Because it is disruptive it will have to solve a new kind of problem for a new market, think 3.5 inch disk drive for laptop computers (the original example from CC). CIS provided a cheap digital imaging option for web cameras, it never intended to “solve the same tasks in a better way” than let say photography’s film or camcorder’s CCD of the 90’s. CC postulated that eventually the disruptive technology will outperform the incumbent, but I don’t think this postulate can work the other way around. The new market from my analysis is “Robotics” at large, not just Industrial Machine Vision of 2010’s, even though this is a significant and dynamic market just emerging from its glorious CCD past. DNN was judged unrealistic a decade ago, non Von Neumann architecture are needed for the next technology step and are actually very much developed right now, this is why I am relatively optimistic DVS can find its way. Let’s discuss 15 years from now, by the way 15 years ago Facebook did not exist. Pierre Cambou – Yole Developpement
ReplyDeletefunny discussion here... the takeaway message for Pierre Cambou and Prophesee marketing department: identify actual killer application(s) for DVS technology where they will demonstrate in action their unique advantage(s) over traditional approaches for machine vision: let’s meet in two years @ VISION 2020 and check progresses of DVS technology at Prophesee’s booth: I am sure they will come up with interesting real-time demonstrations to make their points, like most 3D imaging companies have greatly done this year at VISION 2018 exhibition.
ReplyDeleteI'm pretty sure that they will not come, but they will have a booth at Le Bourget, saying that DVS will dominate the LGM applications :)
Delete