Then Samsung presents its own Dynamic Vision Sensor (DVS), "the most promising device for gesture sensor." While there is no explanation how DVS works, Samsung says:
DVS can effectively detect a moving object in a cost-effective way with fast response by providing events asynchronously on its edges:
The DVS sensor is said to have a resolution of 128 x 128, 40um pixel pitch and consume 23mW of power. Since only the edges are transmitted to the output, rather than full frames, image processing can be simplified and power reduced by using edge-detected images:
The dynamic range of DVS extends to 120 dB, so that DVS can detect the moving object in extreme low light and ultra high light, including outdoors.
Thanks to DM for the link!
We have this function in our logarithmic sensors (till 1280x1024). The motion detection can be done inside the sensor at more than 100Hz in more than 120dB range. I'll upload an video on youtube ...
ReplyDelete-yang ni
Here is a link to youtube video:
ReplyDeletehttp://www.youtube.com/watch?v=FpdGx6psnUA
Please give me your comments ...
-yang ni
I think the DVS concept comes from Tobi Delbruck's lab some years ago. Perhaps Samsung took a license?
ReplyDeleteBTW,
Motion detection was shown by AT&T Bell Labs/JPL in this joint paper from 1995, similar but without the data reduction of the DVS. Yet, it gives smaller pixels.
A. Dickinson, B. Ackland, E-S. Eid, D. Inglis and E.R. Fossum, A 256x256 CMOS active pixel image sensor with motion detection, 1995 IEEE International Solid State Circuits Conference, Digest of Technical Papers, pp. 226-227, San Francisco CA, February 1995.
Examplary output is shown on page 5 of this presentation:
http://ericfossum.com/Presentations/Part%204%20-%20JPL%20Chips.pdf
Eric, the problem is that linear pixels have to be controlled in exposure time, so you have to read them out. This vanishes all the advantages. There are 2 kinds of implementations: 1) using logarithmic pixel (Tobi's and also our work ...) 2) using temproal ratio sensing used by Ruedi at CSEM. Our solution can give you both scene image and also differentiated results.
ReplyDeleteFor the sensor in your paper, how can you control and exposure time and how can you know that image is saturated or under exposed please ?
-yang ni
Well, generally our work looks more like global shutter devices and you can control the exposure with those just fine (but you need another "T") and also look at the absolute level of the two signals to optimize exposure.
DeleteNevertheless, Tobi's DVS and probably your work as well is much more sophisticated, some 15-20 years later so I expect it ought to give better performance! Having worked on smart sensors probably longer than almost anyone else who reads this blog (early 80's) I think it is hard to have a commercially viable product with low fill factors and I always try to think about separating the image capture function from the "smarts". Perhaps with 3D structures it will be more feasible but again, that is just separation in the Z-dimension.
The point of my post is that focal-plane "motion detection" is being rediscovered and most people forget about the earlier work in this area.
There is a dilemma concerning smart sensor: highly complex pixel => large pixel + low resolution => low resolution can be processed fastly by using DSP with much more flexibility. The second problem with smart sensor is that the image given the on-chip photoreceptor has small amplitude and high non-uniformity which result in bad processing quality and reliability.
DeleteBut Eric, you have forgotten that the most successful smart sensor has produced in billons per year even today == optical mouse sensor :)
You are right about optical mouse! A good example of a very focused application. Thanks!
DeleteIf this was truly an amazing technology Samsung would not be publishing papers but keeping it secret until a patent filing (or keeping it a trade secret). Sort of like how nobody knows what Apple is working on until it is announced or legal papers are filed. So I won't hold my breath that this will be in a Galaxy any time soon.
ReplyDelete