Autosens 2016 Conference kindly permitted me publishing few slides from ST presentation "True Flicker-Free HDR Solution for Pixel and Image Sensor" by Tarek Lule:
Aptina has a (similar) patent on this idea. I think that this is not treuly flicker free, the flicker is reduced since there is still "holes" in the exposure.
I seriously doubt this is a solution for the entire spectrum of LED sources, but most likely a single source. Unfortunately, bad LED and multi-source LED exists from signage, tail lights, headlights, lumenaires. "challenge to map 145 dB into the 8b of output devices" --Pinnacle Imaging Systems' LATM - locally adaptive tone mapping maps to 250 dB 16 -24 bit to 8 bit. Their demo video with a Fairchild HWK1910A https://www.youtube.com/watch?v=sHfR9johFoY
The goal of ST sensor is to deal with LED flicker from multi sources, HDR at each frame, full frame rate. This is not only slides and real demos have started.
Drivers will see on the display what their eyes see and Machine Vision algorithms will have all the information frame based. Can it reduce the time for system decision, I do think).
HDR images, videos, need good management of the dynamic (TM, AE...) that's why ST is proposing also HDR ISP. A former version of the HDR ISP rendering from ST can be found here: https://www.youtube.com/watch?v=jGmkUj1TGlg
Hi Anonymous! Would you mind to be more explicit in your criticism? Ignoring the conditions of the demo/presentation doesn't help in building fact-based arguments. The fact is that: a demo was shown with real-life video highlighting the independence of the system vs. the supplier source of the LEDs ("spectrum"). On this other hand, OK, Pinnacle did a great job in HDR and packing data on 8-bit output with the Fairchild component. But this hardly relates to the HDR _AND_ flicker-free combination... Waiting for further reading from you!
Does that mean they use one pixel for long integration, one for medium and one for the short integration? They can't have multiple photodiodes per pixel with such a pixel pitch, can they?
My understanding of those slides is that, you still need different types of pixel/photoreceptor to handle long exposure when you have high ambient light (like sunny day, no clouds ) and fragmented exposure modes ? Or do you just get rid of the information from the "Long Exposure" pixel when there is to much ambient light? I'd love to see color video from those sensors.
This feature well known since years and called Multi Integration. You can find it as "3d range gating mode" in the ev2 76c560 sensor or as "Multi exposure" in the new Sony Pregius sensors with the 3.45µm pixel. The integration is split into several short parts to catch a complete cycle of a flickering illumination. E.g. 100 times 0.1µs exposure gives you 10µs, perfect for sunlight and open iris. Problem is the multiple charge transfer and the accumulating "noise". Applications are brake light detection (200 Hz LED flicker) or visibility thought fog, rain and snow by using a synchronized pulsed illumination.
Hi Tarek and colleague ;-) ST flicker free sensor is HDR, Flicker Free, ASIL B on each frame at full Frame rate. In top of that, No ghost possible! Such sensors need good HDR ISP and it is part of our camera system approach! RGB demo is available and I'm sure few videos will be available on the web!
Simple explanation and Demo from the Conference: https://www.youtube.com/watch?v=27ZQIBJ-iOc&list=PLWa6uO3ZUweAZ-VXnnBsDDsBbz32BLlYf - Sub my Channel for Info and Demos from various Manufacturers of low and high cost Image Sensors and Cameras capable of recording Video with very little light.
The point was addressing the comment "challenge to map 145 dB into the 8b of output devices" by ST and showing there is a company providing this capability. What about the SNR dips? Also, re: the combination, there are other developers who have cracked this nut already in their circuitry beside ST and they are not Sony and e2v. Sony's rep at the recent Image Sensors Americas conference stated they were still in development of a pixel level solution, but did not have one yet.
Aptina has a (similar) patent on this idea. I think that this is not treuly flicker free, the flicker is reduced since there is still "holes" in the exposure.
ReplyDeletePatent number? ID of the component?
DeleteYes I believe this is the paper where they mention it.
Deletehttp://www.imagesensors.org/Past%20Workshops/2015%20Workshop/2015%20Papers/Sessions/Session_14/14-02_Silsby_041.pdf
I seriously doubt this is a solution for the entire spectrum of LED sources, but most likely a single source. Unfortunately, bad LED and multi-source LED exists from signage, tail lights, headlights, lumenaires. "challenge to map 145 dB into the 8b of output devices" --Pinnacle Imaging Systems' LATM - locally adaptive tone mapping maps to 250 dB 16 -24 bit to 8 bit. Their demo video with a Fairchild HWK1910A https://www.youtube.com/watch?v=sHfR9johFoY
ReplyDeleteThe goal of ST sensor is to deal with LED flicker from multi sources, HDR at each frame, full frame rate. This is not only slides and real demos have started.
DeleteDrivers will see on the display what their eyes see and Machine Vision algorithms will have all the information frame based. Can it reduce the time for system decision, I do think).
HDR images, videos, need good management of the dynamic (TM, AE...) that's why ST is proposing also HDR ISP. A former version of the HDR ISP rendering from ST can be found here: https://www.youtube.com/watch?v=jGmkUj1TGlg
Hi Anonymous!
DeleteWould you mind to be more explicit in your criticism? Ignoring the conditions of the demo/presentation doesn't help in building fact-based arguments. The fact is that: a demo was shown with real-life video highlighting the independence of the system vs. the supplier source of the LEDs ("spectrum"). On this other hand, OK, Pinnacle did a great job in HDR and packing data on 8-bit output with the Fairchild component. But this hardly relates to the HDR _AND_ flicker-free combination...
Waiting for further reading from you!
Does that mean they use one pixel for long integration, one for medium and one for the short integration? They can't have multiple photodiodes per pixel with such a pixel pitch, can they?
ReplyDeleteSecret revealed: their electronic designers have tricks... :)
DeletePlease check out the blog on www.harvestimaging.com/blog, where I gave some ideas about the pixel architecture.
DeleteThanks very much, Mr Theuwissen!
DeleteMy understanding of those slides is that, you still need different types of pixel/photoreceptor to handle long exposure when you have high ambient light (like sunny day, no clouds ) and fragmented exposure modes ? Or do you just get rid of the information from the "Long Exposure" pixel when there is to much ambient light?
ReplyDeleteI'd love to see color video from those sensors.
This feature well known since years and called Multi Integration. You can find it as "3d range gating mode" in the ev2 76c560 sensor or as "Multi exposure" in the new Sony Pregius sensors with the 3.45µm pixel.
ReplyDeleteThe integration is split into several short parts to catch a complete cycle of a flickering illumination. E.g. 100 times 0.1µs exposure gives you 10µs, perfect for sunlight and open iris. Problem is the multiple charge transfer and the accumulating "noise".
Applications are brake light detection (200 Hz LED flicker) or visibility thought fog, rain and snow by using a synchronized pulsed illumination.
Hi Tarek and colleague ;-)
ReplyDeleteST flicker free sensor is HDR, Flicker Free, ASIL B on each frame at full Frame rate. In top of that, No ghost possible!
Such sensors need good HDR ISP and it is part of our camera system approach!
RGB demo is available and I'm sure few videos will be available on the web!
Simple explanation and Demo from the Conference: https://www.youtube.com/watch?v=27ZQIBJ-iOc&list=PLWa6uO3ZUweAZ-VXnnBsDDsBbz32BLlYf - Sub my Channel for Info and Demos from various Manufacturers of low and high cost Image Sensors and Cameras capable of recording Video with very little light.
ReplyDeletePhillipe,
ReplyDeleteThe point was addressing the comment "challenge to map 145 dB into the 8b of output devices" by ST and showing there is a company providing this capability. What about the SNR dips? Also, re: the combination, there are other developers who have cracked this nut already in their circuitry beside ST and they are not Sony and e2v. Sony's rep at the recent Image Sensors Americas conference stated they were still in development of a pixel level solution, but did not have one yet.