Fraunhofer Digital Cinema NAB 2013 presentation included "HDR Video with Single Shot Non-Regular Sampling". Fraunhofer IIS scientists placed a special non-regular neutral density filter mask with different attenuations in front of the camera sensor. This makes it possible to capture dark and bright image parts at the same time, where some HDR methods need to take two or more images. The non-regular alignment of the attenuations is said to allow a near lossless reconstruction of the missing pixels in the corresponding dark or bright image. After the reconstruction, the images can be fused together to an HDR image with no motion artifacts:
You probably get 2 reds, 2 blues and 4 greens when using a Bayer pattern. Any other catch?
ReplyDeleteBut it still is a very nice idea.
This idea has been patented by SONY years ago. They called it SVE=Spatial Variable Exposure.
ReplyDeleteNope. With a simple SVE sensor (used by Sony, Fuji etc), the sensor has a regular pattern of pixels with two different sensitivities. This way, the resolution is half the resolution without HDR.
DeleteThe Fraunhofer approach uses a non-regular pattern in the intensity mask. The assumption is that the image signal can be represented sparsely, so that even with less pixels for each of the two sensitivities, (almost) a full resolution image can be reconstructed. Using a non-regular pattern is important.
This approach is similar -- but not identical -- to compressive sensing schemes that have also been mentioned here.
Most recent paper about the prototype:
http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=1568617
From theoretical point of view, sub-sampling will generate information loss. The only difference is that the different way to sub-sampling will generate different artefacts and degradations. Saying that with random sampling there is no loss of information is not scientifically exact.
DeleteI would like to see a colour version of this application. I think that pixels having the same colour with different ND filters on top will generate different cross-talk signals in neighbouring pixels. How can you solve this problem with a classical colour correction matrix ?
ReplyDeleteWe have seen a similar problem when using a (semi-)random filter pattern. Nevertheless interesting idea !
Cross-talk that occurs after color filtering, both optical and electrical, can be eliminated by inverse filtering the mosaicked image prior to demosaicking. Cross-talk that occurs before color filtering does not interfere with the demosaicking process as it can be regarded as part of the optical pipeline. Post demosaicking Sharpening/Deconvolution/Inverse Filtering can compensate for it along with the effects of the OLPF, lens etc.
ReplyDeleteWe have found random CFAs to work quite well, see some of our older results at http://imagealgorithmics.com/RGB_CFAs_Tested.html#RandomRGB