Nikkei: Sony presented its new RGB-IR solution at Infrared Array Sensor Forum 2015 on July 31, 2015. Sony has replaced a half of G color in Bayer CFA with white (W). By using data of the three primary colors in addition to W to perform data processing, it becomes possible to remove only near-infrared light signals, Sony said. As a result, a color image can be taken by without an IR-cut filter. The new technology is said to enable existing image sensors to take an image using not only visible light but near-infrared light just by replacing a color filter and image processing software. The color filter is supplied by Fujifilm.
The
Japanese version of Nikkei article also has a picture explaining the new Sony technology:
The description reminds me Pixim patent US8619143 "
Image sensor including color and infrared pixels" by Ricardo Motta, now belonging to Sony. The Pixim patent uses Gray instead of White in CFA, but looks similar in other aspects:
RGBW does not work well. One would assume that using a white pixel would let you maintain resolution, but your SNR suffers due to crosstalk. Plus you never quite get the colors right because you really don't know how much IR is present.
ReplyDeleteAptina did a much better job using opaque pixels which provide a much better indication of how much IR is in the scene. That allows their algorithms to provide accurate colors.
@ "you never quite get the colors right because you really don't know how much IR is present."
DeleteActually, they can separate visible and IR. With an IR bandpass filter, like one shown on the Sony poster, you get:
R+IR in red pixel
G+IR in green pixel
B+IR in blue pixel
R+G+B+IR in white pixel
So, very roughly speaking, you get 4 equations with 4 variables, and can separate the colors. You can read Motta's patent for a full description.
I meant IR notch filter, not a bandpass.
DeleteWhat you want is a dual-bandpass filter with one band being visible wavelengths and the other notched around the IR wavelength you're using. I've lived through it at two companies who independently took the RGBW approach. They both had the same issues. The Aptina approach seems to work.
DeleteImaging you have a little bit of red, a little bit of green, a little bit of blue and a lot or IR. The IR generates a large signal in every pixel in combination with a lot of photon shot noise. Subtracting the IR from the red, green and blue pixel will result in a very noisy corrected signal. Is that what you want ?
ReplyDeleteSure, a security camera with mechanically movable IR cut filter is much better. But for the case of a cheap security camera where mechanical solution is too expensive, this might be an acceptable solution.
DeleteThe noise is one issue and another issue the accurate color reproduction from this RGB-IR combination. Yes, for cheap security cameras, this will be a good option. But even the mechanical IR cut-off filters are becoming lot cheaper now!!
DeleteThis situation is not so common since halogen light is less and less present, especially in Asia. During daylight, the IR is not that high and during night time, IR LEDs are very often used, but only B/W image is needed. So all-in-all, it's a good solution.
ReplyDelete-yang ni
Two conditions to have nice a COLOR image and NIR image
ReplyDelete- As said Albert, not too much NIR content in the image in respect to VIS (when there is very little VIS light then only the NIR image is of interest...)
- A low noise imager (as the image are numerically substracted, noise increases of ~SQRT(2))
This works as a charm on the Eliixa from e2v for a few years now
http://www.e2v-us.com/products/imaging/cameras/eliixa-uc4uc8/
Pierre
This is multi-line line scanning sensor, so the IR component can be measured at the exact position of RGB pixels. For RGBW pattern, the image processing is much more complicated since you can not get the IR component on the exact positions of RGB pixels.
Delete