Albert Theuwissen continues his excellent reports from ISSCC. The recent one talks about two different approaches to combined RGB/ToF sensors, both presented by Samsung.
AT says in the first paper (and I am a co-author, and primary inventor of this device), that the authors do not discuss the issue of NIR light getting into the RGB pixels. Not correct. The written paper discusses this directly using a combination of a shallow barrier layer in the silicon to block NIR signal absorbed below the barrier, and also to use ISP to remove residual NIR signal whose value is determined from the Z pixel signal. In simulation at least, this works very well, but for the paper, we had not yet put on CFA so it was not proven. In Q&A, the speaker, Mr. Kim, said on-chip pixel-wise NIR blocking and passing filters were being explored.
In the case of the other paper, they use a global mechanical filter wheel to create alternating fields of NIR-only light, and then RGB-only light. I believe they are exploring some E-O switchable filter device for the future. This is not really simultaneous capture since the NIR signal and Vis signals are captured in sequential fields. On the other hand, in almost TOF sensors, sequential fields are needed to capture the 4 phases. So "simultaneous" is a relative term.
There is some internal rivalry within these two different Samsung organizations, but in the absence of known external competition in this area of RGBZ image capture, the rivalry is good for both sides and the company in general as it propels us forward.
Hi Eric, in my comments I mentioned the blocking layer, but I can not remember that the presenter mentioned how well this layer is blocking the near-IR, neither how this blocking layer is realized. Neither I recall that he talked about the ISP, but I might be wrong. I checked the text in the proceedings, and there more information is indeed mentioned, including the ISP functionality, you are right ;-)
Due to paper-length constraints we could not talk about the ISP algorithm and effectiveness at removing residual NIR "fog" but it does work quite well at least so far in simulation. We also did not have room to talk about the new demosaicing algorithm to suppress artifacts due to periodic missing lines in the RGB image. This also works quite nicely, thanks to our ISP team. So, in a sense it is a report in progress. I hope we can write up a longer paper, perhaps for the forthcoming new IISS Journal on Image Sensors....
I heard from someone who saw the SAIT demo of their sensor (the 2nd Samsung paper) that the global mechanical filter wheel switched between NIR and VIS filters at about 1 Hz. I think it is very brave and admirable of SAIT to do a demo of emerging technology at ISSCC. Nevertheless, this particular architecture has a basic problem in that global switchable filters (at perhaps 120 Hz!) do not yet exist in any form suitable for consumer application. Maybe this will be rectified in the not too distant future.
There is a very mature technology for global switchable filters. It is called "ColorWheel" and has been used in DLP Projectors for almost 10 years now. Those filter-wheels separate RGB or more colors at up to 240Hz and basically almost any type of optical filter could be mounted.
I have to say that I enjoyed the first 2 papers of the session quite a bit - the Sony global shutter work, and the Tohoku Univ "terapixel/sec" high speed sensor. Both showed fine technical work and good presentations.
It is too bad that the half hour slots allowed only 5 min of Q&A it seems. It would be far better to have less formal presentation and more Q&A that we can all hear.
I, for one, wanted to know about the decrease in sensitivity of the Sony sensor due to the global storage site, and their plan for global shutter with BSI (much more challenging!).
All the talks in session 22 were good ones. Didn't have the same feeling about session 6 on Monday.
AT says in the first paper (and I am a co-author, and primary inventor of this device), that the authors do not discuss the issue of NIR light getting into the RGB pixels. Not correct. The written paper discusses this directly using a combination of a shallow barrier layer in the silicon to block NIR signal absorbed below the barrier, and also to use ISP to remove residual NIR signal whose value is determined from the Z pixel signal. In simulation at least, this works very well, but for the paper, we had not yet put on CFA so it was not proven. In Q&A, the speaker, Mr. Kim, said on-chip pixel-wise NIR blocking and passing filters were being explored.
ReplyDeleteIn the case of the other paper, they use a global mechanical filter wheel to create alternating fields of NIR-only light, and then RGB-only light. I believe they are exploring some E-O switchable filter device for the future. This is not really simultaneous capture since the NIR signal and Vis signals are captured in sequential fields. On the other hand, in almost TOF sensors, sequential fields are needed to capture the 4 phases. So "simultaneous" is a relative term.
There is some internal rivalry within these two different Samsung organizations, but in the absence of known external competition in this area of RGBZ image capture, the rivalry is good for both sides and the company in general as it propels us forward.
Hi Eric, in my comments I mentioned the blocking layer, but I can not remember that the presenter mentioned how well this layer is blocking the near-IR, neither how this blocking layer is realized. Neither I recall that he talked about the ISP, but I might be wrong. I checked the text in the proceedings, and there more information is indeed mentioned, including the ISP functionality, you are right ;-)
DeleteDue to paper-length constraints we could not talk about the ISP algorithm and effectiveness at removing residual NIR "fog" but it does work quite well at least so far in simulation. We also did not have room to talk about the new demosaicing algorithm to suppress artifacts due to periodic missing lines in the RGB image. This also works quite nicely, thanks to our ISP team. So, in a sense it is a report in progress. I hope we can write up a longer paper, perhaps for the forthcoming new IISS Journal on Image Sensors....
DeleteI heard from someone who saw the SAIT demo of their sensor (the 2nd Samsung paper) that the global mechanical filter wheel switched between NIR and VIS filters at about 1 Hz. I think it is very brave and admirable of SAIT to do a demo of emerging technology at ISSCC. Nevertheless, this particular architecture has a basic problem in that global switchable filters (at perhaps 120 Hz!) do not yet exist in any form suitable for consumer application. Maybe this will be rectified in the not too distant future.
DeleteThere is a very mature technology for global switchable filters. It is called "ColorWheel" and has been used in DLP Projectors for almost 10 years now. Those filter-wheels separate RGB or more colors at up to 240Hz and basically almost any type of optical filter could be mounted.
DeleteRight. Think, cell phone.
DeleteI have to say that I enjoyed the first 2 papers of the session quite a bit - the Sony global shutter work, and the Tohoku Univ "terapixel/sec" high speed sensor. Both showed fine technical work and good presentations.
ReplyDeleteIt is too bad that the half hour slots allowed only 5 min of Q&A it seems. It would be far better to have less formal presentation and more Q&A that we can all hear.
I, for one, wanted to know about the decrease in sensitivity of the Sony sensor due to the global storage site, and their plan for global shutter with BSI (much more challenging!).
All the talks in session 22 were good ones. Didn't have the same feeling about session 6 on Monday.