Sony stacked CIS+iToF sensor (IEDM 2024)

Article (in German): https://www.pcgameshardware.de/Wissenschaft-Thema-237118/News/Fuer-Kameras-Sony-stapelt-Farb-Tiefensensor-keine-Verzerrungen-mehr-1462040/

English translation from Google Translate (with some light editing) below:

 

Depth sensors, which provide an image with spatial information, have become increasingly widespread in recent years. They can be used, for example, to create 3D scans or for targeted, subsequent blur effects - for example in smartphone cameras. In most cases, so-called ToF sensors (Time of Flight) are used, in which each pixel is measured when previously radiated infrared light is reflected back.

Not next to each other, but on top of each
So far, however, there has been a problem in the implementation in combination with normal camera sensors. Either the ToF sensor is located next to the camera sensor. Then there are are concealed areas through the different angles, but above all on edges, and not every color value can be assigned a depth value. Or ToF and color pixels sit on the same sensor and take away the space from each other. In other words: The resolution is reduced.

However, the camera division of Sony now claims to have found a way out. At the IEDM 2024 semiconductor trade fair, a combination sensor was presented in which the camera sensor is located directly above the depth sensor. This is made possible by the use of a new material: normally the color pixels would be located on silicon, but the broadband light would be absorbed and thus the depth pixels covered. However, Sony has apparently solved this problem by means of a new construction on a broadband transparent, organic photo-leading film. Visible wavelength hits the color sensors, while infrared light falls further down on the IR pixels of the ToF sensor.

Above each ToF pixel, which each occupies 4um, there are four RGB pixels with 1um each. In total, there is talk of a resolution of 1004 x 756 pixels for the depth map and 4016 x 3024 pixels for the color image. At least in this respect, the prototype has apparently already reached a usable area.

 

However, it is still unclear whether and when corresponding sensors should go into mass production. However, if Sony can potentially eliminate existing problems, the wide availability of such a sensor would offer numerous options. For example, you could simplify the creation of high-resolution 3D scans for games and movies and also make the data collection of robots significantly more reliable.