Color Photon Counting Sensor

Arxiv.org paper "Megapixel Photon-Counting Color Imaging using Quanta Image Sensor" by Abhiram Gnanasambandam, Omar Elgendy, Jiaju Ma, and Stanley H. Chan from Purdue University and Gigajot presents first color photon counting sensor:

"Quanta Image Sensor (QIS) is a single-photon detector designed for extremely low light imaging conditions. Majority of the existing QIS prototypes are monochrome based on single-photon avalanche diodes (SPAD). Color imaging has not been demonstrated with single-photon detectors due to the intrinsic difficulty of shrinking the pixel size and increasing the spatial resolution while maintaining acceptable intra-pixel cross-talk. In this paper, we present image reconstruction of the first color QIS with a resolution of 1024×1024 pixels, supporting both single-bit and multi-bit photon counting capability. Our color image reconstruction is enabled by a customized joint demosaicing-denoising algorithm, leveraging truncated Poisson statistics and variance stabilizing transforms. Experimental results of the new sensor and algorithm demonstrate superior color imaging performance for very low-light conditions with a mean exposure of as low as a few photons per pixel."

Mitsubishi Develops MWIR and LWIR Graphene Sensors

Nikkei: Mitsubishi claims it has developed a MWIR sensor having the world's highest responsivity among graphene sensors. Due to an internal graphene FET gain, the responsivity is 10x higher than that of quantum-type IR sensors with no internal amplification. Mitsubishi uses graphene FET and leverages its high electron mobility.

Other than a graphene-based FET, there is "a light-amplifying part" that produces photoelectrons and photoholes and is placed under the graphene. At a very low temperature of, for example, 80K, the responsivity increases even more, by a factor of 100x:

Mitsubishi is also developing a LWIR sensor using a dielectric for the light-amplifying. Existing quantum-type LWIR sensors reduce thermal noise by cooling down to liquid nitrogen temperature. Mitsubishi claims that with its new method, the sensor might be able to work at room temperature:

The original article in Japanese has few more figures:

Here is an earlier Nikkei article on the same topic. Mitsubishi is going to present its work at IDTechEx Show in Berlin, Germany on April 10-11, 2019.

Terahertz Imaging Pixel

Japanese Journal of Applied Physics publishes a paper "Wideband terahertz imaging pixel with a small on-chip antenna in 180 nm CMOS" by Yuri Kanazawa, Sayuri Yokoyama, Shota Hiramatsu, Eiichi Sano, Takahiro Ikegami, Yuma Takida, Prasoon Ambalathankandy, Hiroaki Minamide, and Masayuki Ikebe from Hokkaido University, Sony, and RIKEN. The new paper appears to be a further development of their previous paper presented at ISW 2017.

"We propose a Si-CMOS terahertz image sensor to resolve paucity of low-cost and small-size detectors. The imaging pixel consists of an on-chip antenna and an amplifier acting as envelope detector. The pixel uses a microstrip patch antenna for receiving THz waves. However, the antenna's narrow bandwidth and large ground plane size causes major problems. A low-resistivity Si substrate degrades the gain of planar antennas besides the microstrip patch antenna. We introduce an on-chip folded-slot antenna to reduce the pixel size and prevent gain degradation due to the Si substrate. The antenna has a broader bandwidth and higher gain than conventional on-chip slot antenna. We fabricated the patch/folded-slot types of the pixel circuits by using 0.18 μm CMOS process. Measurement results show that the folded-slot antenna has a broader bandwidth from 850 GHz to 1.05 THz."

Venturebeat: In an unrelated news, FCC has started issuing 10-year licenses to experiment in 95GHz to 3THz spectrum range for the companies that are trying to find "the next big thing."

Yantai Raytron Prepares IPO

Reuters: Yantai Raytron Technology Co, a Chinese developer of LWIR imaging and sensors technology is one of the first companies preparing the IPO at Shanghai’s Nasdaq-style Science and Technology board, announced by President Xi Jinping in November 2018. The board’s registration-based listing procedure is supposed to take 3 months and allow start-ups that have yet to turn a profit to list.

Yantai Raytron is developing a low-cost thermal imagers based on microbolometric technology.

Light Co. CEO Keynote at MWC

Light Co. publishes its CEO Dave Grannan talk at MWC 2019 Keynote Session "The Next Unicorn".

IHS Markit on ToF Market

Semiconductor Engineering publishes IHS Markit article on ToF sensing market:

"IHS Markit forecasts that global revenue for ToF sensors in the 3D optical sensing market will surpass $500 million in 2019, up from $370 million last year. We also predict that the ToF market will grow in the coming years, spurred by combo solutions integrated with other light sensors in the same package. This will lead to a cheaper bill of materials (BOM) compared to the BOM for the SL method.

At the same time, IHS Markit forecasts that the total market potential for light sensors will be worth much more, reaching $1.5 billion by year 2022. That’s because after a solid start with gaming consoles, 3D sensing has matured and consolidated in the massive smartphone arena.

A segment of 3D sensing’s future growth will come from other use cases and applications that are emerging outside consumer electronics and mobile. These include people-counting and -tracking in consumer and industrial applications, landing-aid and obstacle-avoidance functions in drones, and car-trunk (boot) opening with foot gestures, as well as gesture recognition and passenger detection in automotive.

All told, the ToF approach appears to have a greater chance than the SL method in gaining a larger market share, leading to a cheaper and smaller BOM along with reduced integration costs in system assembly and calibration."

TU Delft Lensless Imaging

TU Delft publishes a poster on its approach to lensless imager:

TuSimple Uses Sony Sensor in its Night Driving Camera

PRNewswire: TuSimple announces an automotive-grade camera and vision system that will go into volume production in Q2 2019 and will be deployed on its autonomous customer truck fleet by Q3 2019. With $95m in new funding and daily autonomous trips hauling freight, the company is on its way to achieving its milestone of first driverless operations by the end of 2020.

Featuring Sony automotive CMOS sensor, the custom designed camera system allows for reliable night and low light autonomous operation. Adding nighttime operation to TuSimple's already industry leading 1,000 meter perception system will allow fleets to increase truck utilization from an average of 50%, or 12 hours per day, to an average of over 80%. The system addresses complex imaging challenges such as instant light changes when entering and exiting tunnels, flaring during sunrise and sunset, and headlight glare. It also handles the LED flicker created by digital signage and traffic lights.

"Bringing reliable nighttime and low light operation to current perception systems in the market is an important step in the viability of autonomous driving, which is a strategic focus for us," said Tsutomu Haruta, Automotive Business Division Deputy SGM, Sony Semiconductor.

"Like human drivers, autonomous trucks' perception systems are challenged by a wide variety of light conditions which are experienced every day while driving. We weren't able to find a camera system on the market that fit our needs so we created one," said Xiaodi Hou, Founder, President and CTO, TuSimple. "The combined expertise of Sony Semiconductor Solutions Corporation and TuSimple has created a perception system that sees better than the human eye – night and day, rain or shine — in the most challenging driving conditions."

ON Semi Announces Real-Time CIS Model for Automotive Simulations

BusinessWire: ON Semi announces that it provides its sophisticated real-time image sensor modeling technology to the NVIDIA DRIVE Constellation simulation platform. The open, cloud-based platform performs bit-accurate simulation for large-scale, hardware-in-the-loop testing and validation of autonomous vehicles.

ON Semi’s Image Sensor model receives both scene information and control signals from DRIVE Constellation to calculate and output a real-time image based on the inputs. It then transmits the simulated image back to DRIVE Constellation for processing. The complex sensor model will utilize all critical parameters in the path from converting photons to digital output (e.g. QE, Noise, Gain, ADC, BLC, and more) to provide an accurate output of a real-world image sensor.

“ON Semiconductor’s advanced imaging solutions have demonstrated class-leading performance and their innovative Image Sensor model can benefit those engaged in the groundbreaking work to take autonomous driving from concepts under test to mainstream reality,” said Zvi Greenstein, GM at NVIDIA.

Sony Announces Stacked BSI 2.74um Global Shutter Pixel Technology

Sony announces Pregius S, a stacked CMOS sensor technology that employs a proprietary global shutter function with BSI pixel to combine distortion-free, high imaging performance and miniaturization. The new sensor technology is intended for industrial applications.

Conventional CMOS image sensors equipped with global shutter function temporarily store charge signals in the memory area located next to the photodiode to resolve image distortion (focal plane distortion) caused by the time shift due to the row-by-row readout. In FSI CMOS sensors, there is a wiring layer on the silicon substrate forming the photodiode, and with such a structure, the benefit is that it is easy to form a light shield for protecting the charge signal temporarily stored in the memory area from leaked light. For this reason, conventional CMOS image sensors with global shutter function have adopted a FSI pixel structure. However, the wiring on top of the photodiode hinders the incident light, which creates an issue when attempting to miniaturize the pixels.

In response to this, Sony has developed a proprietary pixel structure that achieves the global shutter function on a BSI structure that has superior sensitivity characteristics. Normally, when pixels are miniaturized, the sensitivity and saturation characteristics deteriorate, but the new Sony technology enables a reduction in pixel size to 2.74 μm while maintaining performance of those characteristic, thereby achieving about 1.7 times higher resolution than conventional FSI CMOS sensors. Moreover, thanks to the high degree of freedom of the wiring layout of BSI pixel structures, a high speed of about 2.4 times that of conventional sensor can be achieved. In addition, the sensor’s stacked structure makes it possible to mount various signal processing circuits, whereby it is possible to realize smart functions such as signal processing only for the necessary part of the measurement and inspection images in a smaller size compared to conventional sensors.

Going forward, Sony will work to develop products equipped with this stacked CMOS sensor employing its proprietary global shutter function with BSI pixel structure for various industrial applications and intelligent transportation systems, including development of derivatives for signal processing circuits to be mounted. Sony plans to start shipping sample units in the summer of 2019 or later.