Image Sensors and AI Processors

PRNewswire: Xailient announces face recognition AI for Sony's intelligent vision sensor IMX500 with 97.8% accuracy up to 3 meters distance. On the IMX500, Xailient claims to provide the world's most power-efficient Face Recognition AI.

Shivy Yohanandan, Chief Scientist and co-founder of Xailient, says, "The IMX500 is a game-changer for the smart camera market. Delivering the image sensor and AI processing on a single chip has dramatic power efficiency benefits, which is important to extending battery life."

Ray Edwards, Xailient's VP of Market Development, says, "This should sell like hotcakes! Anyone building a smart camera system would be nuts not to consider the IMX500. This is the new standard."

SynSense and Prophesee announced a partnership. The two companies leverage their respective expertise in sensing and processing to develop ultra-low-power solutions for implementing intelligence on the edge for event-based vision applications.

The partnership combines in one single chip SynSense’s low-power vision SNN processor DYNAP-CNN with Prophesee’s event-based Metavision sensors, and is focused on developing a line of cost-efficient modules that can be manufactured at high volume.

“As IoT applications in smart homes, offices, industries and cities proliferate, there is an increasing need for more intelligence on the edge. Our experience in implementing neuromorphic enabled vision sensing with smart pixel methods complements SynSense’s expertise in neuromorphic processing to make implementing high performance vision more practical. Together we can drive the development of highly efficient, lower cost products that provide more visibility, safety and productivity in a wide range of use cases.” said Luca Verre, CEO and co-founder of Prophesee.

Image Sensors at ISSCC 2022

 ISSCC publishes its 2022 Agenda. There are 13 image sensor-related papers:

1. Charge-Domain Signal Compression in Ultra-High-Speed CMOS Image Sensors
   Keiichiro Kagawa,
   Shizuoka University, Hamamatsu, Japan
2. A 0.37W 143dB-Dynamic-Range 1Mpixel Backside-Illuminated Charge-Focusing SPAD Image Sensor with Pixel-Wise Exposure Control and Adaptive Clocked Recharging
   Y. Ota, K. Morimoto, T. Sasago, M. Shinohara, Y. Kuroda, W. Endo, Y. Maehashi, S. Maekawa, H. Tsuchiya, A. Abdelghafar, S. Hikosaka, M. Motoyama, K. Tojima, K. Uehira, J. Iwata, F. Inui, Y. Matsuno, K. Sakurai, T. Ichikawa,
   Canon, Kanagawa, Japan
3. A 64×64-Pixel Flash LiDAR SPAD Imager with Distributed Pixel-to-Pixel Correlation for Background Rejection, Tunable Automatic Pixel Sensitivity and First-Last Event Detection Strategies for Space Applications
   E. Manuzzato, A. Tontini, A. Seljak, M. Perenzoni
   Fondazione Bruno Kessler, Trento, Italy; Jozef Stefan Institute, Ljubljana, Slovenia
4. An 80×60 Flash LiDAR Sensor with In-Pixel Histogramming TDC Based on Quaternary Search and Time-Gated Δ-Intensity Phase Detection for 45m Detectable Range and Background Light Cancellation
   S. Park, B. Kim, J. Cho, J-H. Chun, J. Choi, S-J. Kim
   Ulsan National Institute of Science and Technology, Ulsan, Korea; SolidVue, Suwon, Korea, Sungkyunkwan University, Suwon, Korea
5. A 38μm Range Precision Time-of-Flight CMOS Range Line Imager with Gating Driver Jitter Reduction Using Charge-Injection Pseudo Photocurrent Reference
   K. Yasutomi, T. Furuhashi, K. Sagawa, T. Takasawa, K. Kagawa, S. Kawahito
   Shizuoka University, Hamamatsu, Japan
6. A 1/1.57-inch 50Mpixel CMOS Image Sensor with 1.0μm All-Directional Dual Pixel by 0.5μm-Pitch Full-Depth Deep-Trench Isolation Technology
   T. Jung, M. Fujita, J. Cho, K. Lee, D. Seol, S. An, C. Lee, Y. Jeong, M. Jung, S. Park, S. Baek, S. Jung, S. Lee, J. Yun, E. S. Shim, H. Han, E. Park, H. Sul, S. Kang, K. Lee, J. Ahn, D. Chang
   Samsung Electronics, Hwasung, Korea
7. A 4.9Mpixel Programmable-Resolution Multi-Purpose CMOS Image Sensor for Computer Vision
   H. Murakami, E. Bohannon, J. Childs, G. Gui, E. Moule, K. Hanzawa, T. Koda, C. Takano, T. Shimizu, Y. Takizawa, A. Basavalingappa, R. Childs, C. Cziesler, R. Jarnot, K. Nishimura, S. Rogerson, Y. Nitta,
   Sony
8. A Fully Digital Time-Mode CMOS Image Sensor with 22.9pJ/frame∙pixel and 92dB Dynamic Range
   S. Kim, T. Kim, K. Seo, G. Han,
   Yonsei University, Seoul, Korea
9. A 64Mpixel CMOS Image Sensor with 0.56μm Unit Pixels Separated by Front Deep-Trench Isolation
   S. Park, C. Lee, S. Park, H. Park, T. Lee, D. Park, M. Heo, I. Park, H. Yeo, Y. Lee, J. Lee, B. Lee, D-C. Lee, J. Kim, B. Kim, J. Pyo, S. Quan, S. You, I. Ro, S. Choi, S-I. Kim, I-S. Joe, J. Park, C-H. Koo, J-H. Kim, C. K. Chang, T. Kim, J. Kim, J. Lee, H. Kim, C-R. Moon, H-S. Kim,
   Samsung Electronics, Hwaseong, Korea
10. A 200 x 256 Image Sensor Heterogeneously Integrating a 2D Nanomaterial-Based Photo-FET Array and CMOS Time-to-Digital Converters
    H. Hinton, H. Jang, W. Wu, M-H. Lee, M. Seol, H-J. Shin, S. Park, D. Ham
    Harvard University, Cambridge, MA; Samsung Advanced Institute of Technology, Suwon, Korea
11. A 0.8V Intelligent Vision Sensor with Tiny Convolutional Neural Network and Programmable Weights Using Mixed-Mode Processing-in-Sensor Technique for Image Classification
    T-H. Hsu, G-C. Chen, Y-R. Chen, C-C. Lo, R-S. Liu, M-F. Chang, K-T. Tang, C-C. Hsieh
    National Tsing Hua University, Hsinchu, Taiwan
12. Augmented Reality – The Next Frontier of Image Sensors and Compute Systems
    C. Liu, S. Chen, T-H. Tsai, B. De Salvo, J. Gomez
    Meta Reality Labs, Redmond, WA
13. Concepts, Architectures and Circuits for Sub-THz Sensing and Imaging
    A. Stelzer,
    Linz University, Linz, Austria

EDoF-ToF Paper

Rice University publishes an OSA Optica paper "EDoF-ToF: extended depth of field time-of-flight imaging" by Jasper Tan, Vivek Boominathan, Richard Baraniuk, and Ashok Veeraraghavan.

"Conventional continuous-wave amplitude-modulated time-of-flight (CWAM ToF) cameras suffer from a fundamental trade-off between light throughput and depth of field (DoF): a larger lens aperture allows more light collection but suffers from significantly lower DoF. However, both high light throughput, which increases signal-to-noise ratio, and a wide DoF, which enlarges the system’s applicable depth range, are valuable for CWAM ToF applications. In this work, we propose EDoF-ToF, an algorithmic method to extend the DoF of large-aperture CWAM ToF cameras by using a neural network to deblur objects outside of the lens’s narrow focal region and thus produce an all-in-focus measurement. A key component of our work is the proposed large-aperture ToF training data simulator, which models the depth-dependent blurs and partial occlusions caused by such apertures. Contrary to conventional image deblurring where the blur model is typically linear, ToF depth maps are nonlinear functions of scene intensities, resulting in a nonlinear blur model that we also derive for our simulator. Unlike extended DoF for conventional photography where depth information needs to be encoded (or made depth-invariant) using additional hardware (phase masks, focal sweeping, etc.), ToF sensor measurements naturally encode depth information, allowing a completely software solution to extended DoF. We experimentally demonstrate EDoF-ToF increasing the DoF of a conventional ToF system by 3.6 ×, effectively achieving the DoF of a smaller lens aperture that allows 22.1 × less light. Ultimately, EDoF-ToF enables CWAM ToF cameras to enjoy the benefits of both high light throughput and a wide DoF."

Sony - Qualcomm Joint Lab to Work on Image Sensor and Processing Optimizations

During the recent Qualcomm's mobile processor announcements, there was a part on establishing a joint lab with Sony in San Diego working on image sensor and processor co-optimization:

BAE Presents 12MP 240fps APS-C Sensor

BAE (Fairchild Imaging) presents 12MP 240fps APS-C sensor LTN4625A with global and rolling shutter modes:

Advantest Speeds Up its CIS Production Tester

GlobeNewswire: Advantest has begun shipping the fourth generation of its high-speed image-processing engine that applies heterogeneous computing technology to detect defects in the data output from CMOS image sensors. When integrated on the proven T2000 ISS platform, the new T2000 IP Engine 4 (Image Processing Engine 4) system provides the means of evaluating the latest high-resolution, high-speed CIS devices used in advanced smart phone cameras.

The new T2000 IP Engine 4 features enhanced computing power to handle huge volumes of imaging data while also reducing test times and the cost of test. Used along with Advantest’s 4.8GICAP image capture module, the new tester can perform high-volume, at-speed testing of the most advanced mobile CIS including 3.5-Gsps C-PHY and 4.8-Gbps D-PHY devices. Image-processing accelerators enable fast testing of high-resolution CIS with more than 200MP.

Qualcomm Snapdragon 8 Gen 1 Camera Features

Qualcomm announces its next generation mobile processor Snapdragon 8 Gen 1:

"Snapdragon Sight Technology includes the first commercial 18-bit mobile ISP, capturing over 4000x more camera data than its predecessor for extreme dynamic range, color, and sharpness at staggering speeds up to 3.2 gigapixels per second. This is also the first 8K HDR video capture in a mobile platform and it’s capable of capturing in the premium HDR10+ format that’s loaded with over a billion shades of color. Video will look even more stunning thanks to the new Bokeh Engine which adds beautiful soft backgrounds to videos; it’s like Portrait Mode for video capture. Snapdragon 8 also includes a fourth separate ISP, the new Always-On ISP, which allows the camera to run with extremely low power consumption so users can experience always-on face unlocking and locking if your face isn’t present for heightened privacy."

Tyrafos Raises "Hundreds of Millions of RMB" in Round A+

iNews, Lieyunwang, iFeng: Guangzhou Tyrafos (Chinese name Guangzhou Yinxin Semiconductor Technology Co., Ltd.) announces the A+ round of financing led by Yunqi Capital. By the end of this round of financing, Yinxin Semiconductor is expected to receive hundreds of millions of yuan in the near future.

Tyrafos also announces announced the release of "the world's first non-single photon dToF" (possibly, they mean dToF that is not based on SPADs) and ELISA molecular biological detection solution basedon image sensors.

"The world's first non-single-photon dToF released by Yinxin Semiconductor is the world's first technical solution to achieve direct ToF using CMOS process. Not only the accuracy and resolution are several times higher than the existing SPAD (Single Photon Avalanche Diode) solution. In addition, the overall power consumption of the laser light source at the transmitting end and the sensor at the receiving end has been reduced by 90%, catching up with the core technology of foreign manufacturers such as Apple and STMicroelectronics in this field, and the underlying architecture and circuit IP are independently developed by Yinxin Semiconductor, it solves the technical problem of the neck stuck in the field of 3D perception, and is a milestone breakthrough in the field of 3D perception."

"In the field of fingerprints under the optical screen, Yinxin Semiconductor adopts the innovative design of pixel architecture level to break through the area and cost limits. The chip area is 30%~50% smaller than that of competing products, and the module volume is more than half smaller than that of competing products. It has an ultra-high cost performance. Competitive advantage, in 2019, through cooperation with customers, it became the first optical under-screen fingerprint recognition solution certified by South Korea's Samsung mobile phone. At present, Yinxin Semiconductor has reached an agreement with 4-5 global leading fingerprint solution providers to purchase under-screen fingerprint chips. Intentional orders are expected to contribute hundreds of millions of yuan in revenue."

"The world's first digital ELISA molecular biological detection solution released by Yinxin Semiconductor is the world's first digital solution that directly performs molecular biological detection on the surface of an image sensor. Enzyme Linked Immunosorbent Assay (Enzyme Linked Immunosorbent Assay, abbreviated ELISA or ELASA) refers to a qualitative and quantitative detection method that combines soluble antigens or antibodies on solid-phase carriers such as polystyrene, and uses antigen-antibody specific binding for immunoreaction. . Taking advantage of the company's advantages in high-speed and high-sensitivity image sensors, Yinxin Semiconductor has created a complete set of micro-molecular biological detection equipment based on ELISA standards, achieving a zero breakthrough in this field, and surpassing the United States in detection accuracy and speed. NASDAQ listed company Quanterix (the core technology comes from the Harvard University technical team, the technology founder David Walt is an academician of the American Academy of Engineering and the Medical School, and is also the scientific founder of Illumina) SiMoA (Single-molecule Array) system, and Yinxin Semiconductor In terms of equipment cost and consumables cost, the digital ELISA technology of the company has a significant improvement over Quanterix's SiMoA system technology. The sensitivity of the digital ELISA technology is more than 1000 times higher than that of the traditional ELISA. Its appearance brings protein detection technology directly into the era of single-molecule and digital detection, becoming the true king in the field of fg-level ultra-low abundance protein detection. The optimization of detection accuracy, speed and cost is more conducive to application promotion. In the future, it will have great application potential in the fields of coronavirus detection, influenza virus detection, and early cancer screening."

Tyrafos has been established in May 2019 and its core team coming from TI, Omnivision, Samsung, Himax, TSMC, Mediatek, ASE, Foxconn, and Chi Mei. The company has >90 full-time R&D and operations employees. The core members of the team have 15-20 years experience in the semiconductor industry. The company names Sony and Samsung as its strategic partners, and TSMC, Samsung, and Tower as its foundries.

Sony Automotive Sensor Explained

 Lucid Vision Labs publish an explanatory video about Sony IMX490 5.4MP automotive sensor:

Lucid Vision Labs also publishes a measured EMVA1288 performance of the sensor:

SWIR Quantum Dot Triodes

Nature paper "Silicon: quantum dot photovoltage triodes" by Wen Zhou, Li Zheng, Zhijun Ning, Xinhong Cheng, Fang Wang, Kaimin Xu, Rui Xu, Zhongyu Liu, Man Luo, Weida Hu, Huijun Guo, Wenjia Zhou & Yuehui Yu from Chinese Academy of Sciences and ShanghaiTech University presents a concept of 1550nm photodetector with internal gain:

"Silicon is widespread in modern electronics, but its electronic bandgap prevents the detection of infrared radiation at wavelengths above 1,100 nanometers, which limits its applications in multiple fields such as night vision, health monitoring and space navigation systems. It is therefore of interest to integrate silicon with infrared-sensitive materials to broaden its detection wavelength. Here we demonstrate a photovoltage triode that can use silicon as the emitter but is also sensitive to infrared spectra owing to the heterointegrated quantum dot light absorber. The photovoltage generated at the quantum dot base region, attracting holes from silicon, leads to high responsivity (exceeding 410 A·W−1 with Vbias of −1.5 V), and a widely self-tunable spectral response. Our device has the maximal specific detectivity (4.73 × 1013 Jones with Vbias of −0.4 V) at 1,550 nm among the infrared sensitized silicon detectors, which opens a new path towards infrared and visible imaging in one chip with silicon technology compatibility."

SOL Presents its Lens-Free Sensor Solutions

SOL Inc. presents its IEC-standardized lensless biosensor solutions: