Albert Theuwissen Posts His IISW 2017 Papers

Albert Theuwissen makes his IISW 2017 papers avaliable for download.

216. Xiaoliang Ge, Albert Theuwissen : A 0.5e- Temporal Noise CMOS Image Sensor with Charge-Domain CDS and Period-Controlled Variable Conversion Gain.

217. Fei Wang, Liqiang Han, Albert Theuwissen : A Highly Linear CMOS Image Sensor with a Digitally Assisted Linearity Calibration Method.

Sony Fast Readout May Create Banding Artifacts in LED Light

DPReview publishes an analysis of banding artifacts in Sony full-frame A9 camera with electronic rolling shutter. Apparently, its stacked stacked sensor reads 12 rows in parallel to progress through the entire 24MP sensor in 6.25ms time. This creates noticeable banding artifacts on LED advertisement panels and nearby objects in the examples shot at a soccer game:

DPReview concludes: "The a9's electronic shutter proceeds in 12-row chunks; a mechanical shutter is more analogous to an electronic shutter proceeding line by line, which would yield smoother bands. Not the more hard-edged, lower frequency (and therefore more readily identifiable) 12-row bands that are visually more distracting than if they had been gradual, single-pixel rows transitioning from one color or brightness to another."

ADAS Camera Market

ResearchInChina publishes "Global and China Automotive Camera ADAS Industry Report, 2016-2021:"

"Broadly-defined camera ADAS includes 360-degree surround view system and park assist system. In 2016, broadly-defined camera ADAS shipments totaled roughly 40.1 million sets, of which, park assist (reversing camera) makes up the highest 34% or so, followed by 360-degree surround view system (about 20%) and FCW (about 13.7%). It is expected that by 2021 camera ADAS shipments will reach 71.4 million sets, of which park assist, 360-degree surround view system and AEB will capture 30.5%, 21.2% and 19.5%, separately. Narrowly-defined camera ADAS (excluding 360-degree surround view system and park assist system) shipments totaled approximately 15.2 million sets. By shipments, Continental Automotive, Gentex, Magna, ZF TRW, NIDEC SANKYO, Valeo, Sony and Clarion are successively ranked. In terms of camera ADAS processor, Mobileye, TI, Xilinx, Ambraella, Renesas, Toshiba, Hitachi, ADI and NXP are successively ranked by shipments.

The global camera ADAS market was worth USD2,667 million in 2016, an increase of 21.3%, and is expected to rise 24.4% to USD3,320 million in 2017. With the advent of L3/L4 self-driving in 2020, the deeply-learned embedded system will bring another marketing leap to an estimated USD7,760 million in 2021. Automotive camera module shipments totaled approximately 66.5 million sets in 2016 and are expected to rise 21.5% to 80.8 million sets in 2017 and then 117 million sets in 2021. In the unmanned era, Lidar will enter into rivalry with camera over the position of master sensor and is more likely to win out. Camera module won't see dramatic increase in shipments in the unmanned era."

ResearchInChina keeps believing in stereo camera market potential:

"Technologically, stereo camera's price has gone down and competitiveness has been boosted as shipments increase. Concerning luxury cars, main products of Benz, BMW, JLR and Lexus are likely to be configured with stereo camera in 2021. At the end of 2016, Denso launched small stereo camera, which would find wide application in Toyota compact cars. Besides, Honda and Hyundai are developing stereo camera system. It is predicted that by 2020 stereo camera will win one-third market share. Trifocal essentially as monocular camera poses no threat to stereo camera and has very limited market potential (only to be employed by Tesla and Volvo)."

John Ladd Died

While we are at sad news, BillingsGazette reports that John Ladd has passed away. John used to work for Aptina, Rambus, BAE, and Altasens and holds over 70 patents, mostly in image sensor technology, which was an impressive number at his age of 38.

Howard Rhodes Passed Away

Omnivision CTO Howard Rhodes died from cancer on June 24, 2017 in a NY hospital. The company says in an email message:

“We are incredibly saddened by the sudden passing of our dear friend and colleague, Dr. Howard E. Rhodes. Part of OmniVision’s leadership team since 2005, Dr. Rhodes’ pioneering work in CMOS imaging has shaped the course of OmniVision, and indeed much of the digital imaging industry. Holding over 200 patents, Dr. Rhodes earned his Ph.D. degree in Solid State Physics from the University of Illinois. His work and contributions to our groundbreaking BSI (backside-illumination technology) have impacted our industry in profound ways and we will be forever grateful for his loyalty, dedication, and technical brilliance. His unwavering determination to see OVT succeed has been an inspiration and, to a great extent, has defined us as a company. Our heartfelt thoughts and condolences go out to his family at this difficult time.”

Omnivision web site has not been updated yet:

Update: Omnivision posted a memorial page on its Facebook site:

Yole Reports on Status of the CMOS Sensor Industry 2017

Yole Developpement publishes a report "Status of the CMOS Image Sensor Industry 2017." Few quotes from the report are below:

"CMOS imaging has benefited from huge market demand and a technology-driven environment, resulting in an $11.6B industry in 2016.

The mobile market is key for the CMOS image sensor (CIS) industry. Despite saturation in the number of handsets, the CIS market has been able to maintain a 10.5% compound annual growth rate (CAGR) for the 2016-2022 period due to the introduction of dual and 3D cameras.

Sony has established itself as industry leader, market and technology wise. However, the 2016 earthquake in Japan slowed down its operation and helped maintain the growth of its close competitors. Indeed, despite Toshiba’s exit from the market, two thirds of players have seen growth in the last year. Samsung, Omnivision and Panasonic have delivered 15% year-on-year growth. Those large players are increasing the weight Asia carries in the CIS industry.

In the US, On Semi has suffered from the public relations mess that followed a fatal crash involving Tesla’s semi-autonomous driving system, which included a Mobileye sensor. Automotive cameras are safety-critical and therefore the reaction is strong if performance does not match expectations.

Another noticeable activity has been STMicroelectronics’ revival of its CIS business through the development of 3D ‘Time of Flight’ (ToF) devices. They will probably be the biggest event of the end of 2017, if they materialize in the Apple iPhone 8 as is rumored.

The impact of 3D semiconductors is currently a key element of the competition. Samsung has finally joined Sony as a provider of stacked back-side illuminated (BSI) sensors. Omnivision and SK Hynix will also soon launch this technology. Meanwhile, SMIC and STMicroelectronics will probably be the next players to unveil their versions of stacked technology.

Increases in CIS production capacity have momentarily stopped due to this stacked trend. This is currently mainly due to Sony, which can source its logic wafers from TSMC. In coming years Sony, Samsung, STMicroelectronics, HLMC and SK Hynix will all announce extra CIS capacity in order to meet market demand, despite stacked BSI adoption."

Samsung Presents ISOCELL Brand, Four Sub-Brands

Samsung presents its image sensor brand "ISOCELL". First introduced in 2013, ISOCELL technology separates each pixel with a physical barrier that reduces color crosstalk among pixels. This allows a better color fidelity even with smaller pixels.

Samsung ISOCELL is complemented by four technological sub-brands—Bright, Fast, Slim and Dual—that respond to specific market demands:

• ISOCELL Bright sensors deliver bright and sharp images with high color fidelity and reduced noise in low light environments
• ISOCELL Fast sensors provide fast autofocus onto still or moving objects even when dark
• ISOCELL Slim sensors adopt the smallest pixel sizes available in the market at 0.9-1.0um, yet produce high quality images for the slimmest devices
• ISOCELL Dual sensors can be mixed and matched in various combinations on consumer devices to bring about features demanded in the latest dual camera trend

“Samsung ISOCELL is a brand that represents the essence of our leading pixel technologies. We expect the ISOCELL brand to help consumers easily acknowledge and confide in camera performance as well as overall quality of the device,” said Ben Hur, VP of System LSI marketing at Samsung Electronics. “With our advanced image sensor technologies, Samsung will continue to bring innovation to cameras used in smartphones and other applications.”

IC Insights Forecasts Nonstop Image Sensor Sales Records Through 2021

IC Insights report says "CMOS image sensor sales are on pace to reach a seventh straight record high this year and nothing ahead should stop this semiconductor product category from breaking more annual records through 2021.

After rising 9% in 2017 to about $11.5 billion, worldwide CMOS image sensors sales are expected to increase by a compound annual growth rate (CAGR) of 8.7% to $15.9 billion in 2021 from the current record high of $10.5 billion set in 2016, based on the five-year forecast.

Automotive systems are forecast to be the fastest growing application for CMOS image sensors, rising by a CAGR of 48% to $2.3 billion in 2021 or 14% of the market’s total sales that year.

CMOS image sensor sales for cameras in cellphones are forecast to grow by a CAGR of just 2% to $7.6 billion in 2021, or about 47% of the market total versus 67% in 2016 ($7.0 billion). Smartphone applications are getting a lift from dual-camera systems that enable a new depth-of-field effect (known as “bokeh”).

Competition among CMOS image sensor suppliers is heating up for new three-dimensional sensing capability using time-of-flight (ToF) technology and other techniques for 3D imaging and distance measurements... CMOS technology has progressed to the point of supporting integration of ToF functions into small chip modules and potentially down to a single die. Sony, Samsung, OmniVision, ON Semiconductor, STMicroelectronics, and others have rolled out and developed 3D image sensors. Infineon has also jumped into the image sensor arena with a 3D offering that is built in ToF-optimized CMOS technology."

Sony Integrates ToF-based Face Recognition in Xperia Smartphone

Techcrunch reports that Sony is to demo an integrated Softkinetic ToF camera inside Xperia smartphone paired with KeyLemon software for "3D face recognition system that could let users authenticate themselves with one photo." Softkinetic PR partially confirms this info.

Imatest on Challenges in Automotive Image Quality

Imatest publishes Norman Koren presentation "Challenges in Automotive Image Quality" presented in Autosens Detroit in May:

Thrown Camera Imaging

Ishikawa Watanabe Laboratory of Tokyo University shows how throwing a high-speed camera up can be useful to create images from unique points of view:

Omnivision Announces Dual Camera Set, Single Camera Sensors

PRNewswire: Omnivision's OV13A10 and OV13A1Q is a pair of 13MP stacked-die sensors with second-generation PureCel Plus 1.0um pixel. The new sensors are intended for 2x optical zoom to front- and rear-facing cameras in mainstream smartphones. The OV13A10 is a telephoto sensor designed specifically for dual-camera applications with a module z-height of less than 6mm, and features zig-zag HDR (zHDR) and PDAF. A customized chief ray angle (CRA) enables the OV13A10 to be used as a tele-sensor in a 2X optical zoom camera configuration.

PRNewswire: OmniVision also announces the OV16B10, a 16MP stacked image sensor designed for the next generation of flagship smartphones. Built on the second-generation, 1.12um PureCel Plus-S pixel. It includes PDAF and zHDR too. zHDR uses a long and short exposure in a single frame. When compared with traditional frame-based HDR techniques, the zHDR is said to have increased DR with minimum ghosting artifacts.

The OV16B10 has a built-in feature that synchronizes the frames and supports context switching when it is used in dual-camera configurations, supporting image fusion while simplifying camera system architecture. Additionally, the OV16B10 features a gyro interface that reads and synchronizes the motion data from an external gyroscope to enable precise image stabilization for video and still capture.

The new sensors are currently available for sampling and are expected to enter volume production in Q4 2017.

Multi-Resolution Image Sensor Paper

MDPI Sensors starts publishing a Special Issue on Image Sensors. The first paper in this issue is "A Multi-Resolution Mode CMOS Image Sensor with a Novel Two-Step Single-Slope ADC for Intelligent Surveillance Systems" by Daehyeok Kim, Minkyu Song, Byeongseong Choe, and Soo Youn Kim from Dongguk University-Seoul, Korea. Here is the paper's main idea explanation:

Two Recent Theses

University of Michigan publishes PhD thesis "CMOS Sensors for Time-Resolved Active Imaging" by Jihyun Cho. After a quick overview of image sensor architectures and noise, the thesis describes a single shot FLIM imager and a ToF imager with background light suppression:

Lund and Linköping Universities, Sweden, publish MSc thesis "Demosaicing using a Convolutional Neural Network approach" by Karin Dammer and Ronja Grosz. The main intention of CNN approach is reduction of artifacts:

The results are somewhat mixed: "Using convolutional neural networks is a valid method for demosaicing images with good results and it could replace a method using linear interpolation. Our CNN method outperforms the multilayer perceptron by a difference of 7.14 dB in the peak signal to noise ratio. The convolutional neural network performs well when using L2 and PSNR as loss functions when training the network, however SSIM does not perform as well. Despite the relatively good result the network would benefit from using an error metric that is better at indicating the presence of image artifacts and color errors. The network did not significantly benefit from the residual layer nor a deconvolution layer."

Light Field Sensor on a Chip

Light Field Forum notices a recently published SK Hynix patent application US20170179180 "Light Field Imaging Device and Method for Fabricating the Same" by Jong Eun Kim proposing the integration of the whole light field system in the microlens stack:

New Sony Products

Sony presents 4/3-inch 10.71MP IMX294CJK image sensor for 4K 120fps video and security applications. Thanks to the 4.63 µm large pixel achieves SNR1s of 0.14 lx, and use of a Quad Bayer pixel structure (see Figure 1) is claimed to realize an HDR with no time difference artifacts.

Sony also unveils 1-inch 20.48MP IMX183CLK-J / CQJ-J Monochrome/Color sensor pair. The sensor features 2.4 µm BSI pixel with a proven record in products for security camera and industrial applications.

Sony also announces 2MP, 6MP, and 12MP monochrome sensors for industrial camera applications based on BSI pixels: IMX290LLR, IMX178LLJ and IMX226CLJ.

Light Talks about L16 Camera Complexity while Gearing up for Mass Production

Light Co. is gearing up for mass production of the L16 multi-camera. There are more than 3,000 different parts in the L16 camera, which are supplied from at least six different countries around the world. Once they arrive at our factory in China, it takes 60 sets of hands to assemble each camera. Our manufacturing team follows an extensive 79-step process—and that doesn’t even include the 13 times we check each camera’s functionality.

The company is preparing to start shipping cameras to pre-order customers on July 14th, 2017, less than a month from now.

The company's video explains the camera design complexity:

Tesla Autopilot Management Changes

Techcrunch: Tesla gets rid of Apple Swift language manager Chris Lattner who was brought to the company 6 months ago to lead the autonomous driving division. The company says:

"Chris just wasn’t the right fit for Tesla, and we’ve decided to make a change. We wish him the best.

Andrej Karpathy, one of the world’s leading experts in computer vision and deep learning, is joining Tesla as Director of AI and Autopilot Vision, reporting directly to Elon Musk. Andrej has worked to give computers vision through his work on ImageNet, as well as imagination through the development of generative models, and the ability to navigate the internet with reinforcement learning. He was most recently a Research Scientist at OpenAI.

Andrej will work closely with Jim Keller, who now has overall responsibility for Autopilot hardware and software."

Dual Camera Patent Wars Looming?

Digitimes reports that Taiwan-based camera module Altek is suing Shenzhen O-film Tech, and Beijing Jingdong Century Information Technology, a sales agent in the China market, in the Beijing Intellectual Property Court for infringement of its dual camera module patent.

The dual-lens module, supplied by Shenzhen O-film, used in the Hong-mi Pro smartphone launched by China-based Xiaomi infringes upon its patent, Altek alleges. Beijing Jingdong acts as the China sales agent for the smartphone.

Altek was first to unveil smartphone dual-lens camera module in 2014 and is said to supply its dual camera modules for more than 30 smartphone models by HTC, Huawei, ZTE, Coolpad, Nubia, GiONEE and Smartisan.

InstantFlashNews reports that the patent depicts a camera module with its optical axes adjusted to provide different photography effects. As this dual camera module is manufactured by O-Film, and JD.com is selling the handset, Xiaomi is not being sued for now.

Analog Devices Demos HDR & Contextual Awareness

Analog Devices posts another demo video showing the capabilities of its SNAP sensor:

Thesis on Open-Source Eye Tracking Database

Presenting PhD thesis for discussion a week before the defense seems to become a new trend in Finland. University of Eastern Finland presents PhD Thesis by Ana Gebejes entitled "Spectral video: application in human eye analysis and tracking" to be defended on June 30, 2017.

Using a 450nm-950nm spectral video device in eye-tracking, the researchers from the University of Eastern Finland have created a novel – first of its kind – combined spectral video/spectral image database: the SPectral Eye vidEo Database, SPEED.

A Youtube video explains the database creation work:

Caltech Presents OPA Camera

Caltech Professor Ali Hajimiri with graduate students present Optical Phase Array (OPA)-based camera: "What the camera does is similar to looking through a thin straw and scanning it across the field of view. We can form an image at an incredibly fast speed by manipulating the light instead of moving a mechanical object." Their OSA paper "An 8x8 Heterodyne Lens-less OPA Camera" explains the principle:

The OPA chip placed on a penny for scale.

2016 Gartner Image Sensor Report

InstantFlashNews publishes August 2016 Gartner report on image sensor market. A few quotes from the report:

Key Findings:

• The growth in the biggest market for CMOS image sensors — smartphones — is ebbing. Leading suppliers are focusing on new markets, such as automotive, industrial and wearables, to drive revenue growth.
• Capturing share in new markets requires new camera technologies, such as 3D cameras, dual cameras, software technologies, and algorithms for image processing and recognition.
• New safety regulations for automobiles are emerging across key markets, such as North America, Europe, Japan and China. Compliance with these regulations will demand increased use of image sensors in a variety of automobile electronic equipment over the next three years.

CMOS image sensor product managers must:

• Focus on developing specific products targeting new markets, such as industrial, automotive and wearable devices, and allocate resources to gain share quickly in these markets.
• Develop computational camera software technologies and algorithms, such as high-speed processing, image recognition (face or object) and AI technologies, which will be adopted into, for example, the smartphone, automotive and industrial markets. And they should integrate them into one chip or a single package.
• Consider acquiring packaging, lens and camera module suppliers and IP companies (image signal processors, connectivity, memory and artificial intelligence) to provide a complete product to the end customer.

TechInsights Survey of Stacked Image Sensors

TechInsights publishes a second blog post based on Ray Fontaine presentation at IISW 2017. This part covers stacked image sensor technology from TSV and oxide bonding to Cu-to-Cu direct bonding in the recent products:

IISW 2017 Awards

IISW 2017 held on May 30-June 2 in Hiroshima, Japan has announced a number of Awards:

2017 Walter Kosonocky Award for the best paper published in 2015-16 goes to

“A 0.13 μm CMOS System-on-Chip for a 512 × 424 Time-of-Flight Image Sensor With Multi-Frequency Photo-Demodulation up to 130 MHz and 2 GS/s ADC”
Cyrus S. Bamji, Patrick O’Connor, Tamer Elkhatib, Swati Mehta, Barry Thompson, Lawrence A. Prather, Dane Snow, Onur Can Akkaya, Andy Daniel, Andrew D. Payne, Travis Perry, Mike Fenton, and Vei-Han Chan
Microsoft, USA, IEEE JSSC, Vol. 50, No. 1, pp. 303-318, January 2015

Pioneering Achievement Award for Contribution to R&D and commercialization of high-performance and high-resolution CCD image sensors is presented to Tetsuo Yamada:

Exceptional Lifetime Achievement Award Significant Contributions to the Advancement of Solid-State Image Sensors Including the Development Of On-chip Microlens Technology and VOD Structures for anti-blooming is presented to Yasuo Ishihara:

The Best Poster Award goes to:

“Fully Depleted, Monolithic Pinned Photodiode CMOS Image Sensor Using Reverse Substrate Bias”
Konstantin D. Stefanov, Andrew S. Clarke, James Ivory and Andrew D. Holland. The Open University, UK

Thanks to JN for the docs!