Canon Super-35mm Cinema Sensor Explained

Canon published few whitepapers talking about design considerations of its Super-35mm sized CMOS sensor for recently announced EOS C300 video camera. The first one "New 35mm CMOS Image Sensor for Digital Cine Motion Imaging" gives the sensor spec:

The color filter array is the classic Bayer. Canon explains the resolution choice:

Illustrating the separate CFA array and the CMOS imager while also
showing the CFA separated into its component color filters to better expose
the structure of their respective sparsely sampled lattices

"The image sensor readout strategy radically departs from the customary “De-Bayer” deployment of quincunx sampling of the green photosites to maximize the green video resolution (and hence the matriced Luma resolution). The design strategy of this new sensor is to not to seek any form of “4K” resolution — but rather to specifically confine the reconstruction of each of the R,G, and B video components to a full digital sampling structure of 1920 (H) x 1080 (V) — according to the SMPTE 274M HDTV Production Standard."

Showing the concept of structuring the final Green video component within the
pre-processing LSI from the two dual video readouts from the CMOS image sensor

"The dual Green process offers the following significant technical advantages:

1. Doubles the effective saturation level of the summed Green output video
2. Increases the noise of the final Green output only by a factor of square root of two
3. Combination of 1) and 2) increases the effective dynamic range of the green signal — and as a consequence, that of the matriced Luma signal
4. Increases the effective output green video bit depth
5. The half-pixel offset between the two separate green sampling lattices — both horizontally and vertically — virtually eliminates the first order sideband spectra associated with the sensor sampling process. This eliminates green aliasing.
6. Creates an effective FIR* filter within the readout process that aids the optimization of the horizontal MTF and the progressive vertical MTF and associated aliasing."

The summation of two greens is said to increase the DR from 70dB to 73.5dB in green (in fact, from 70.5 to 73.5) or to 72dB in luma (12 stops).

Although the camera frame rate is 24p fps, the readout speed is 1/60s to reduce rolling shutter effects:

In 60i mode each half-frame is read at 1/120s - same per-row speed as in 24p mode.

The low read noise is achieved by limiting the readout amplifier bandwidth, as shown below:

Another Canon whitepaper "RGB Resolution Considerations in a New CMOS Sensor for Cine Motion Imaging" shows advantages of the proposed green processing in resolution extension:

Showing the two separate green 1920 (H) x 1080 (V) photosite lattices
and the horizontal and vertical timing offsets between each of the
two “diagonal” pixels that are summed during the readout process

The resulting horizontal and vertical MTFs of the whole system are improved:

The summary says: "A new CMOS image sensor has been described. It represents a definitive decision by Canon to enter the global field of digital cinematic motion imaging. It is anticipated that there will be many progressive advances in the years ahead. Accordingly, a priority was assigned to taking a first step into this important field of imaging by placing an initial focus on originating a very high quality RGB video component set specifically intended for high-performance High definition video production."

Another whitepaper is titled "Sensitometric Characteristics of EOS C300 Digital Cine Camera" and mainly focused on system processing of the video signal, introduces "Canon-Log" response.

Samsung IEDM 2011 Paper

Eric Fossum put his Samsung IEDM 2011 paper on-line:

"A 192×108 pixel ToF-3D image sensor with single-tap concentric-gate demodulation pixels in 0.13 μm technology"
T.Y. Lee, Y.J. Lee, D.K. Min, S.H. Lee, W.H. Kim, S.H. Kim, J.K. Jung, I. Ovsiannikov,
Y.G. Jin, Y.D. Park, E.R. Fossum, and C.H. Chung

"A 3D-ToF FSI image sensor using novel concentric photogate [CG] pixels with single-tap operation is described. Through the use of CG structure, we are able to achieve high DC at larger pixel pitches. The new CG pixel structure substantially improves DC [demodulation contrast] to 53% at 20MHz at 28 μm pixel pitch. Recent initial results from a backside-illuminated (BSI) implementation of the same sensor show further improved performance and will be reported elsewhere."

Truesense Imaging Inc. and Digital Optics Corp.

As written in comments, the recently acquired Kodak Image Sensor Solutions has been quietly renamed to Truesense Imaging, Inc. Kodak has first used Truesense name for its W-RGB color filter products almost 3 years ago. I wonder if the new company name meant to emphasize the W-RGB products importance.

Meanwhile, Tessera renamed its imaging and optics division into Digital Optics Corporation. The new entity is responsible for wafer-scale optics (former Shellcase), EDoF (former Eyesquad and Dblur), MEMS AF motors (former Siimpel), micro-optics (the original bearer of Digital Optics Corporation name, acquired by Tessera in 2006) and image enhancement software (former Fotonation). It appears that the division has been renamed and separated into the wholly owned subsidiary in June 2011.

Another part of Tessera dealing with chip packaging is separated and renamed too. Its new name is Invensas. In Nov. 2011 Invensas acquired patent assets of California-based TSV foundry ALLVIA. It does not seem to target image sensor applications though.

1.8 Gigapixel Camera Deployed on Helicopter Drones

BBC, US Army: The A160 Hummingbird helicopter-style drones with 1.8 Gigapixel color cameras are being developed by the US Army promising "an unprecedented capability to track and monitor activity on the ground".

A statement added that three of the sensor-equipped drones were due to go into 1-year trial service in Afghanistan in either May or June 2012 as a part of a Quick Reaction Capability, an acquisition approach aimed at delivering cutting-edge and emerging technologies to theater. The army developers and engineers are now finishing up some wiring work on the A160 aircraft and performing ground tests with the ARGUS sensor suite.

Boeing built the first drones, but other firms can bid to manufacture others. The 1.8 Gigapixel ARGUS-IS camera is developed and manufactured by BAE Systems.

The army said that was enough to track people and vehicles from altitudes above 20,000 feet (6.1km) across almost 65 square miles (168 sq km). In addition, operators on the ground can select up to 65 steerable "windows" following separate targets to be "stared at".

DARPA is also working with the UK-based division of BAE Systems to develop a more advanced version of the Argus-IS sensor that will offer night vision. It said the infrared imaging sensors would be sensitive enough to follow "dismounted personnel at night". In addition, the upgrade promises to be able to follow up to 130 "windows" at the same time. The system's first test flight has been scheduled to take place by June 2012.

Thanks to CDM for the link!

Digitimes: Samsung and Sony to Supply Sensors for Next Generation iPads

Digitimes quotes its sources saying that next generation iPad 3 would be released in two versions. The high end version will feature 8MP camera with Sony sensor. As for the mid-range model, Samsung is said to be among the suppliers of its 5MP sensor.

The new iPad 3 tablets are to be announced at iWorld on Jan. 26, 2012, according to the newspaper. The original version of iPad was announced on Jan. 27, 2010, while the iPad 2 was first shown on March 2, 2011.

Microsoft Proposes Double Helix PSF for Depth Sensing

Microsoft patent application US20110310226 "Use of wavefront coding to create a depth image" by Scott McEldowney proposes a fresh idea to acquire image depth information.

Here is the original description:

"[A] 3-D depth camera system includes an illuminator and an imaging sensor. The illuminator creates at least one collimated light beam, and a diffractive optical element receives the light beam, and creates diffracted light beams which illuminate a field of view including a human target. The image sensor provides a detected image of the human target using light from the field of view but also includes a phase element which adjusts the image so that the point spread function of each diffractive beam which illuminated the target will be imaged as a double helix. [A] ...processor ...determines depth information of the human target based on the rotation of the double helix of each diffractive order of the detected image, and in response to the depth information, distinguishes motion of the human target in the field of view."

Actually, it's much easier to understand this idea in pictures. Below is the illuminator with a diffractive mask 908:

There is another mask 1002 on the sensor side:

Below is the proposed double-helix PSF as a function of distance. One can see that the two points line angle changes as a function of depth:

The orientation angle of the PSF points depends on wavelength (not shown here, see in the application) and the distance (shown below):

From this angle the object distance can be calculated - this is the idea. Microfoft gives an image example and how it changes with the distance in what looks like Wide-VGA sensor plane:

Update: As written in comments, University of Colorado, Denver has been granted a patent US7705970 on a very similar idea. A figure in the patent looks very similar:

1/f and RTS Noise Reduction

As mentioned in Theses post, Oregon State University published Drake A. Miller's PhD Thesis "Random Dopants and Low-Frequency Noise Reduction in Deep-Submicron MOSFET Technology". The thesis is quite rich in experimental data os pixel source follower noise. The figure below shows more than order of magnitude variations in 1/f noise across the wafer:

Noise spectral power plots of 10 devices taken from
10 different locations across the wafer (see inset).

Any channel doping, such as Vth adjust, significantly increases 1/f and RTS noise:

Box plots of source follower noise power spectrum plots.
Red (Dark) boxes are doped devices.
Green (Light) boxes are undoped “native” transistors.

Few Vth adjust splits were measured:

It's not clear why S4 and S7 are not shown, but S1-S3 clearly show noise improvement:

The total read noise histogram clearly demonstrates the advantage of lightly doped source follower:

RTS Statistics shows the same:

Photons to Bits and Beyond Presentation On-Line

Eric Fossum published the pdf notes of his lecture "Photons to Bits and Beyond. The Science and Technology of Digital Imaging".

Recent Image Sensor Theses

There are few recently published image sensor theses:

"Pixel and Readout Circuit of a Wide Dynamic Range Linear-Logarithmic Current-Mode Image Sensor"
MS Thesis by Elham Khamsehashari, Aug. 2011
ÉCOLE POLYTECHNIQUE DE MONTRÉAL

"This thesis presents a current-mode CMOS image sensor operating in linear-logarithmic response. The objective of this design is to improve the dynamic range of the image sensor, and to provide a method for mode detection of the image sensor response. One of the motivations of using current-mode has been the shrinking feature size of CMOS devices. This leads to the reduction of supply voltage which causes the degradation of circuit performance in term of dynamic range. Such problem can be alleviated by operating in current-mode. The column readout circuits are designed in current-mode in order to be compatible with the image sensor. The readout circuit is composed of a firstgeneration current conveyor, an improved current memory is employed as a delta reset sampling unit, a differential amplifier as an integrator and a dynamic comparator."

"Single Shot High Dynamic Range and Multispectral Imaging Based on Properties of Color Filter Arrays"
MS Thesis by Paul M. Simon
UNIVERSITY OF DAYTON, May 2011

"This paper addresses the difficulty of generating High Dynamic Range (HDR) images using current Low Dynamic Range (LDR) camera technology. Typically, several LDR images must be acquired using various camera f-stops and then the images must be blended using one of several exposure bracketing techniques to generate HDR images. Based on Fourier analysis of typical Color Filter Array (CFA) sampled images, we demonstrate that the the existing CFA sampled images provide information that is currently underutilized. This thesis presents an approach to generating HDR images that uses only one input image while exploiting that underutilized CFA data. We propose that information stored in unsaturated color channels is used it to enhance or estimate details lost in saturated regions."

One must note that the DR extension is not that big and is based on the assumption that not all colors saturate simultaneously.

"Analysis, Modeling and Dynamic Optimization of 3D Time-of-Flight Imaging Systems"
PhD Thesis by Mirko Schmidt
Ruperto-Carola University of Heidelberg, Germany, July 2011

"This thesis covers four main contributions: A physical sensor model is presented which enables the analysis and optimization of the process of raw image acquisition. This model supports the proposal of a new ToF sensor design which employs a logarithmic photo response.
Due to asymmetries of the two read-out paths current systems need to acquire the raw images in multiple instances. This allows the correction of systematic errors. The present thesis proposes a method for dynamic calibration and compensation of these asymmetries. It facilitates the computation of two depth maps from a single set of raw images and thus increases the frame rate by a factor of two.
Since not all required raw images are captured simultaneously motion artifacts can occur. The present thesis proposes a robust method for detection and correction of such artifacts.
All proposed algorithms have a computational complexity which allows real-time execution even on systems with limited resources (e.g. embedded systems). The algorithms are demonstrated by use of a commercial ToF camera."

"Random Dopants and Low-Frequency Noise Reduction in Deep-Submicron MOSFET Technology"
PhD Thesis by Drake A. Miller
Oregon State University, March 2011

Quite significant RTS and 1/f noise reduction in image sensors has been reported:
"In the case of this research it was shown that once the noise source and mechanism was understood necessary steps could be taken to reduce the source of the noise. Two examples shown here are the impact of substrate bias and modification of the doping levels. Substrate biasing is a relatively straight forward approach to reducing the noise and has been show here to have this repeatable effect. With additional understanding of the percolation currents modification of the channel dopant profile can serve as an additional means for device noise improvement. Once understood, these relatively easy steps, as in the case of reducing the implant dose in the channel, verified the theory and model developed during this research and resulted in a superior performing CMOS image sensor
product."

e2v Applies for Electron Multiplying CMOS Sensor

e2v applies for a patent extending its EMCCD technology to the realm of CMOS sensors: "Electron multiplication image sensor and corresponding method" by Frédéric Mayer (France). Fig. 1 of the US20110303822 application shows a prior art 4T pixel having a pinned photodiode PHD:

e2v proposes to split the PHD into two with the "accelerating gate" GA in between, as on Fig. 2. By applying multiple voltage pulses on GA the electrons can be moved in and out of it, as shown on Fig. 3.

"The electron multiplication takes place during the charge integration and in the photodiode itself in the sense that the electrons (photogenerated or resulting already from the impacts of carriers with atoms) are accelerated in turn from the photodiode towards the accelerating gate and from the accelerating gate towards the photodiode. During these movements, impacts with atoms of the semiconductor layer of the photodiode region or of the region located beneath the accelerating gate make other electrons in the valence band pass into the conduction band. These electrons lose energy during these impacts but they are again accelerated by the electric field that is present.

The number of alternations in potential applied to the accelerating gate defines the overall multiplication coefficient obtained at the end of an integration period T, i.e. between two successive pulses for transferring charge from the photodiode to the charge storage region."

Fig. 4 shows one of the possible pixel layouts with GA located in the middle of PHD.

Update: As said in comments, in 2009 Sanyo published a different idea of electron multiplying CMOS pixel. The idea is shown on the figure below:

Update #2: As EF said in comments, Sanyo presented its electron multiplying sensor at ISSCC 2009 (paper, presentation). The pixel structure and the gain non-uniformity are taken from the presentation slides: