Eric Fossum on Image Sensor History

Eric Fossum posted on dpreview.com public forums a few interesting bits on image sensors history:

1st post about CCD vs. CMOS:
"I invented the CMOS technology so of course I feel pretty strongly that CMOS is better. We have had a lot of discussion about this in the imaging community but I think I have prevailed. I sold my company, Photobit, to Micron. Micron is now the biggest sensor manufacturer in the world, at least the past few quarters, and of course uses my CMOS technology. Sony just invested the better part of a billion dollars in a plant to make CMOS sensors. Truthfully, I am surprised by how fast CMOS sensor technology has entered into the dSLR market, but that is mostly due to some excellent CMOS sensors built by Canon (and now Sony). Most camera phones also use my technology and the sensor quality there is excellent. Too bad the rest of the camera does not measure up but that is just a matter of economics and not fundamental.
I started in CCDs and love that technology. My job at JPL was to come up with a better technology and I think we did, with the CMOS active pixel sensor. It has just taken 10 yrs or so for it to catch on. My only disappointment is that NASA is so so so slow to adopt new technology, even technology it spawned."

2nd post, CCD vs CMOS:
"The CMOS active pixel sensor was designed to take advantage of all the things we knew worked well in a CCD (e.g. pinned photodiode) and yet avoid the problems of a CCD which center around charge transfer. By putting the amplifier in each pixel ("active pixel") the signal could be read out of the sensor across wires instead of by using charge transfer. This saves power (important in cell phones and interplanetary spacecraft) and allows for fast readout (most high speed cameras use CMOS active pixel sensors now), reduced sensitivity to radiation (spacecraft again), and most importantly, manufacturability on process fabrication lines that are like mainstream microelectronics vs. the specialized line for CCDs. This makes them affordable. It also brought image sensor technology back to the USA, for those of us that care about that."

3rd post, Who invented pinned photodiode in CMOS sensor:
"I was there. Tom [Lee] and I were at the whiteboard and he and I invented the CMOS APS with pinned photodiode. I am not sure Paul [Lee] was even in the room at the moment, although he was part of the project team. Don't get me into a discussion of why Caltech (JPL) was not included on that particular patent despite a co-authored NASA New Technology Report that was filed.....politics..."

4th post, conitinuation of the above:
"In my opinion, Tom was the key CCD technologist at Kodak until he retired about 10 years ago. Paul Lee and Russell Gee (from JPL) did most of the layout work - Paul on the pixel and Russell on the surrounding circuits. We reported the first CMOS APS using a pinned photodiode at the 1995 IEEE Workshop on CCDs and Image Sensors in a joint JPL/Kodak paper. Kodak did almost nothing with the technology but did some tech transfer to Motorola in a short lived 3-way collaboration with Photobit. Photobit did sensor design, Motorola did wafer fab. It is a shame Kodak did not move more quickly or efficiently. They could have been way ahead of the rest of the world. Sigh. Natural selection I guess. But, a nice bunch of guys at Kodak and a pleasure to work with."

Post on Micron's UDTV project:
"At the Japanese Aichi World Expo this past summer, NHK demonstrated their 32 megapixel Ultra-Definition Television (UDTV) system that uses 4 8Mpixel CMOS active pixel sensors with a dichroic prism color splitting optical system. They also produced a big screen display system to demonstrate the (breathtaking) video footage to large audiences. The line to get in was one of the longest at the expo. We built these sensors around CY2000 for the NHK project.
-Eric (on behalf of the Photobit/Micron engineering team) "

Post on DSLR sensor opportunities:
"Canon makes all its own DSLR sensors. Samsung can make its own sensors (they have a huge CMOS image sensor group). But I agree with your point. Nice opportunity for a pure play sensor company like Micron.
btw, don't forget IBM makes sensors for Kodak, and DALSA (formerly Philips) makes high performance CCDs. There is still places for the DSLR manufacturers to shop for sensors."

Update Jan. 30: Post on Motorola problems:
"Moto fab was problematic in my opinion for 3 main reasons. (1) Very slow turnaround times, (2) technical problems with dark current and lag and (3) Very expensive wafer pricing (not sure this ever really emerged as a practical problem since eclipsed by 1 & 2. You may remember that Fisher came to Kodak as CEO from Moto and I think that was a key reason Kodak went this route despite our best advice."

Update Feb. 6: Post on trusted people network in CMOS imaging:
"When we started transferring the CMOS active pixel sensor technology out of JPL to US industry, one of the transferees was National Semiconductor. The principals there were Kevin Brehmer and Dick Merrill. Kevin went on to start his own CMOS sensor company (I forget the name) and soon sold it to Zoran (I think). Dick went and co-founded Foveon with his multi-layer photodiode idea, using the same readout architecture we transferred to National. Getting color information from a stacked multiphotodiode structure is an older idea, but Dick was the first to incorporate it into the CMOS APS architecture. Meanwhile, Carver Mead was on one of the JPL advisory boards and became a good friend and supporter of our CMOS APS work at JPL. Furthermore, their recent VP for Biz Dev was someone I hired and worked with at Photobit. That guy lasted longer than I could at Micron, but eventually left Micron for Foveon. (and now he works for me again). So, you are not going to find me saying too many negative things about Foveon. Good group of guys and I feel I had some influence on its creation. I just hope they can overcome the sensor limitations I refer to so they can be financially successful."
The company whose name Eric forgot is Pixelcam led by Kevin Brehmer. In more recent history, Zoran closed its image sensor department as failing to win any significant marketplace.

Cypress Samples 9MP Imagers

From Cypress Q4 2005 Report:
"Cypress sampled two new 9-megapixel CMOS image sensors for high-end digital still cameras. The CYIHDSC9000AA (color) and CYIHDSM9000AA (monochrome) sensors offer low-dark-current performance comparable with more expensive charge-coupled devices, delivering high-end image quality at an economical price."

Sharp 1.90um Pixel CCD

Sharp announced 1/2.5" 7MP CCD featuring 1.90um pixels - fo far the smallest pixels in mass production (link in japanese). The part number is RJ23S3BA0ET.

EETimes on Image Sensor Market

EETimes published a review on image sensor market re-capturing articles and interviews in the past year.
Quotes:
"Today, there are some 40 suppliers of CMOS image sensor (CIS) products in the booming but overcrowded market. Research firm Gartner Dataquest Inc. recently estimated the global CIS market would increase from $2.4 billion in 2004 to $3.2 billion in 2005 and surge to $5.6 billion by 2008."
According to our count, there are more than 60 of CIS companies, dealing with CIS. Some of them are in stealth mode yet, some are providing design services for others.

"Micron entered the CIS business by purchasing Photobit Corp., a developer of CMOS imaging chips for professional cameras. Today, Micron claims to be the world’s leading producer of CIS products with more than a 30 percent market share in the arena."

A conroversial quote: "CIS providers can agree on one thing: CCDs are running out of steam. For years, digital still cameras, camcorders and other products have used CCDs. But CMOS image sensors are making inroads in the consumer segment — at the expense of CCDs."
According to some research, CCDs are re-gaining market share in DSC.

InfoTrends Camera-Phone Forecast

Cellular-News: InfoTrends projects that worldwide camera phone shipments will grow from 233 million units in 2004 to 903 million units in 2010. By 2010, camera phones are expected to account for 87% of all mobile phone handsets shipped.

The chart below shows worldwide phones shipments in millions of units:

Following Foveon?

Xerox PARC and Research Center Jülich, Germany, presented a color "vertical sensor" without Bayer color filter array (Optics.org).
Very much like Foveon, the vertically integrated sensor works by detecting color information as a function of depth. However, the sensor materials are different: Blue light is absorbed by the top diode and green and red wavelengths pass through to silicon carbon and amorphous silicon absorption layers below. The vertical sensors should be deposited on top of CMOS readout electronics.

Transchip 2MP Sensor+ISP

HardwareZone: Transchip announced the release of its second generation fully programmable, CMOS camera module, the TC7040.
"Even in the dark, AlgoCam TC7040 captures rich, vibrant colors, due to a new pixel design with enhanced image processing algorithms," said Dr. Viktor Ariel, TransChip CEO.
AlgoCam TC7040 offers enhancements in light sensitivity, lower dark current noise, lower cross talk, increased sensitivity to blue hues and new radial anti-shading. The embedded, fully programmable image signal processor (ISP) in AlgoCam TC7040 performs automatic adjustments based on external lighting conditions, controls the auto-focus and zoom drivers directly in a closed loop and delivers necessary image enhancements.

Sony ClearVID Sensor

Sony announced a new generation of CMOS sensors for video cameras - ClearVID. The information about ClearVID is quite scarce.
Camcorderinfo.com reports that its pixels are tilted on 45 degrees and the number of green pixels is increased because human eye is more sensitive to green. The color filter is primary RGB.
The sensor contains 2M pixels, but Sony claims the combination of the new pixel alignment and signal processing produces equivalent quality of 4MP camera.
The official Sony press-release menitons also better low-light movie quality - quite a common statement for all recent sensor announcements.

Update Jan. 24, 2006: DPReview published article on ClearVID (Clear-Vivid) sensor. It has a nice picture explaining the idea:

First Production Liquid Autofocus Lens

Sunex and Varioptic announce first production liquid autofocus lens.
Sunex AFL872 lens is designed for CMOS sensors up to 1/3” format and up to 3 megapixel resolution. The lens has a full field of view of 62º with an F/2.8 aperture. Threaded for M8x0.35, the AFL872 will fit standard sensor boards, or can be modified to meet your mounting requirements.

Omnivision Sensor Enables Ultra-Thin Modules

Yahoo-Finance: Omnivision trumpets about shrink of VGA camera-phone module to just 6 x 6 x 4.1 mm.
OV7670 sensor inside the module has 1/6 inch format with 3.6um pixel pitch. The sensor has "unique non-linear micro lens shift technology" which "allows a reduction in the distance between sensor and lens, lowering the module height even further without losing image quality or camera performance". The (unnamed part number) module uses a simplified 2P lens design.
The module is available in samples now.

Sony 2.03um pixel CCDs

Sony announced 6MP 1/2.5" CCD with 2.03um pixel pitch - part name ICX624 (CX News).
To my knowledge this is the smallest pixel sensor in mass production.

Motorola and Kodak forge mobile-imaging alliance

DMasia.com: Motorola and Kodak formed a 10-year global product, cross licensing and marketing alliance for mobile imaging.
Motorola agrees to combine Kodak's image science and system integration expertise with Motorola's mobile devices.
DMasia.com: "The collaboration covers licensing, sourcing, software integration and marketing, and also extends to co-development of devices with joint engineering teams. For example, Kodak expects to supply its CMOS sensors to Motorola for use in its camera phones, as well as in any future devices the companies co-develop.
During 2006, Motorola plans to launch handsets with integrated software to enrich the Kodak Easyshare experience."
Forbes: "The collaboration will simplify the process of making prints from camera phones, allowing consumers to directly print photographs from home printers, retail kiosks or upload the pictures to Kodak's photo-sharing EasyShare gallery Web site.
The Kodak-enabled camera phones are expected to hit the market in late 2006."

No doubt, this is a big success for Kodak image sensor group, so far lacking first tier customers.

Planet82 Demos 0.1Lux Image Sensor

Market Wire: Planet82 makes the first U.S. demonstration of its innovative and highly sensitive image sensor, Single Carrier Modulation Photo Detector (SMPD). Planet82 claims its sensor is 2,000 times more sensitive to light than other image sensors. In a contradictory note ExtremeNano tells it's 1000 times more sensitive. It allows taking clear images even when the light level is less than 1 lux (or up to 0.1 lux).

SMPD technology is licensed to Planet82 by Korea Electronics Technology Institute (KETI) in 2003 (other source names ETRI (Korean Electronics and Telecommunications Research Institute) as the original patent owner). Planet82 CTO, Dr. Hoon Kim, is director of the Nano Scale Quantum Devices Research Center at KETI at the same time.

To make that high sensitivity possible, Planet82 applied the principles of quantum mechanics to produce thousands of electrons out of one photon. There is an unusual claim of minimized aperture ratio - quite the opposite of what other companies are trying. The sensor uses standard CMOS process.

Planet82 expects SMPD image sensor will firstly be available in CCTVs, camera phones and vehicle rear-view sensors in Q1/Q2 2006.

The company web site presents a little more info:

The sensors resolution is VGA/1MP, pixel pitch is 4um (quite large in today's standards). Aperture ratio is 1.87% - very small, I'm wondering how they integrate it with a compact lens.

A picture from the company web site shows the aperture ration is very small indeed (link):



Update: The CES demo sample chip is B&W, made in TSMC 0.35um process with no microlens. The company plans to have a color chip within 3-4 months.

Update: ABCnews-PCWorld: "Really Low-Light Photography: Planet82's SMPD sensor chip could make digital cameras more sensitive to light than the naked eye. In very low light, the chip produced an image of objects that I could barely see; a camera with a CCD chip was unable to capture an image. The image was black and white, though company representatives say color support is on the way soon. If this chip proves capable of delivering high enough resolution and low noise, you may never use your flash again. —Eric Butterfield"

Explanation of Sony CCD Defects

NE Asia Online: Sony CCD packaging defects came from insufficient bonding wires adhesion and corrosion to the wire bonding surface by the iodine contained in the adhesive used to secure the CCD cover.

"Bonding strength dropped because of "improvements" made in the production line at the Kokubu Plant of Sony Semiconductor Kyushu Corp of Japan in October 2002. Two wire bonding machines were replaced to boost productivity on the CCD line. Unfortunately, this change prevented adequate alloying between Au wires and Al electrodes.

On top of the reduced bond strength came the iodide compound from the adhesive used to secure the CCD top glass cover to the package. The adhesive absorbs moisture in high-temperature, high-humidity environments, and a reaction between the iodide compound and the water generates a corrosive gas inside the package."

Magnachip announces 2MP SOC sensor

Magnachip announced MC521DA 2MP sensor SOC (Yahoo-Finance). It is 2.8um pixel pitch 1/4" format sensor. The power consumption is 160mW at 15fps full resolution - quite impressive.
Interesting quote: "Through the use of optimized processing, special lens correction algorithms, and carefully selected optical positioning, MagnaChip is able to offer a 2.8u pixel (1/3.2" format) SOC sensor for 1/4" format modules."
The analog supply is dual voltage 1.8V and 2.7V - to save power probably. Oddly, that digital supply is 1.8V and 3V combo. Does this mean that the chip requires 3 supply voltages?

Just two weeks before Magnachip has announced 3.2MP sensor MC532MA (Digitimes, module pictured below). It features 2.57um pixel size and consumes 150mW at 12fps full resolution operation (Yahoo-Finance). Jason Hartlove, Senior Vice President and General Manager of the Imaging Solutions Division said: "By leveraging the best-in-class engineering teams we have in Korea, Japan, and the US, we expect to achieve considerable growth in our markets next year."

With such a slew of product introductions Magnachip seems to be well on track to reach their ambitious goals this year.

The 3MP module size is 11x11x7mm, including AF (from Digitimes):

IC Insight Camera-Phones Forecast

NE Asia Online: IC Insight predicts camera-phone shipments to reach 910M units in 2009.


Share of camera phones increases from 45% in 2005 to 75% in 2009 (73% in the graph). Most of the high-resolution camera phones sold in 2009 are expected to have higher than 2Mpixel capability.