Two Recent Theses

University of Michigan publishes PhD thesis "CMOS Sensors for Time-Resolved Active Imaging" by Jihyun Cho. After a quick overview of image sensor architectures and noise, the thesis describes a single shot FLIM imager and a ToF imager with background light suppression:

Lund and Linköping Universities, Sweden, publish MSc thesis "Demosaicing using a Convolutional Neural Network approach" by Karin Dammer and Ronja Grosz. The main intention of CNN approach is reduction of artifacts:

The results are somewhat mixed: "Using convolutional neural networks is a valid method for demosaicing images with good results and it could replace a method using linear interpolation. Our CNN method outperforms the multilayer perceptron by a difference of 7.14 dB in the peak signal to noise ratio. The convolutional neural network performs well when using L2 and PSNR as loss functions when training the network, however SSIM does not perform as well. Despite the relatively good result the network would benefit from using an error metric that is better at indicating the presence of image artifacts and color errors. The network did not significantly benefit from the residual layer nor a deconvolution layer."

Light Field Sensor on a Chip

Light Field Forum notices a recently published SK Hynix patent application US20170179180 "Light Field Imaging Device and Method for Fabricating the Same" by Jong Eun Kim proposing the integration of the whole light field system in the microlens stack:

New Sony Products

Sony presents 4/3-inch 10.71MP IMX294CJK image sensor for 4K 120fps video and security applications. Thanks to the 4.63 µm large pixel achieves SNR1s of 0.14 lx, and use of a Quad Bayer pixel structure (see Figure 1) is claimed to realize an HDR with no time difference artifacts.

Sony also unveils 1-inch 20.48MP IMX183CLK-J / CQJ-J Monochrome/Color sensor pair. The sensor features 2.4 µm BSI pixel with a proven record in products for security camera and industrial applications.

Sony also announces 2MP, 6MP, and 12MP monochrome sensors for industrial camera applications based on BSI pixels: IMX290LLR, IMX178LLJ and IMX226CLJ.

Light Talks about L16 Camera Complexity while Gearing up for Mass Production

Light Co. is gearing up for mass production of the L16 multi-camera. There are more than 3,000 different parts in the L16 camera, which are supplied from at least six different countries around the world. Once they arrive at our factory in China, it takes 60 sets of hands to assemble each camera. Our manufacturing team follows an extensive 79-step process—and that doesn’t even include the 13 times we check each camera’s functionality.

The company is preparing to start shipping cameras to pre-order customers on July 14th, 2017, less than a month from now.

The company's video explains the camera design complexity:

Tesla Autopilot Management Changes

Techcrunch: Tesla gets rid of Apple Swift language manager Chris Lattner who was brought to the company 6 months ago to lead the autonomous driving division. The company says:

"Chris just wasn’t the right fit for Tesla, and we’ve decided to make a change. We wish him the best.

Andrej Karpathy, one of the world’s leading experts in computer vision and deep learning, is joining Tesla as Director of AI and Autopilot Vision, reporting directly to Elon Musk. Andrej has worked to give computers vision through his work on ImageNet, as well as imagination through the development of generative models, and the ability to navigate the internet with reinforcement learning. He was most recently a Research Scientist at OpenAI.

Andrej will work closely with Jim Keller, who now has overall responsibility for Autopilot hardware and software."

Dual Camera Patent Wars Looming?

Digitimes reports that Taiwan-based camera module Altek is suing Shenzhen O-film Tech, and Beijing Jingdong Century Information Technology, a sales agent in the China market, in the Beijing Intellectual Property Court for infringement of its dual camera module patent.

The dual-lens module, supplied by Shenzhen O-film, used in the Hong-mi Pro smartphone launched by China-based Xiaomi infringes upon its patent, Altek alleges. Beijing Jingdong acts as the China sales agent for the smartphone.

Altek was first to unveil smartphone dual-lens camera module in 2014 and is said to supply its dual camera modules for more than 30 smartphone models by HTC, Huawei, ZTE, Coolpad, Nubia, GiONEE and Smartisan.

InstantFlashNews reports that the patent depicts a camera module with its optical axes adjusted to provide different photography effects. As this dual camera module is manufactured by O-Film, and JD.com is selling the handset, Xiaomi is not being sued for now.

Analog Devices Demos HDR & Contextual Awareness

Analog Devices posts another demo video showing the capabilities of its SNAP sensor:

Thesis on Open-Source Eye Tracking Database

Presenting PhD thesis for discussion a week before the defense seems to become a new trend in Finland. University of Eastern Finland presents PhD Thesis by Ana Gebejes entitled "Spectral video: application in human eye analysis and tracking" to be defended on June 30, 2017.

Using a 450nm-950nm spectral video device in eye-tracking, the researchers from the University of Eastern Finland have created a novel – first of its kind – combined spectral video/spectral image database: the SPectral Eye vidEo Database, SPEED.

A Youtube video explains the database creation work:

Caltech Presents OPA Camera

Caltech Professor Ali Hajimiri with graduate students present Optical Phase Array (OPA)-based camera: "What the camera does is similar to looking through a thin straw and scanning it across the field of view. We can form an image at an incredibly fast speed by manipulating the light instead of moving a mechanical object." Their OSA paper "An 8x8 Heterodyne Lens-less OPA Camera" explains the principle:

The OPA chip placed on a penny for scale.

2016 Gartner Image Sensor Report

InstantFlashNews publishes August 2016 Gartner report on image sensor market. A few quotes from the report:

Key Findings:

• The growth in the biggest market for CMOS image sensors — smartphones — is ebbing. Leading suppliers are focusing on new markets, such as automotive, industrial and wearables, to drive revenue growth.
• Capturing share in new markets requires new camera technologies, such as 3D cameras, dual cameras, software technologies, and algorithms for image processing and recognition.
• New safety regulations for automobiles are emerging across key markets, such as North America, Europe, Japan and China. Compliance with these regulations will demand increased use of image sensors in a variety of automobile electronic equipment over the next three years.

CMOS image sensor product managers must:

• Focus on developing specific products targeting new markets, such as industrial, automotive and wearable devices, and allocate resources to gain share quickly in these markets.
• Develop computational camera software technologies and algorithms, such as high-speed processing, image recognition (face or object) and AI technologies, which will be adopted into, for example, the smartphone, automotive and industrial markets. And they should integrate them into one chip or a single package.
• Consider acquiring packaging, lens and camera module suppliers and IP companies (image signal processors, connectivity, memory and artificial intelligence) to provide a complete product to the end customer.