EDoF Rebirth

Extended Depth of Focus (EDoF) techniques used to be a popular topic 10-15 years ago, as long as the mainstream camera phone resolution has not exceeded 2MP. However, EDoF companies were unable to scale their resolution beyond that point.

Technology University of Tampere, Finland and FLIR seem to find a good application for EDoF in MWIR cameras, where resolutions are still low up to these days. Their paper "A novel two- and multi-level binary phase mask design for enhanced depth-of-focus" by Vladimir Katkovnik, Nicholas Hogasten, and Karen Egiazarian propose a novel algorithm and its implementation for MWIR cameras:

"A midwave infrared (MWIR) system is simulated showing that this design will produce high quality images even for large amounts of defocus. It is furthermore shown that this technique can be used to design a flat, single optical element, systems where the phase mask performs both the function of focusing and phase modulation."

University of Linz Lensless Camera

University of Linz, Austria, publishes a paper "Thin-film camera using luminescent concentrators and an optical Söller collimator" by Alexander Koppelhuber and Oliver Bimber.

"We discuss optical imaging capabilities and limitations, and present first prototypes and results. Modern 3D laser lithography and deep X-ray lithography support the manufacturing of extremely fine collimator structures that pave the way for flexible and scalable thin-film cameras that are far thinner than 1 mm (including optical imaging and color sensor layers)."

Thanks to TL for the link!

Low Quality LiDARs Restrain Self-Driving Car Progress

MIT Technology Review: Cheaper LiDARs may not deliver the quality of data required for driving at highway speeds:

"At 70 miles per hour, spotting an object at, say, 60 meters out provides two seconds to react. But when traveling at that speed, it can take 100 meters to slow to a stop. A useful range of somewhere closer to 200 meters is a better target to shoot for to make autonomous cars truly safe.

That’s where cost comes in. Even an $8,000 sensor would be a huge problem for any automaker looking to build a self-driving car that a normal person could afford."

Graeme Smith, chief executive of the Oxford University autonomous driving spinoff Oxbotica, told MIT Technology Review that he thinks a trade-off between data quality and affordability in the lidar sector might affect the rate at which high-speed autonomous vehicles take to the roads. Smith thinks that automakers might just have to wait it out for a cheap sensor that offers the resolution required for high-speed driving. “It will be like camera sensors,” he says. “When we first had camera phones, they were kind of basic cameras. And then we got to a certain point where nobody really cared anymore because there was a finite limit to the human eye.”

Image Sensor Companies Genealogy

I've prepared a list of image sensor companies genealogy, with a kind help of EF and DG. As one can understand, nobody's knowledge is complete, so please feel free to add more info and correct mistakes in comments. The link is also available in the left hand side links, next to the image sensor companies list.

Essential Dual Camera Tuning

Essential startup tells what does is take to tune an image processing pipeline for smartphone dual camera (RGB + Monochrome):

"Objective tuning is meant to ensure that each camera module sent to production is operating at an acceptable baseline level. It began with picking the correct golden and limit samples from the factory.

The golden samples are the modules whose characteristics most closely align to the average of our camera and the experience that most of our users will have. Once golden samples were collected, we used them to capture a series of images under various laboratory-controlled test conditions. The images from the golden samples were then used to train the ISP to recognize the unique characteristics of those modules. In other words, we taught the ISP to see the world in a certain way. We also tested other limit and random samples, which have different characteristics that are saved in the factory calibration data, to ensure that they are behaving like the golden samples in those scenes too. The objective tuning process lasted three months. By the end, all of our cameras were responding to the predefined lab scenes in an accurate and predictable fashion.

But even when a camera can repeat actions in a lab, it still needs to be taken into the field— because in real life a camera must be able to take the right picture in millions of different scenarios. Subjective tuning is what makes this possible. It is a painstaking, iterative process—but also one I find incredibly rewarding.

Our subjective tuning process began in January 2017, and during that time, we have gone through 15 major tuning iterations, along with countless smaller tuning patches and bug fixes. We have captured and reviewed more than 20,000 pictures and videos, and are adding more of them to our database every day."

Via: DPReview

AI News: Machine Learning for Stereo Depth Mapping, DNN Processor for Event Driven Sensors

San Francisco-based stealth startup PerceptiveIO publishes an open-access paper "UltraStereo: Efficient Learning-based Matching for Active Stereo Systems" by Sean Ryan Fanello, Julien Valentin, Christoph Rhemann, Adarsh Kowdle, Vladimir Tankovich, Philip Davidson, and Shahram Izadi.

"Mainstream techniques usually take a matching window around a given pixel in the left (or right) image and given epipolar constraints find the most appropriate matching patch in the other image. This requires a great deal of computation to estimate depth for every pixel.

In this paper, we solve this fundamental problem of stereo matching under active illumination using a new learning-based algorithmic framework called UltraStereo. Our core contribution is an unsupervised machine learning algorithm which makes the expensive matching cost computation amenable to O(1) complexity. We show how we can learn a compact and efficient representation that can generalize to different sensors and which does not suffer from interferences when multiple active illuminators are present in the scene. Finally, we show how to cast the proposed algorithm in a PatchMatch Stereo-like framework for propagating matches efficiently across pixels."

ETH Zurich publishes PhD Thesis "Deep Neural Networks and Hardware Systems for Event-driven Data" by Daniel Neil.

"This thesis introduces hardware implementations and algorithms that use inspiration from deep learning and the advantages of event-based sensors to add intelligence to platforms to achieve a new generation of lower-power, faster-response, and more accurate systems."

Qualcomm announces Snapdragon Neural Processing Engine (NPE) SDK running on Kryo CPU, Adreno GPU or Hexagon image processing DSP. Facebook announced plans to integrate the Snapdragon NPE into the camera of the Facebook app to accelerate Caffe2-powered AR features. By utilizing the Snapdragon NPE, Facebook can achieve 5x better performance on the Adreno GPU, compared to a generic CPU implementation.

Deutsche Bank Analysts on Apple 3D Sensing Plans

Deutsche Bank analysis of active alignment (AA) systems supplier ASM Pacific Technology (ASMPT) has interesting info on 3D sensing adoption in future iPhones and dual cameras in future Samsung phones:

"We expect ASMPT’s AA machine sales to grow only 10% YoY in 2018 and stay flat YoY in 2019, after 56% YoY growth in 2017 (Figure 8). Most camera module makers should upgrade their AA machines in 2017. Notably, we believe Apple will not implement 3D sensing for 4.7” and 5.5” iPhones in 2018. This means Apple supply chain will not procure new AA machines for 3D sensing from ASMPT in 2018 (i.e., ASMPT is benefiting from Apple’s adoption of 3D sensing for 5.8” OLED iPhone in 2017).

We estimate camera module makers could upgrade their AA machines every three years due to rapid specs migration in dual cameras for smartphones. This is shorter than the normal duration of five to six years for a CIS (CMOS image sensor) machine. However, ASMPT’s AA business could still see a sales growth deceleration in 2018/19, even assuming a shorter duration of AA machines."

Light Publishes L16 Full Resolution Images

Two weeks after Light L16 computational camera shipments start, there is still no single user review anywhere on the web. However, LightRumors notices that Light Co. has released few full resolution images on its web site. The images are processed using Light’s proprietary software, Lumen, which is powered by Light's proprietary Polar Fusion engine. The engine computationally fuse the many images captured by the L16 to create one high-quality image.

Light Co. also publishes a nice tutorial explaining the L16 camera operation and technology.

Samsung to Allocate Capex for CIS Foundry Business

Samsung Q2 2017 earnings report mentions a capex allocation fro CIS foundry business:

"The Foundry Business is... to allocate sizable capex for converting part of line 11 from DRAM to image sensor production in the second half [of 2017]."

The company also mentions a healthy state of the image sensor sales:

"The System LSI Business increased sales of mobile processors and image sensors.

System LSI Business earnings improved QoQ... Sales of image sensors also contributed to earnings.

Growing market adoption of dual camera solutions will also boost image sensor shipments."

ST Reports Strong FlightSense Sales

ST Micro reports Q2 2017 results. Regarding the imaging business, the company says "As anticipated, Imaging revenues in the second quarter decreased slightly on a sequential basis to $68 million, while we prepare for the ramp of new programs.

On a year-over-year basis, Imaging revenues increased 60% in the second quarter, and for the first half 2017 rose 83% to $140 million driven by ST’s innovative Time-of-Flight technology.

In the second quarter we continued to gain design-wins while delivering high volumes of our “FlightSense” Time-of-Flight proximity and ranging sensors to multiple smartphone OEMs. We now have reached cumulative shipments of over 300 million Time-of-Flight sensors and are in more than 80 smartphone models from 15 different OEMs.

In our Imaging business, we anticipate strong sequential growth, as the key new program ramps in Q3, followed by further revenue acceleration in the fourth quarter of this year."

EETimes speculates that the "key new program ramps in Q3" might mean ToF sensor in Apple iPhone 8.

SeekingAlpha publishes the earnings call transcript with a clarifying question in Q&A session:

Janardan Menon - Liberum Capital Ltd.

And just a brief follow-up on the Time-of-Flight, which is in your other division. After a big jump in the second half of last year, that revenue has sort of flattened out. But you are continuously reporting higher number of models and OEM on that particular product. And now I understand that from the second half, that revenue will increase sharply because of the 3D of the special program.

But just on the Time-of-Flight itself, can you give some reason why that revenue is not really rising as a number of model. Is that price pressure coming there? Or what are the dynamics which is happening there?

Carlo Bozotti - STMicro CEO:

I think on the Time-of-Flight, we have enormous number of customers in our end. Of course, we are also working on new technologies for the Time-of-Flight. So, there would be a new wave, but we are pretty happy that the growth is impressive in Imaging and we are investing a lot for the new initiative. This is visible of course in terms of expenses in the P&L, but we have now sort (47:46) the $300 million business of Time-of-Flight that we want to keep going and we have the opportunity. I think it's pretty good and it's a pretty good business. I would say it's very good business, but in parallel, we are investing on new things and this will make – will allow us to make another important step.

Princeton IR Tech Announces 1.2MP 95fps ITAR-free SWIR Sensor

IMVEurope, Photonics: Prinston Infrared Technologies announces its first InGaAs SWIR camera to fall under the no ITAR restrictions. The 1280SciCam, features a 1,280 x 1,024-pixel image sensor on a 12µm pitch, having long exposure times, low read noise, 14-bit digital output, and full frame rates up to 95Hz. The camera is designed for advanced scientific and astronomy applications, and is now classified by the Export Administration Regulations as EAR 6A003.b.4.a for export.

The US government’s export control has been going through a process of reform, which began in 2009 as part of the Obama Administration's Export Control Reform (ECR) initiative. The technology from Princeton Infrared no longer falls under ITAR control, which is equipment specially designed or modified for military use, but now falls under EAR. This, in theory, makes it easier to export the technology outside the USA.

Bob Struthers, sales director at Princeton Infrared Technologies, says: ‘Our 1280SciCam has already generated sales and applications with leading research entities overseas. An EAR export classification will propel our ability to serve these customers promptly and efficiently. This will be very valuable to their upcoming projects and equally beneficial to the growth of our young company.’

IMVEurope: A year ago, Xenics SWIR cameras have been granted Commodity Jurisdiction (CJ) approval. This new CJ means that all SWIR cameras supplied by Xenics are now ITAR-free in the US.

Pyxalis and Framos Extend Cooperation

Presseagentur: Framos and Pyxalis extend their custom sensor design cooperation. The companies have been cooperating for several years and now have entered into a formal agreement. This partnership provides Framos partners with fully customized, high performance sensors, including sensor specification elaboration support, sensor architecture, design, prototyping, validation, industrialization and manufacturing.

“We’re delighted to work with FRAMOS Technologies in Europe and North America. As a 7-year-old company supplying custom image sensors, we’ve built successful partnerships with customers in many applications from niche markets (aerospace, scientific, defense) to medium volume (industrial, medical) and consumer markets (biometrics, automotive). Thanks to this cooperation with FRAMOS, it is now time to reach a larger market and to provide our capabilities and technologies to a greater number of customers.” says Philippe Rommeveaux, PYXALIS’s President and CEO.

HDPYX Customized Sensor

EI Image Sensors and Imaging Systems 2017 Papers in Open Access

EI Symposium Image Sensors and Imaging Systems 2017 papers are published in open access. There is quite a lot of good papers:

• Accurate Joint Geometric Camera Calibration of Visible and Far-Infrared Cameras
  Authors: Shibata, Takashi; Tanaka, Masayuki; Okutomi, Masatoshi
• High Sensitivity and High Readout Speed Electron Beam Detector using Steep pn Junction Si diode for Low Acceleration Voltage
  Authors: Koda, Yasumasa; Kuroda, Rihito; Hara, Masaya; Tsunoda, Hiroyuki; Sugawa, Shigetoshi
• A full-resolution 8K single-chip portable camera system
  Authors: Nakamura, Tomohiro; Yamasaki, Takahiro; Funatsu, Ryohei; Shimamoto, Hiroshi
• Filter Selection for Multispectral Imaging Optimizing Spectral, Colorimetric and Image Quality
  Authors: Wang, Yixuan; Berns, Roy S.
• The challenge of shot-noise limited speckle patterns statistical analysis
  Authors: Tualle, J.-M.; Barjean, K.; Tinet, E.; Ettori, D.
• Hot Pixel Behavior as Pixel Size Reduces to 1 micron
  Authors: Chapman, Glenn H.; Thomas, Rahul; Koren, Israel; Koren, Zahava
• Octagonal CMOS Image Sensor for Endoscopic Applications
  Authors: Wäny, Martin; Santos, Pedro; Reis, Elena G.; Andrade, Alice; Sousa, Ricardo M.; Sousa, L. Natércia
• Optimization of CMOS Image Sensor Utilizing Variable Temporal Multi-Sampling Partial Transfer Technique to Achieve Full-frame High Dynamic Range with Superior Low Light and Stop Motion Capability
  Kabir, Salman; Smith, Craig; Armstrong, Frank; Barnard, Gerrit; Guidash, Michael; Vogelsang, Thomas; Endsley, Jay
• A Lateral Electric Field charge Modulator with Bipolar-gates for Time-resolved Imaging
  Authors: Morikawa, Yuki; Yasutomi, Keita; Imanishi, Shoma; Takasawa, Taishi; Kagawa, Keiichiro; Teranishi, Nobukazu; Kawahito, Shoji
• A 128x128, 34μm pitch, 8.9mW, 190mK NETD, TECless Uncooled IR bolometer image sensor with column-wise processing
  Authors: Alacoque, Laurent; Martin, Sébastien; Rabaud, Wilfried; Beigné, Edith; Dupret, Antoine; Dupont, Bertrand
• Residual Bulk Image Characterization using Photon Transfer Techniques
  Author: Crisp, Richard
• RTS and photon shot noise reduction based on maximum likelihood estimate with multi-aperture optics and semi-photon-counting-level CMOS image sensors
  Authors: Ishida, Haruki; Kagawa, Keiichiro; Seo, Min-Woong; Komuro, Takashi; Zhang, Bo; Takasawa, Taishi; Yasutomi, Keita; Kawahito, Shoji
• Linearity analysis of a CMOS image sensor
  Authors: Wang, Fei; Theuwissen, Albert
• Fast, Low-Complex, Non-Contact Motion Encoder based on the NSIP Concept
  Authors: Anders, Åström; Robert, Forchheimer
• In the quest of vision-sensors-on-chip: Pre-processing sensors for data reduction
  Authors: Rodríguez-Vázquez, A.; Carmona-Galán, R.; Fernández-Berni, J.; Brea, V.; Leñero-Bardallo, J.A.

TechInsights Reviews Pixel Isolation Structures

TechInsights keeps publishing parts from Ray Fontaine's presentation at IISW 2017. The third part reviews modern pixel-to-pixel crosstalk reduction measures: Front-DTI and Back-DTI:

Sony dielectric-filled B-DTI structure from the 1.4 µm pixel featuring a 2.9 µm thick substrate extends to a depth of 1.9 µm from the back surface, although it extends to a depth of 2.4 µm deep at B-DTI intersections:

Samsung 1.12 µm pixel generation B-DTI trenches extend 1.3 µm deep into a 2.6 µm deep substrate:

Omnivision 1.0 µm pixel B-DTI extends 0.45 µm deep into the back surface of a 2.5 µm thick substrate:

DVS Camera for Drones

Zurich University spin-off and event-driven sensor patents licensee Insightness presents its camera for drone navigation and obstacle avoidance:

Sony Unveils Variable-Speed Global Shutter Sensor

Sony publishes a flyer of IMX428LLJ/LQJ monochrome global shutter sensor featuring "variable-speed shutter function (resolution 1 H units)":

Update: There is also a faster version IMX420LLJ/LQJ achieving 200fps at 8b resolution:

Videos from AutoSens Detroit Demo Sessions

AutoSens publishes a number of short videos from its Detroit conference held in May 2017:

Why Use SWIR?

Photonics publishes Sensors Unlimited Doug Malchow presentation on SWIR band advantages:

Forza Compares CIS Foundries and Their Offerings

Forza Silicon's President & Co-Founder, Barmak Mansoorian, compares different image sensor foundries and processes in this video:

Event-based Vision Workshop Materials On-Line

It came to my attention that the International Workshop on Event-based Vision at ICRA'17 has been held on June 2, 2017 in Singapore. The workshop materials are kindly made available on-line, including pdf presentations and videos.

The Workshop organizers have also created a very good Github-hosted list of Event Based Vision Resources.

Chronocam, ETH Zurich, Samsung are among the presenters of event driven cameras:





ETH Zurich and University of Zurich also announces Misha Award for the achievements in Neuromorphic Imaging. The 2017 Award goes to "Event-based Vision for Automomous High Speed Robotics" work by Guillermo Gallego, Elias Muggler, Henry Rebecq, Timo HorstSchafer, and Davide Scaramuzza from University of Zurich, Switzerland.

Thanks to TD and GG for the info!

Isorg and FlexEnable Win Award for Flexible Image Sensor

ALA News: Isorg announces that its first large-sized high-resolution (500 dpi) flexible plastic fingerprint sensor, co-developed with FlexEnable (former Plastic Logic), won the 2017 Best of Sensors Expo - Silver Applications Award.

The high-resolution, ultra-thin, 500 dpi flexible image sensor (sensitive from visible to near infrared) has unique advantages in performance and compactness. Its ability to conform to three-dimensional shapes sets it apart from conventional image sensors. The device provides dual detection: fingerprinting as well as vein matching. Due to its large-area sensing and high-resolution image quality, the device is suited to biometric applications from fingerprint scanners and smartcards to mobile phones, where accuracy and robustness as well as cost-competiveness are key.

Designed on a large area (3” x 3.2”; 7.62 x 8.13cm) plastic substrate, the flexible image sensor is ultra-thin (300 microns), therefore remarkably lightweight, compact and highly resistant to shock. Central to the 500 dpi flexible image sensor is an Organic Photodiode (OPD), a printed structure developed by Isorg that converts light into current – responsible for capturing the fingerprint. Isorg also developed the readout electronics, the forensics quality processing software and the optics to enable seamless integration in products. FlexEnable, the leader in developing and industrializing flexible organic electronics, developed the Organic TFT backplane technology, an alternative to amorphous silicon. This partnership between the two companies began in Q4 2013.

Yole IR Imaging Forum

Yole Developpement 2nd IR Imaging Forum to be held on Sept. 7 in Shenzhen, China, publishes its agenda:

• Uncooled IR Imaging Market Perspectives
  Eric Mounier, Senior Analyst, Yole Développement
• State of the art of High End Thermal Image Sensors performances in mass production
  Sebastien Tinnes, Marketing Manager, ULIS
• The Status and Challenges of Thermal Imaging in Security Applications
  Guo Haixun, Product Director of Thermal Imaging, Hikvision
• Progress on low cost Thermopile Arrays for high volume applications – eg. office automation, person detection and thermal imaging
  Joerg Schieferdecker, CEO and Co-Founder, Heimann Sensors
• New ultra-compact infrared cameras with 500 nm spectral response for metal industry
  Torsten Czech, Head of Product Management, Optris
• Uncooled Infrared Imaging System for Forest Fire Detection and Monitoring
  Wang You, Uncooled Infrared Imaging Senior Expert, JIR Infrared
• Ion Beam Deposition of VOx films for uncooled bolometer and thermal sensor applications
  David I C Pearson, Ion Beam Senior Technologist, Oxford Instruments Plasma Technology
• Modern Assembly technology for Packaging of IR Microbolometers
  Alex Voronel, Director of Global Sales, SST Vacuum Reflow Systems
• Prospect of commercial chalcogenide glasses used for uncooled infrared imaging system
  Rongping Wang, Senior Fellow, The Australian National University
• MOEMS components with subwavelength structures for hyperspectral imaging
  Steffen Kurth, Department manager, Fraunhofer Institute for Electronic Nano Systems (ENAS)

Image Sensors America Agenda

IS America conference to be held on October 12-13, 2017 in San Francisco has published its agenda:

• Keynote Presentation: Lifestyle Image Sensor Requirements
  Farhad Abed, Image quality engineer of GoPro
• Image Sensor Venture and M&A Activity: An Overview of Recent Deals, Trends, And Developments
  Rudy Berger, Managing Partner of Woodside Capital Partners
• Image Quality Oriented Sensor Characterization
  Zhenhua Lai, Imaging Optics System Engineer of Motorola Mobility
• A New Frontier in Optical Design: Segmented Optics Combined with Computational Imaging Algorithms
  Dmitry V. Shmunk, CTO of Almalence Inc
• IR Bolometer Technology
  Patrick Robert, Electronic Design Manager of ULIS
• Global Shutter vs. Rolling Shutter: Performance And Architecture Trade Off
  Abhay Rai, Director of product marketing of Sony Electronics
• Enhancing the Spectral Sensitivity of Standard Silicon-based Imaging Detectors
  Zoran Ninkov, Professor in the Center for Imaging Science (CIS) of Rochester Institute of Technology
• TDI Imaging Using CCD-in-CMOS Technology: An Optimal Solution for Earth Observation, Industrial Inspection and Life Sciences Applications
  Arye Lipman, Strategic Alliances Manager of Imec
• Semiconductor Sequencing Technology: A Scalable, Low-Cost Approach to Using Integrated CMSOS Sensor Arrays
  Brian Goldstein, Sr. Staff Engineer in Sensor Design Engineering in the Clinical Next-Generation Sequencing Division of Thermo Fisher Scientific
• Photon-to-Photon CMOS Imager: Optoelectronic 3D Integration
  Gaozhan Cai, Design team leader, focusing on designing custom CMOS image sensors of Caeleste
• Going Beyond 2x Optical Zoom In Dual Cameras: The Future of Dual Camera Technology
  Gal Shabtay, GM and VP R&D of Corephotonics
• Image Sensors for the Endoscopy Market: Customer Needs and Innovation Opportunities
  Dave Shafer, Managing Fellow of Intuitive Surgical
• Will Your Next Sensor Assist in Replacing Your Job?
  Yair Siegel, Director of Strategic Marketing of CEVA
• Enabling Always –On Machine Vision
  Evgeni Gousev, Senior Director of Qualcomm Technologies Inc.
• PanomorphEYE Human Sight Sensor For Artificial Intelligence Revolution
  Patrice Roulet, Director of Engineering and Co-Founder of Technology of Immervision
• High-Speed Imaging: Core Technologies and Devices Achieved
  Takashi Watanabe, Developer of log-type imagers and range image sensors of Brookman Technology, Inc.
• Tools and Processes Needed to De-risk the Design-In of Image Sensors
  Simon Che’Rose, Head of Engineering of FRAMOS
• Single Module Solution for Depth Mapping
  TBD
• Image Sensor Requirements for 3D Cameras
  Rich Hicks, Senior Camera and Imaging Technologist of Intel, Global Supply Management
• Laser Diode Solutions for 3D Depth Sensing LiDAR Systems
  Tomoko Ohtsuki, Product Line Manager, Lumentum
• A Comparison Of Depth Sensing Solutions For Image Sensors, LiDAR And Beyond
  Scott Johnson, Director of Technology Business Alignment of ON Semiconductor

ISSCC 2017 Plenary on High-Speed DNA Sequencing

High-Speed DNA Sequencing is an emerging application for image sensor and sister devices (such as ion sensors, pH sensors, etc.). The ion sensor part starts at about 15:00 time in this ISSCC 2017 plenary session video by Jonathan Rothberg, Yale University:

ICFO Graphene Image Sensor Video

Circuit Cellar publishes a nice interview with Stijn Goossens, one of ICFO developers of graphene image sensor announced in May:

Mobile Phone Food Analysis

Open-source Sensors journal publishes a paper "Smartphone-Based Food Diagnostic Technologies: A Review" by Giovanni Rateni, Paolo Dario, and Filippo Cavallo from BioRobotics Institute, Italy. Smartphone with image sensor turns to be quite a versatile platform:

CIS History Diagram

Techbriefs magazine publishes an article "CMOS, The Future of Image Sensor Technology" by Gareth Power, Marketing Manager, Teledyne e2v. The main trends in industrial and scientific sensors are said to be higher speeds and lower prices. There is also a diagram on image sensor companies spin-offs and mergers:

Some parts are not exactly correct here, like Avago has not been spun-off from Micron. Also, Far Eastern companies are not there, like no Toshiba-Sony, nor Siliconfile-Hynix, nor others. But as a first attempt to make such a diagram, it looks really nice.

Thanks to LH for the link!

Optimal Coding Functions for I-ToF Imaging

University of Wisconsin-Madison and Columbia University publish a technical report "What Are Optimal Coding Functions for Time-of-Flight Imaging?" by Mohit Gupta, Andreas Velten, Shree Nayar, and Eric Breitach.

"Almost all current C-ToF systems use sinusoid or square coding functions, resulting in a limited depth resolution. In this paper, we present a mathematical framework for exploring and characterizing the space of C-ToF coding functions in a geometrically intuitive space. Using this framework, we design families of novel coding functions that are based on Hamiltonian cycles on hypercube graphs. Given a fixed total source power and acquisition time, the new Hamiltonian coding scheme can achieve up to an order of magnitude higher resolution as compared to the current state-of-the art methods, especially in low SNR settings."

The "geometrically intuitive hypercube graphs" look like this:

Caeleste Gets New CEO

Photonics: Caeleste has appointed Geert De Peuter as its new CEO. Geert De Peuter spent much of his career at Alcatel, now Nokia Bell Labs.

Light Co. Starts Shipping its Computational Camera

Light starts shipping the L16 camera to its pre-order customers, after 4 years of development. The company explains what it has accomplished in these 4 years:

• It took us three years to design and build our own custom ASIC chips, which are needed to control all 16 camera modules at the same time.
• We also developed our own 70mm and 150mm camera modules, complete with custom optics and electrical components. To put this in perspective, most smartphone cameras contain 30mm or 50mm lenses. The higher focal length lenses we were looking for weren’t even on the market yet, so we had to invent them ourselves.
• We created proprietary image-fusing algorithms and processing pipelines that align each of the base camera modules.
• We produced Android software to operate our camera and a Mac/Windows application for depth-of-field editing.
• We implemented an e-commerce platform and initiated a complex global manufacturing and supply chain.

Light Director of Hardware Engineering, Brian Gilbert,
with the first 'lunch box' prototype

Light final product. Each lens is annotated
with its range of distances, focal length, and aperture

TSMC Wafer Bonding Applications

TSMC patent application US20170186798 "Stacked SPAD image sensor" by Ming-hsien Yang, Ching-chun Wang, Dun-nian Yaung, Feng-chi Hung, Shyh-fann Ting, and Chun-yuan Chen is said to improve SPAD pixel fill factor:

TSMC patent application US20170186796 "Frontside illuminated (FSI) image sensor with a reflector" by Min-feng Kao, Dun-nian Yaung, Jen-cheng Liu, Jeng-shyan Lin, Hsun-ying Huang, and Tzu-hsuan Hsu proposes wafer bonding to add a reflector 102 under the PD 104 to improve FSI pixel QE:

5T Pixel in SPICE

International Journal of Advanced Research in Computer and Communication Engineering publishes a paper "Optimized Design of Active Pixel Sensor using CMOS 180 nm Technology" by Dipti, Rajesh Mehra, and Deep Sehgal.

Contrary to the whole industry spending a lot of money on expensive device simulators, the authors simulate everything in SPICE:

The proposed timing sequence works well, just does not perform CDS and the small signals do not go through, not to talk about extra sensitivity to FD leakage:

Column-Parallel ADC Theses

Middle East Technical University, Ankara, Turkey, publishes MSc thesis "Column Level Two-Step Multi-Slope Analog to Digital Converter for CMOS Image Sensors" by Can Tunca.

The design is realized for pixel pitch of 6.7µm. Power consumption per column [12 bit] ADC is 88 µW and sampling speeds larger than 50kS/s is supported.

The outline of the operation of the Two-Step Integrating ADC is explained below and an example conversion sequence is illustrated in Figure 3.1.

• In the first step, K-bit coarse conversion is performed using a ladder shaped ramp. When the decision is made, the ramp value is latched into a memory capacitor. Furthermore, the global counter value latched in the digital coarse memory block.
• Secondly the residue between the latched ramp value and the input voltage is compared to the fine ramp in the L-bit fine conversion phase. Likewise, when the decision is made global counter value is latched into the digital fine memory block.
• On the final step, coarse and fine conversion results are superimposed and fed to the output stage.

Milano Politecnico, Italy, publishes MSc thesis "Sigma-Delta Analogue-to-Digital converter for column-parallel CMOS image sensors" by Michele Sannino.

In this master’s thesis project a column-parallel ADC for high data-rate image sensors was designed using TowerJazz 0.18µm process.

The ADC was required to achieve 12 bits of resolution in the competitive conversion time of 1us. Other design specifications include a constraint on the maximum input noise, which had to be less than 100uVrms, and on the average power consumption, to be contained within 330uW. The converter, which was laid out in a column-parallel topology with 15um pitch, was also required to occupy an area smaller than 10,000um2 (hence its length should be smaller than 670um). Meeting this specification makes the ADC suitable to be implemented in a stacked chip in future developments, which would push further the limit of achievableframe-rate.

Future of Mobile Imaging

Engadget publishes an article "The future of the smartphone camera: where next?" There are no big revelations in the list of innovations:

• Augmented reality
• Dual-lens cameras
• Better lenses
• 4K recording
• Thermal imaging
• Optical zoom
• 360 video

Meanwhile, Strategy Analytics sees mobile imaging future in 3D cameras:

"Sales of 3D Imaging smartphones are poised to take off. Advanced security and Augmented Reality (AR) solutions will be the main drivers. We predict that 3D Imaging will see over 1700% growth during next six years and will become one of the key differentiators in higher-end smartphones."

On the other hand, analysts do not give a significance to other stuff:

• No interest in Visible Light Communication (VLC, LiFi)
• No higher resolution push. 41MP in old Nokia phones remains the world's record forever
• No high speed imaging future. Sony Xperia 960fps camera does not attract analysts attention

HDR Pixel Thesis

Glasgow University, UK, publishes a PhD thesis "High dynamic range image sensor using tone mapping operation" by Waqas Mughal. Here is the HDR pixel principle:

"WDR capture can be performed by introducing a monotonically increasing reference signal Vref. It is possible to capture high intensity information by comparing the integrated voltage at node N to a reference voltage Vref.

The pixel output follows a known reference signal, which is sampled and held at a value when the photo-generated signal on the diode becomes lower than the reference voltage. The potential at which these two signals are equal is recorded and is used as the pixel’s response. In the pixel, M1, M2, M3 and M4 are reset device, reference voltage switch, source follower and row select switch."

The Vref sweep function can be used for the tone mapping, once the pixel FPN issues are solved:

SK Hynix 8-inch CIS Foundry Officially Starts as a Separate Business

Korea Herald: SK Hynix foundry (SK Hynix System IC) starts operating as a separate entity, offering a "cost-effective CIS process":

Mobile Imaging Report from China

China Galaxy International publishes it analysis on mobile imaging industry in China.

"Currently, Sony dominates the smartphone CMOS image sensor market with over a 35% market share in 2016, followed by Samsung with a 19% market share and Omnivision with a 12% market share.

The lens market is dominated by Largan Precision [3008.TT], Sunny Optical [2382.HK] and Genius Electronic Optical [3406.TT]. The total market share of these three companies was 53.3% in 2015. Currently, most smartphone cameras are equipped with 6P lenses. Only Largan Precision can achieve a decent defect-free rate (over 70%) and has enough capacity to meet demand.

The percentage of VCMs used in smartphones increased from 62.4% in 2013 to 76.6% in 2016. The VCM market is dominated by Japanese and Korean companies, which have an aggregate 60%-70% market share. VCM can also be used in many other areas, such as VR/AR, drones and medical equipment.

The five largest single-cameras module providers are Sunny Optical, O-Film Tech [002456.SZ], Hon Hai Precision [2354.TT], Cowelle Holdings [1415.HK] and Samsung Electro-Mechanics, which have a market share of 8.9%, 8.7%. 5%, 4.7% and 4.5%, respectively. The dual-camera module market is dominated by three companies: LG, Sunny Optical and O-Film Tech, which have an approximate aggregate market share of 83%. We believe there is large potential for domestic dual-camera module manufacturers to grow since the dual-camera design is getting more and more popular, and the gross margin is of dual-cameras is about 2% higher than that of single-camera modules."

The major trends have been identified:

"From Single- to Dual-Camera: The dual-camera trend is now well-established in the smartphone industry, and dual-cameras can capture sharper images with more details than single-cameras can. Over 19 dual-camera equipped smartphones were released last year and 14 new dual-camera smartphones were introduced in the first four months of this year. The penetration rate of dual-cameras was only 5.6% in 2016, but it is expected to reach 15% this year, according to Sunrise Big Data. Generally, there four types of dual-cameras: a) Bayer + Mono (Huawei P9); b) wide + tele (iPhone 7 Plus); c) symmetrical (Huawei Honor 6 Plus); and d) asymmetrical (Xiaomi Redmi Pro).

At the current stage, 3D cameras on smartphones are used mostly in static conditions (e.g. facial recognition) and do not require a high frame rate, so structured light is the better choice for smartphone manufacturers. The cost of a structured light equipment is approximately US$20. The projector accounts about 50% of the total cost, or about US$10, algorithm chips cost US$4-$6, or 25%-30%, and receivers cost US$5-$6, or 20%-30%. Apple Inc. [AAPL.US] acquired PrimeSense, a 3D sensor company, for US$350m-$360m in 2013. PrimeSense is one of the major structured light players, and it provided research and related support for the development of Microsoft’s Kinect. As a result, we believe there is a strong possibility the new iPhone will use structured light, and Android smartphone companies will closely follow this trend. In Q4 2016, 432m smartphones were sold globally, of which 352m ran Android (81.7%) and 77m ran iOS (17.9%). We believe the increasing application of 3D sensing technology in Android smartphones will further boost the growth of the 3D camera industry.

360 degree fisheye panoramic camera: Huawei cooperated with Insta360 to announce the Honor VR 360-degree camera in February 2017, which allows users to take high-resolution 360-degree videos and photos. The VR camera has two 210-degree fisheye cameras, offering a seamless livestreaming experience. ProTruly [600074.CH], which is a lesser-known brand in China, presented the world’s first VR smartphone at Mobile World Congress 2017."

ST FlightSense Presentation

ST presentation at IoT World in May 2017 unveils a number of its future ToF products: