Qualcomm Snapdragon 855 Enhances Camera Features

Qualcomm announces its 5G-ready 7nm Snapdragon 855 platform featuring an enhanced imaging functionality:

"Snapdragon 855 sets new standards for capturing stunning photos and videos. The new Qualcomm Spectra™ 380 ISP integrates numerous hardware accelerated computer vision (CV) capabilities, enabling the world’s first announced CV-ISP to provide cutting-edge computational photography and video capture features while at the same time offering up to 4x power savings. The CV-ISP includes hardware-based depth sensing which enables video capture, object classification, and object segmentation all in real-time in 4K HDR at 60fps. This means a user can capture a video and accurately replace selected objects or backgrounds in the scene in real-time all with 4K HDR resolution using over 1 billion shades of color. Even further, the new Qualcomm Spectra 380 ISP is the first announced ISP to support video recording using HDR10+, so the more than 1 billion shades of color can be captured with exceptional contrast and visual brilliance. Finally, in order to efficiently store this amazing content, Snapdragon 855 adds hardware acceleration for HEIF file format encoding, reducing file sizes by 50% for efficiently saving and sending user generated content."

Qualcomm Spectra 380 Image Signal Processor:

• Dual 14-bit CV-ISPs; 22MP @30 fps concurrent dual cameras; 48MP @ 30 fps single camera
• Hardware CV functions including object detection & tracking (Histogram of Oriented Gradients, Harris Corner Detection, Normalized Cross Correlation, Linear classification and optical flow) and stereo depth processing
• Advanced HDR solution including improved zzHDR and 3-exposure Quad Color Filter Array (QCFA) HDR
• 4K60 HDR video capture (HDR10, HDR10+ and HLG) with Portrait Mode (bokeh), 10-bit color depth and Rec. 2020 color gamut
• Hardware-based Multi-Frame Noise Reduction (MFNR) for snapshot and Motion Compensated Temporal noise Filtering (MCTF) for video
• Hardware-based Electronic Image Stabilization (EIS) solution within camera subsystem
• A new modular ISP design with more flexibility to tap in and out of the imaging pipeline both in the RAW and YUV pixel domains
• High frame rate capture for slow motion video (720p @ 480fps)
• HEIF photo capture, HEVC (H.265) video capture

Update: Qualcomm publishes its SD855 camera presentation. Few additional slides from it:

Unconstrained Machine Learning Search Confirms that Bayer CFA is Optimal

Occipital HW Leader Evan Fletcher did an interesting work on searching for an optimal CFA pattern among arbitrary large sized patterns in an unconstrained machine learning search. The result might surprise the companies proposing better CFA mosaics, such as Fujifilm:

"After training for ~24 hours, the learned color filter array looks quite familiar:

The network seems to have learned something very similar to a RGGB Bayer pattern, complete with 2×2 repetition and the pentile arrangement of the green pixels! This was quite surprising, especially given that there is no spatial constraint on arrangement or repetition in this network design whatsoever.

This optimization appears to have independently confirmed the venerable Bayer pattern as a good choice for a mosaic function – it was fascinating to see a familiar pattern arise from an unconstrained optimization."

Apple and Samsung to Integrate ToF Cameras in their 2019 Phones

PhoneArena quotes WSJ and Korean media claiming that next year Samsung Galaxy 5G S10 smartphone is expected to have a rear ToF camera. Later in the year, Samsung A-series smartphones will feature ToF cameras too. Samsung ToF module suppliers are said to be SEMCO and Patron.

Next year's Apple iPhone is to feature a rear ToF camera as well, said to be sourced from LG.

Huawei Smartphone to use Sony ToF Sensor for 3D Imaging

Bloomberg reports that an oncoming Huawei smartphone possibly incorporate Sony ToF sensor:

"The phone, code-named Princeton internally, will be announced this month and go on sale within a few weeks, according to one of the people who requested anonymity discussing private plans. The technology uses sensors developed by Sony Corp. that can accurately measure distances by bouncing light off surfaces.

Besides generating pictures that can be viewed from numerous angles, Huawei’s new camera can create 3-D models of people and objects that can be used by augmented-reality apps, according to one of the people. The new camera will also let developers control apps and games in new ways, such as hand gestures, the person said, who added that some of the details may change as developers work with the technology."

Gpixel Unveils Full-Frame 51MP 80fps Global Shutter Sensor

Gpixel announces a 51MP, 8424 x 6032 resolution, 35 mm full-frame global shutter image sensor. The GMAX4651 is capable of capturing full resolution images at frame rates of up to 80 fps in standard 12-bit read out mode and 40 fps in dual gain HDR mode.

With more than 83dB DR, 1/50,000 shutter efficiency and wide angular response, the GMAX4651 is aimed to advanced optical inspection and machine vision applications (Automated Optical Inspection (AOI) and Flat Panel Display (FPD) inspection) as well as high-end 8k/4k video broadcast and aerial imaging applications.

“The GMAX4651 is a really unique addition to our GMAX series, outperforming existing CCD solutions on all performance levels,” commented Wim Wuyts, Chief Commercial Officer of Gpixel. “It combines the dual gain HDR used in our successful GSENSE sensors, with excellent global shutter pixel technology and unprecedented frame rates. All together this will set a new industry standard for high-end industrial inspection and the video industry.”

Based on a 4.6 µm charge domain global shutter technology with true CDS, the GMAX4651 delivers more than 25 ke- linear FWC and up to 50 ke- when using 1x2 binning and ultra-low noise read out down to 1.5 e-RMS. Under dual gain HDR mode, where the sensor reads out the same exposure with two different gain settings for off-chip HDR reconstruction, DR can reach 83dB.

GMAX4651 mono sensor sample and evaluation kit will be commercially available in March 2019. Color samples will be made available in Q2 2019 and volume production is targeted for Q3 2019.

Sheba Microsystems Promises MEMS AF, Zoom, and OIS

Toronto, Canada-based startup Sheba Microsystems promises MEMS-based AF, zoom, and OIS:





Some demo videos are published in the company's Youtube channel.

Event-Driven Sensor in Structured Light 3D Camera

Arxiv.org paper "Event-Based Structured Light for Depth Reconstruction using Frequency Tagged Light Patterns" by T. Leroux, S.-H. Ieng, and R. Benosman from University of Pittsburgh, Carnegie Mellon University, and Sorbonne University proposes to use (apparently) Prophesee sensor for a structured light 3D camera. Although the paper appears a bit rushed to publication and needs some proofreading, the presented ideas look nice:

"This paper presents a new method for 3D depth estimation using the output of an asynchronous time driven image sensor. In association with a high speed Digital Light Processing projection system, our method achieves real-time reconstruction of 3D points cloud, up to several hundreds of hertz. Unlike state of the art methodology, we introduce a method that relies on the use of frequency tagged light pattern that make use of the high temporal resolution of event based sensors. This approach eases matching as each pattern unique frequency allow for any easy matching between displayed patterns and the event based sensor. Results are shown on the real scenes."

Orbbec 3D Depth Sensing in Oppo Find X

SystemPlus publishes a teardown report of Orbbec face recognition solution in Oppo Find X smartphone:

"Located in the Find X’s front side, around the speaker, the 3D system is packaged in one metal enclosure. The system features a dot projector, a flood illuminator, a red/green/blue camera and a NIR camera sensor. An additional SoC component is soldered on the main board to process the signal from the latter devices.

The system uses standard components found on the market. That includes a GS image sensor featuring 3µm size pixels and standard resolution of 1 megapixel and a vertical cavity surface emitting laser (VCSEL). The system is therefore very cost efficient compared to other solutions. The camera and dot projector module assembly uses standard wire bonding to connect the sensor or the VCSEL dies, along with an optical module comprising four lenses for both modules."

ST Officially Announces 2-Memories GS Sensors for in-Cabin Cameras

After few months of presentations at different conferences and exhibitions, STMicroelectronics has officially unveiled two new automotive global shutter image sensors, the VG5661 and VG5761, aimed to driver monitoring.

ST’s VG5661 and VG5761 are 1.6MP and 2.3MP automotive global shutter sensors. ST has created a unique automotive global shutter pixel with two memory cells while keeping a small pixel size of 3.2μm. The two memory zones allow double-image storage, enabling linear HDR up to 98dB or background removal without lag effects and with no need for additional processing by the host system. This offloads the host processor and reduces instances of correction artefacts. In addition, HDR and high MTF in NIR further minimizes interference from natural light sources.

The VG5661 and VG5761 are supplied in standard BGA packages, or as bare die for direct integration in automotive OEM systems produced in high volumes. They are qualified to AEC-Q100 grade 2 and include complex safety-integrity features as required for an ASIL-B camera system in accordance with the automotive safety standard ISO 26262.

Sony Polarsens Sensors

Lucid Vision Labs presentation has some data on Sony polarization sensors: