Panasonic publishes an arxiv.org paper "Smart Home Appliances: Chat with Your Fridge" by Denis Gudovskiy, Gyuri Han, Takuya Yamaguchi, and Sotaro Tsukizawa proposing an AI-equipped camera in a fridge:
"Current home appliances are capable to execute a limited number of voice commands such as turning devices on or off, adjusting music volume or light conditions. Recent progress in machine reasoning gives an opportunity to develop new types of conversational user interfaces for home appliances. In this paper, we apply state-of-the-art visual reasoning model and demonstrate that it is feasible to ask a smart fridge about its contents and various properties of the food with close-to-natural conversation experience. Our visual reasoning model answers user questions about existence, count, category and freshness of each product by analyzing photos made by the image sensor inside the smart fridge. Users may chat with their fridge using off-the-shelf phone messenger while being away from home, for example, when shopping in the supermarket. We generate a visually realistic synthetic dataset to train machine learning reasoning model that achieves 95% answer accuracy on test data. We present the results of initial user tests and discuss how we modify distribution of generated questions for model training based on human-in-the-loop guidance. We open source code for the whole system including dataset generation, reasoning model and demonstration scripts."
Sony IEDM paper "High-definition Visible-SWIR InGaAs Image Sensor using Cu-Cu Bonding of III-V to Silicon Wafer" by S. Manda, R. Matsumoto, S. Saito, S. Maruyama, H. Minari, T. Hirano, T. Takachi, N. Fujii, Y. Yamamoto, Y. Zaizen, T. Hirano, and H. Iwamoto describes a process of bonding small InGaAs dies onto a Si wafer:
"We developed a back-illuminated InGaAs image sensor with 1280x1040 pixels at 5-um pitch by using Cu-Cu hybridization connecting different materials, a III-V InGaAs/InP of photodiode array (PDA), and a silicon readout integrated circuit (ROIC). A new process architecture using an InGaAs/InP dies-to-silicon wafer and Cu-Cu bonding was established for high productivity and pixel-pitch scaling. We achieved low dark current and high sensitivity for wavelengths ranging from visible to short-wavelength infrared (SWIR)."
"A new methodology is presented using well known electrical characterization techniques on dedicated single devices in order to investigate backside interface contribution to the measured pixel dark current in BSI CMOS image sensors technologies. Extractions of interface states and charges within the dielectric densities are achieved. The results show that, in our case, the density of state is not directly the source of dark current excursions. The quality of the passivation of the backside interface appears to be the key factor. Thanks to the presented new test structures, it has been demonstrated that the backside interface contribution to dark current can be investigated separately from other sources of dark current, such as the frontside interface, DTI (deep trench isolation), etc."
"With these MOS capacitor and W-shield gate transistor test structures, it is possible to electrically characterize the backside interface of BSI technology at the end of a process using a tungsten shield. By means of two known characterization methods, Dit and NEFF, which are the two important parameters for dark current, can be extracted. It is, therefore, possible to investigate if the dark current mainly comes from the backside interface, and to discriminate the origin of the backside dark current.
In the case presented in this study, the difference in Idark behavior is explained by quality passivation differences of the backside interface between wafers. COCOS measurements are useful to characterize the interface just after a material deposit, however, it cannot be used with a fully processed wafer, unlike the methodology used on the new structures presented in this study. A drawback of this method is the presence of a charging effect that forces some caution on the execution of measurements, but this effect can be recovered and is not present in pixel operating conditions. In addition to these Idark contribution studies, these dedicated devices with associated characterizations can be helpful for process monitoring, TCAD calibration, and reliability works."
PRNewswire: OmniVision announces the OS12D40, a 1.4um pixel, 11.3MP image sensor with on-chip remosaic (4-cell to Bayer) color converter and on-chip HDR processing. When in full-HD 1080p mode, this sensor's 3-exposure HDR with on-chip combination and tone mapping provides best in class video captures. This is superior to the competing method, known as staggered HDR, which relies on additional passes that introduce motion artifacts, especially in low light. Additionally, OmniVision's PureCel Plus-S stacked architecture enables each pixel to perform optimally to further improve HDR in scenes with widely contrasting bright and dark areas.
"With this new image sensor, we're setting the standard for best in class, mass market security camera performance," said David Shin, product marketing manager for the security segment at OmniVision. "This means both commercial and home security systems will now be able to better capture moving objects across all lighting conditions in full-HD 1080p mode, while using artificial intelligence (AI) or human operators to selectively take 4K2K images without HDR. The latter is important when the need for greater detail is identified, such as capturing an intruder's facial features or reading a car's license plate number. Additionally, we achieve 2.8 micron-equivalent pixel performance using 4-cell binning, to provide excellent low light image quality in 1080p mode."
Industry analysts predict that the security and surveillance camera market is growing at a more than 15% CAGR, and will exceed 400M units in 2024.
Other features include a large 1/2.49" optical format, 9 degree CRA, a 10b ADC and a 4-lane MIPI transceiver (2.5 Gbps/lane).
Integrated selective conversion gain technology allows the pixel conversion gain to be dynamically switched between low and high, depending on the scene being captured in combination with the sensor's other features, including PureCel Plus-S stacked pixel technology for reduced crosstalk and maximum QE.
The OS12D40 uses a 4-cell color filter pattern. It has an on-chip 4-cell to Bayer remosaic converter, in order to provide 4K video at 60fps with 20% additional pixels for EIS. In a 4-cell binned mode, it can output an impressive 2.8MP/1080p resolution with 20% additional pixels for EIS video and images at four times the sensitivity. This sensor also supports both CPHY and DPHY interfaces, and can output 11.3MP, 4512x2512 16:9 captures at 60fps, 4K video at 60fps and 1080p video at 240fps.
OS12D40 samples are available now in a fan-out and chip-scale wafer level package.
PRNewswire: OmniVision announces the expansion of its BSI GS sensor family with new VGA imagers that feature the industry's smallest pixel size of 2.2um—the OG0VA image sensor and OC0VA CameraCubeChip™ wafer-level camera module. Additionally, the OC0VA is the first CameraCubeChip with Nyxel technology. They have a high QE of 40% at 940nm and 60% at 850nm.
The OG0VA image sensor provides 640x480 VGA resolution at 240fps and 320x240 QVGA resolution at 480 fps, in the optical format of 1/10 inches. The OC0VA CameraCubeChip combines this sensor with image signal processing and optics into a 2.69 x 3.04 x 3.04mm wafer-level camera module. Additionally, their low light sensitivity is excellent, with significantly lower gain than the industry's typical 3.0um pixel size for an improved SNR.
"There is a growing need for global shutter technology at a variety of resolution levels to accurately capture the images of moving objects, along with excellent NIR performance and small size," said Devang Patel, senior staff marketing manager for the security and emerging segments at OmniVision. "The OG0VA and OC0VA expand our family of the industry's smallest GS imagers by providing VGA resolution options with the best NIR performance in a global shutter device."
Omnivision also continues the series of videos on its 8MP automotive sensor features:
VentureBeat, EETimes, JerusalemPost: TriEye is partnering with Porsche to us SWIR sensing technology, with the hopes of advancing the performance of ADAS and autonomous vehicles.
“Our collaboration with Porsche has been exceptional from day one, and we look forward to growing this potential,” said TriEye CEO and co-founder Avi Bakal. “The fact that Porsche, a leading car manufacturer, has decided to invest in TriEye and evaluate TriEye’s CMOS-based SWIR camera to help further improve Advanced Driver Assistance Systems is a significant vote of confidence in our technology.”
Porsche executive board and development member Michael Steiner says, “We see great potential in this sensor technology that paves the way for the next generation of driver assistance systems and autonomous driving functions. SWIR can be a key element: It offers enhanced safety at a competitive price.”
BusinessWire, Thomas-PR: RoboSense solid-state LiDAR RS-LiDAR-M1Simple (Simple Sensor Version) is now ready for customer delivery, priced at $1,898. The new RS-LiDAR-M1Simple is less than half the size of the previous version, with dimensions of 4.3” x 1.9” x 4.7” (110mm x 50mm x 120mm), and its hardware performance is "virtually equal to the serial production version provided to OEMs." The main body design of this automotive-grade solid-state LiDAR is finalized and ready for shipment.
In addition, RoboSense will demonstrate the world’s first smart solid-state LiDAR, the RS-LiDAR-M1Smart (Smart Sensor Version) with an on-vehicle public road test. The RS-LiDAR-M1 family is said to have the performance advantages of traditional mechanical LiDAR, simultaneously also taking into consideration requirements for the mass production of vehicles. The RS-LiDAR-M1 family meets every automotive-grade requirement, including intelligence, low cost, stability, simplified structure and small size, vehicle body design friendliness, and algorithm processed semantic-level perception output results.
“The RS-LiDAR-M1 is an optimal choice for the serial production of self-driving cars, far superior to mechanical LiDAR. The sooner solid-state LiDAR is used, the sooner production will be accelerated to mass-market levels,” said Mark Qiu, RoboSense COO.
RS-LiDAR-M1 Family Features:
125 laser beams: the RS-LiDAR-M1 has a field of view of 120°*25°, which is the MEMS solid-state LiDAR’s largest field of view among released products worldwide. RoboSense uses 905nm lasers with low cost, automotive grade and small size instead of expensive 1550nm lasers. RoboSense breaks ranging ability limits to 150m at 10% NIST target, which is also MEMS solid-state LiDAR’s longest detection range. The frame rate of RS-LiDAR-M1 is increased to 15Hz, which can reduce the impact of point cloud distortion caused by target movement.
World’s smallest MEMS solid-state LiDAR: the size has been reduced by half, one-tenth the size of conventional 64-beam mechanical LiDAR.
Reduced parts for lower cost, shorter production time, and large-scale production capacity. Parts have reduced from hundreds to dozens in comparison to traditional mechanical LiDARs, greatly reducing the cost and shortening production time -- achieving a breakthrough in manufacturability. The coin-sized module processes the optical-mechanical system results to meet autonomous driving performance and mass production requirements.
Modular design: the scalability and layout flexibility of the optical module lay the foundation for subsequent MEMS LiDAR products and support the customization of products for different application cases.
Stable and reliable: the RS-LiDAR-M1 uses VDA6.3 as the basis for project management, and the development of all modules undergoes a complete V model closed loop. RoboSense fully implemented IATF16949 quality management system and ISO26262 functional safety standards, combining ISO16750 test requirement and other automotive-grade reliability specifications to verify the RS-LiDAR-M1 series of products. MEMS mirror is the core component in RS-LiDAR-M1. According to the AEC-Q100 standard, combining the characteristics of MEMS micro-mirror, a total of ten verification test groups are designed covering temperature, humidity, packaging process, electromagnetic compatibility, mechanical vibration and shock, life-time, and others. The cumulative test time for all test samples has now exceeded 100,000 hours. The RS-LiDAR-M1 uses 905nm lasers to achieve long-distance and also meets Class 1 laser safety. The longest-running prototype has been tested for more than 300 days, while the total road test mileage exceeds 150,000 kilometers with no degradation found in various testing scenarios.
All-weather: In Vienna, Austria, the RS-LiDAR-M1 was tested for rain and fog under different light and wind speed conditions. The test results prove that the RS-LiDAR-M1 has met the standards, and the final mass-produced RS-LiDAR-M1 will adapt to all climatic and working conditions.
Minimal wear and tear: as a solid-state LiDAR, the RS-LiDAR-M1 has minimal wear and tear vs. movable mechanical structures, eliminating potential optoelectronic device failures due to mechanical rotation. The characteristics of solid-state provide a reasonable internal layout, heat dissipation, and stability -- a leap in quality as compared to mechanical LiDAR.
The hardware-only version of RS-LiDAR-M1 is currently available for customers, with a retail price of $1898. RoboSense will deliver current orders from key customers and upgrade the production line in Q1 2020, completing retail product delivery by Q1 2020.
Reuters: Bosch announces that it has developed LiDAR too: "The new Bosch sensor will cover both long and close ranges – on highways and in the city. By exploiting economies of scale, Bosch wants to reduce the price for the sophisticated technology and render it suitable for the mass market."
BusinessWire: Velodyne is to present its Five Diamonds rating system to clarify and standardize terminology for ADAS features. The system aims to encourage transparency in the marketplace and promote the maximum positive effect of ADAS technologies.
FinancialTimes: Sony joins the race to develop automotive LiDAR.
“Smartphones probably made the biggest impact in the 21st century in terms of changing people’s lives. Mobility is next,” says Kenichiro Yoshida, Sony CEO. The company's new solid-state LiDAR is said to be Si-based, long-range, lower-cost, compact, and insensitive to vibrations, according to FT sources.
In spite of success on image sensor market, Sony penetration to automotive applications has been quite limited. According to TSR, in 2018, Sony was on 5th place in automotive image sensors with a 3% market share, compared with 62% of ON Semi and 20% of OmniVision.
“I kept on asking why we couldn’t reverse our market position despite our sensors obviously being better than others,” says Terushi Shimizu, EVP of Sony Semiconductor group. “But we didn’t want to be drawn into the cost-cutting competition. We want our sensors to be used because our technology is better.”
PRNewswire: OmniVision announces the OA805, a video processor that supports HEVC compression with the lowest power consumption in the industry.
The OA805 has a boot-up time that is significantly faster than its nearest competitor. This rapid startup eliminates any delay between motion detection and video recording, potentially allowing the camera to instantly alert users of suspicious activities. Within 0.1 seconds, the OA805 can go from completely powered off to fully functional.
"High-end surveillance cameras need video processors that can cope with high-definition 4K resolution video streams. However, high resolution video translates into high power consumption, and manufacturers have had to either settle for lower resolution video to conserve power in their battery-powered systems, or to rely on hard-wired solutions," said David Ho, product marketing manager at OmniVision. "With the OA805, this power versus resolution trade-off is eliminated. Its support for both HEVC and H.264 video compression, in combination with the industry's lowest power consumption and fastest boot-up time, allows designers to incorporate leading-edge performance into products that their customers can quickly and easily install anywhere, so they never miss a thing."
Its HDR processing capability allows the OA805 to accept input from RBG/IR image sensors, for videos taken during the day or at night, in conditions with widely contrasting bright and dark images.
As an upgrade from OmniVision's OV798, the OA805 adds HEVC capability, consumes less power, boots up faster and offers higher resolution processing. This video processor accepts up to 16-megapixel captures from an image sensor and outputs up to 4K resolution video at 30 fps using HEVC encoding and decoding. It also supports multiple video streams at lower resolution, including H.264 1080p resolution at 60fps, as well as HDR and RGB-IR.
The OA805 video processor is available now.
In an unrelated news, Omnivision Product Manager Celine Baron features in a number of videos about the company's 8MP automotive sensor:
"We present the first 1Mpixel SPAD camera ever reported. The camera features 3.8ns time gating and 24kfps frame rate; it was fabricated in 180nm CIS technology. Two pixels have been designed with a pitch of 9.4μm in 7T and 5.75T configurations, respectively, achieving a maximum fill factor of 13.4%. The maximum PDP is 27%, median DCR 2.0cps, variation in gating length 120ps, position skew 410ps, and rise/fall time less than 550ps, all FWHM at 3.3V of excess bias. The sensor was used to capture 2D/3D scenes over 2m with an LSB of 5.4mm and a precision better than 7.8mm. Extended dynamic range is demonstrated in dual exposure operation mode. Spatially overlapped multi-object detection is experimentally demonstrated in single-photon time-gated ToF for the first time."
"We present an approach for single-frame three-dimensional (3-D) imaging using multiwavelength array projection and a stereo vision setup of two multispectral snapshot cameras. Thus a sequence of aperiodic fringe patterns at different wavelengths can be projected and detected simultaneously. For the 3-D reconstruction, a computational procedure for pattern extraction from multispectral images, denoising of multispectral image data, and stereo matching is developed. In addition, a proof-of-concept is provided with experimental measurement results, showing the validity and potential of the proposed approach."
