Sony Announces CIS with Embedded AI Processor

PRNewswire: Sony announces the upcoming release of two intelligent vision sensors, said to be the first image sensors in the world to be equipped with AI processing functionality, the IMX500 and IMX501. Including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, which, when using cloud services, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs. The new products are aimed to a diverse range of applications in the retail and industrial equipment industries and contributing to building optimal systems that link with the cloud.

The pixel chip is back-illuminated and has approximately 12.3MP eff. In addition to the conventional image sensor operation circuit, the logic chip is equipped with Sony's original DSP dedicated to AI signal processing, and memory for the AI model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for edge AI systems.

Signals acquired by the pixel chip are run through an DSP and AI processing is done in the process stage on the logic chip, and the extracted information is output as metadata, reducing the amount of data handled. Ensuring that image information is not output helps to reduce security risks and address privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, including ISP format output images (YUV/RGB) and ROI specific area extract images.

The new sensor products from Sony perform ISP processing and high-speed AI processing (3.1ms processing for MobileNet V1) on the logic chip, completing the entire process in a single video frame. This design makes it possible to deliver high-precision, real-time tracking of objects while recording video.