Thursday, May 14, 2020

Sony Announces CIS with Embedded AI Processor

PRNewswire: Sony announces the upcoming release of two intelligent vision sensors, said to be the first image sensors in the world to be equipped with AI processing functionality, the IMX500 and IMX501. Including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, which, when using cloud services, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs. The new products are aimed to a diverse range of applications in the retail and industrial equipment industries and contributing to building optimal systems that link with the cloud.




The pixel chip is back-illuminated and has approximately 12.3MP eff. In addition to the conventional image sensor operation circuit, the logic chip is equipped with Sony's original DSP dedicated to AI signal processing, and memory for the AI model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for edge AI systems.

Signals acquired by the pixel chip are run through an DSP and AI processing is done in the process stage on the logic chip, and the extracted information is output as metadata, reducing the amount of data handled. Ensuring that image information is not output helps to reduce security risks and address privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, including ISP format output images (YUV/RGB) and ROI specific area extract images.

The new sensor products from Sony perform ISP processing and high-speed AI processing (3.1ms processing for MobileNet V1) on the logic chip, completing the entire process in a single video frame. This design makes it possible to deliver high-precision, real-time tracking of objects while recording video.

6 comments:

  1. While it has been obvious for 30-40 years that this is where smart image sensors needed to go, actually making it happen (assuming this is not hype) is a major achievement. I am jealous but also happy to have lived long enough to see this vision come to reality.

    ReplyDelete
    Replies
    1. The same feeling as you. When I was PhD student, we dreamed about "one-processor per pixel". It's not yet the case, but it will come very soon ...

      Delete
  2. A significant breakthrough also seems the DVDD of 0.84V. Quite amazing considering most sensors are at 1.05V. Any indication on logic wafer process node?

    ReplyDelete
    Replies
    1. I think that the data communications are very local so the noise margin can be low.

      Delete
  3. performace per cost is terrible. Buzzword, AI needs perfomace ranking.

    ReplyDelete
  4. What is the purpose? Low power? More processing speed?

    ReplyDelete

All comments are moderated to avoid spam and personal attacks.