Wednesday, November 27, 2019

Samsung Paper on AI Demosaicing

A preprint paper "Deep Demosaicing using ResNet-Bottleneck Architecture" by Divakar Verma, Manish Kumar, and Srinivas Eregala from Samsung R&D Institute Bengaluru, India, proposes CNN network to reduce demisaicing artifacts:

"Demosaicing is a fundamental step in a camera pipeline to construct a full RGB image from the bayer data captured by a camera sensor. The conventional signal processing algorithms fail to perform well on complex-pattern images giving rise to several artefacts like Moire, color and Zipper artefacts. The proposed deep learning based model removes such artefacts and generates visually superior quality images. The model performs well on both the sRGB (standard RGB color space) and the linear datasets without any need of retraining. It is based on Convolutional Neural Networks (CNNs) and uses a residual architecture with multiple `Residual Bottleneck Blocks' each having 3 CNN layers. The use of 1x1 kernels allowed to increase the number of filters (width) of the model and hence, learned the inter-channel dependencies in a better way. The proposed network outperforms the state-of-the-art demosaicing methods on both sRGB and linear datasets."

5 comments:

  1. Why using "AI" in the title? The paper does't mention this term.

    ReplyDelete
    Replies
    1. It's a sort of tag to simplify search of the news items afterwards.

      Delete
  2. Small format high resolution sensors may have lateral chromatic aberration that exceeds pixel size, making de-mosaic to lose translation invariance. Pure CNN will not work in that case.

    Here is an illustration how Bayer mosaic may look in off-center area:
    https://blog.elphel.com/wp-content/uploads/2017/01/chromatic_lateral_600.png

    ReplyDelete
    Replies
    1. That's why chromatic aberrations are usually removed in image DSP before demosaic

      Delete
    2. When you remove chromatic aberration, there will be no uniform RG/GB pattern left. And demosaic will not be purely convolutional as it will need to be aware of the specific to the FoV Bayer mosaic distortion. Article mentions CNN - it can only work with simulated images with undistorted Bayer mosaic.

      Delete

All comments are moderated to avoid spam and personal attacks.