Lists

▼

Wednesday, December 05, 2018

Unconstrained Machine Learning Search Confirms that Bayer CFA is Optimal

Occipital HW Leader Evan Fletcher did an interesting work on searching for an optimal CFA pattern among arbitrary large sized patterns in an unconstrained machine learning search. The result might surprise the companies proposing better CFA mosaics, such as Fujifilm:

"After training for ~24 hours, the learned color filter array looks quite familiar:


The network seems to have learned something very similar to a RGGB Bayer pattern, complete with 2×2 repetition and the pentile arrangement of the green pixels! This was quite surprising, especially given that there is no spatial constraint on arrangement or repetition in this network design whatsoever.

This optimization appears to have independently confirmed the venerable Bayer pattern as a good choice for a mosaic function – it was fascinating to see a familiar pattern arise from an unconstrained optimization."
at

8 comments:

  1. AnonymousDecember 6, 2018 at 4:38 AM

    Unfortunately this result might be meaningless because that the training image set is most probably from bayer pattern image sensors.

    ReplyDelete
    Replies
    1. Vladimir KoifmanDecember 6, 2018 at 8:25 AM

      May be. The author mentions this too as one of the factors that he needs to double check in future:

      "As the training images were captured using cameras that almost certainly use a Bayer pattern themselves, there is a concern that Bayer artifacts (i.e. reduced resolution in red/blue vs green) may be what is driving the Bayer pattern to arise in the learned color filter array, despite efforts to hide or reduce these Bayer artifacts in the training set.

      I may try training with fully synthetic rendered data, which would never involve a Bayer pattern, in an effort to eliminate this possibility.      "

      Delete
  2. Offline dreamsDecember 6, 2018 at 8:27 AM

    There is another result suggesting that more complex CFA might be better for color sensing in noisy environments: http://inf.ufrgs.br/~bhenz/projects/joint_cfa_demosaicing/
    The result is 4x4 CFA with about 8 distinct colors

    ReplyDelete
  3. AnonymousDecember 6, 2018 at 11:54 AM

    They should use foveon sensor.

    ReplyDelete
    Replies
    1. Vladimir KoifmanDecember 6, 2018 at 11:57 AM

      You mean Foveon as a source for training and test pictures, right?

      Delete
    2. AnonymousDecember 6, 2018 at 1:18 PM

      Or capture a scene using a monochrome sensor/ camera with several colour filters? That would be the best reference right? Is there a pixel size or resolution dependency qua cross talk?

      Delete
  4. Offline dreamsDecember 6, 2018 at 8:24 PM

    As an owner of Sigma DP2m with foveon sensor can say that output color is highly regularized and denoised, so it's a very bad source of training data for such algorithims

    ReplyDelete
  5. absoluteBeginnerDecember 7, 2018 at 8:20 AM

    In my humble opinion, you would need true hyperspectral data, simulate the spectral bandwidth of filters and figure out which filters are best. There was an Israeli startup who claimed that they could build a sort of hyperspectral camera using only 5 or 4 filters (they figured it out using principal component analysis). They also checked which set of filters would lead to the optimal color response. I think the company was called hc.vision.

    ReplyDelete

All comments are moderated to avoid spam and personal attacks.