So, the application proposes to modify the image sensor readout architecture so that compressed sensing techniques can be applied to save power. The idea is to divide the pixel array into small blocks 12 and add a MUX at the ADC input combined with an ADC activation generator "generating a random activation code that determines which of the plurality of A/D converters is activated; and obtaining a digital read-out of the analog signals from the activated A/D converters."
The readout timing is modified, so that for each pixel block 12, first the reset level is digitized and then the signal level goes through oversampled ADC, such as 1st order sigma-delta modulator:
Then the MUX select control PMX and ADC activation code ACT work in concert to distribute the signal across a number of ADCs, whereas the reset and signal level from the each pixel are processed by the same ADC:
"According to compressed sensing a compressible signal can be recovered from a small number of random measurements by sparsity promoting non-linear recovery algorithms. Therein, the number of A/D conversions represented by M can be sufficiently less than the number of pixels in a unit of pixel block represented by N, to recover the original image. When M number of A/D converters are simultaneously dedicated to N pixels, the image sensor outputs M digital codes for the N pixels so that the compression ratio M/N can be sufficiently less than 1.
The power consumption and bandwidth of both the A/D conversion and input-output transmission can be suppressed by a factor of M/N in comparison to a normal operation wherein N, A/D conversions are required to obtain an image of N pixels. For example, 16 A/D converters dedicated for a unit of pixel block of 256 pixels achieves a compression ratio of 1/16. Note that the compression ratio can easily be varied by changing the number of A/D converters dedicated for a unit of pixel block at the multiplexers. Further, this feature also enhances the frame rate for a given power consumption level, as frames are outputted at the rate of N/M"
I love compressive sensing, and while for may applications it works quite nice, for imaging it's quite tricky in my opinion. It's a very nice concept in theory, however it turns out it's very hard to squeeze out actual performance improvement from these things in practice. Four problems I observed to be quite limiting:
ReplyDelete(1) randomization of pixel values into random measurements turns any pixel non-linearity error into a random error. Basically, your pixel response non-linearity becomes readout noise that is very hard to cancel out (CDS can't help you there).
(2) Based on what I observed in my experiments, the scene measurements actually need to be linear. That's a problem because the linearity of the ADCs becomes important now (ADC is now converting the linear measurements). And to make ADCs more linear you need to pay with... More power. :-(
(3) third problem is that you will always get to reduce power (or frame rate) by sacrificing the image quality (compression). However, since CS is not a great compression model, in that case you might be just better off reducing the ADC resolution (neglecting the last few LSBs) and saving power that way. Lower resolution ADCs can give you significantly lower power anyway. Unfortunately, in most of the circuit cases and algorithms I tried, I got better results (image PSNR for the same power) by simply reducing ADC resolution than performing compressive sensing in the first place.
(4) Images are spatially large signals so reconstruction algorithms take a long time to converge and are hard to guarantee the timing for any practical frame rates.
I guess with some specialized ADC techniques (such as the one in Oike's work) you can limit the effect of (1) and (2). But (3) and (4) still remain to be solved...