Friday, July 11, 2014

Faster Readouts, Smaller Rolling Shutter Distortions

The batch of patent applications is supposed to speedup the image sensor readouts or solve the problems with slow readouts.

First, Sony patent application US20140160334 "Image pickup device, control method, and image pickup apparatus" by Hayato Wakabayashi proposes to read 4 rows simultaneously, while keeping a smooth progression of the rolling shutter:

The ramp DACs 142 and 152 work with a delay to have a smooth shutter curtain advance:

Omnivision application US20140183333 "Conversion circuitry for reducing pixel array readout time" by Robert Johansson proposes to alternate the readout signal polarity in even and odd columns:

The successive approximation ADC cycle starts from switching the cap array reference to an opposite one. Having the input inversion allows even and odd column ADCs use the opposite polarity references, and thus reduce the reference load and speed up the conversion: "Conventional image sensors are not configured to have conversion circuitry with SAR ADCs having capacitors that charge to opposite voltage references. In some conventional image sensors, all of the SAR ADCs in the conversion circuitry are all the same and the capacitors arrays charge to the same voltage reference. This can cause longer settling times for voltage reference because the capacitor arrays inside the SAR ADCs all tax the same voltage reference (at around the same time) by drawing charge from the voltage reference during the SAR ADC's binary search sequence."

Once we talk about the high speed shooting and necessity to reduce the rolling shutter distortions, Anyony Cog brought to my attention Samsung patent application US20130188069 "Methods and apparatuses for rectifying rolling shutter effect" by Omry Sendik, German Voronov, and Michael Slutsky. The application is said to have an unusually detailed description of the compensating algorithm. Not to go too much into the algorithmic details, below is a general structure of the proposed solution:


  1. Wouldn't a global shutter solve these issues?

    1. Do you mean distortions? Yes, it would, but nobody can make small pixels with global shutter so far.

  2. They need to simply add a Strobe Pin to the Sensor Array with a simple Sample/Hold.

    Rolling Shutter (reading line by line when the information is still being updated) does not cause the "Rolling Shutter Effect" (the wobbly and wavy Image or worse, one composed of different pieces) any more than a good Lens can create a good Video.

    The "Rolling Shutter Effect" is caused by motion of the Camera Operator (Pan / Dolly) AND motion of Objects in the Scene; IE: all of the Image is not affected to the same extent and a "true" correction of the Data is akin to creating a 3D Image from a 2D one (not impossible, just difficult).

    Trying filming a Street with large rectangular Vehicles (Buses / Trucks) and panning side to side.

    Notice that the Vehicles that are closer seem to move quicker and are subject to more / less Rolling Shutter Effect than other Objects in the Frame depending on their apparent speed and whether they are travelling left to right, or right to left.

    Film a few Shots where you hold the Camera as still as possible. Now apply Post Image Stabilization to that Scene -- note the wobbly jelloness of the Image, that is the Rolling Shutter Effect being applied to your handshaking.

    IF every Pixel simply skewed a bit to the left or a bit to the right depending on which way you moved the Camera then Rolling Shutter would not be hard to fix, just skew the image back a pixel or two ..

    With every Object subject to a different degree of the Rolling Shutter Effect, and the effect varying with each individual Object, depending on the particular moment in time, a trivial Solution to the Rolling Shutter Effect will not look any better than Rolling Shutter itself, the "error" will only look different.

    Only by strobing all the Sensor's Pixels (and telling them to hold their current value while they wait to be counted) OR by reading all the Pixels simultaneously can one obtain an Image where all parts are temporally aligned.

    You can see a bit of wobbly Video on my YT Channel where I use post-stabilization of most of my Videos. The best (worst) example is this Video:


All comments are moderated to avoid spam.