Raw preprocessing

1 Row noise correction using black reference columns

Using octave/matlab-like pseudocode.

1.1 For previous frame, compute:

For each row:

med_left(y)  = median(im(y,1:8))                   % median for first 8 columns
med_right(y) = median(im(y,end-7:end))             % median of last 8 columns

For the entire image:

offset_odd_left   = median(med_left(1:2:end))      % median of medians from odd left rows
offset_odd_right  = median(med_right(1:2:end))     % median of medians from odd right rows
offset_even_left  = median(med_left(2:2:end))      % median of medians from even left rows
offset_even_right = median(med_right(2:2:end))     % median of medians from even right rows

1.2 For current frame:

1. Subtract a dark frame (constant image)

im -= dark_frame

2. Fix black level offsets:

• these are expected to change very slowly over time, not from frame to frame, so we are reusing them from previous frame to avoid multiple passes
• subtract an offset for odd rows and another offset for even rows
• the offset varies linearly from left to right
• we will attempt to keep a constant black level: let's say target_black_level = 128
• subtract these values from each row:

offset_odd(x)  = offset_odd_left + (offset_odd_right - offset_odd_left) * x / width - target_black_level;
offset_even(x) = offset_even_left + (offset_even_right - offset_even_left) * x / width - target_black_level;
im(1:2:end,x) -= offset_odd(x)
im(2:2:end,x) -= offset_even(x)

3. Fix dynamic row noise

• this changes with every frame, and is not correlated from frame to frame
• we have a noisy estimation of this row noise in the black reference columns
• average black columns (16 values for each row):

mb(y) = mean(black_columns)(y) = mean(im(y, [1:8 end-7:end])

• from each row, we will subtract a scalar value: k * mb(y), where k is 0.6 for gain x1:

im(y,:) -= k * mb(y)

• proof that 0.6 is optimal: https://github.com/apertus-open-source-cinema/misc-tools-utilities/commit/48de47b2a544dc32bbd5a8fd7701bb44a31ea850#diff-624053a553f49c0036b4d31282e58b2fR301

Optional steps (to be tested):

• apply a gain frame:

im = (im - target_black_level) .* gain_frame + target_black_level

• apply a clip frame in overexposed highlights:

im(im > clip_thr) -= clip_frame(im > clip_thr)  % clip_thr is about 2500 for gain x1; smooth transition might be also needed

• apply a look-up table (either per-channel or global):

im = lut(im)

That's it.

Doing the math on 16 bits is preferred, and when getting back to 12 bits, add one bit of random noise (better results). See pack12/unpack12 on raw2dng for details on the implementation, and http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html for the rationale behind it.