PLR

Some attempts to reverse engineer the PLR high dynamic range mode from CMV12000.

1 Registers

• 79: Number_slopes: 1,2,3.
• 75-78: Exp_kp1, Exp_kp2: exposure times for highlights (same formula as Exp_time)
• 106: Vtfl2, Vtfl3: knee point locations (range: 0-63; units: unknown)

Let's check the effect.

1.1 Register effects

I'll use an IT8 chart, exposed at 30 ms (normal exposure) and 100 ms (overexposed). A little dark in the lab today, but shouldn't be a big problem.

To analyze the images, I'll use octave 4.0, compiled with 16-bit image support. The scripts should run in Matlab as well, with minimal changes.

1.1.1 Linear exposures

Let's check if the first image is really exposed to the right, in octave.

a = read_raw('30ms-lin.DNG');
prctile(a(:),99) - 128        % note: black level is forced to 128 in raw2dng
ans =  2269                   % clipping starts at about 2400-2500 above black

Let's check if the clipping point is, indeed, where I say:

figure, hold on
colors = 'rgcb';                                  % meaning: red, gren1, green2 (plotted as cyan), blue (raw data from Bayer channels)
[g30,c30]   = sample_it8('30ms-lin.DNG', 0);      % read median RGGB swatch values from IT8 chart
[g100,c100] = sample_it8('100ms-lin.DNG', 0);     % first output arg is grayscale, second is full color, each column is a Bayer channel
for i = 1:4
  plot(c30(:,i), c100(:,i), ['.' colors(i)]);     % plot the 100ms image vs the 30ms one (extremely rough guess for response curve)
end

• this approximates (but it's not equal to!) the response curve without PLR
• this plot assumes the sensor response in the 30ms image (which was not overexposed) is linear (but it's probably not)
• the sensor doesn't clip very harshly to white (could be the PLR circuits kicking in with a very low exposure time? to be checked)

1.1.2 2-segment PLR exposures

Let's start with a 2-segment PLR exposure, 100ms/10ms, vtfl2=32. That means, Number_slopes = 2, Exp_time = 8072, Exp_kp1 = 805, Vtfl=96.

Although the image may look a little overexposed at first sight (because of the magenta cast), the raw levels seem to be alright:

c = read_raw('100ms-10ms-32.DNG');
prctile(c(:),99)
ans =  1957

Let's check our "response curve" approximation:

[g100p,c100p] = sample_it8('100ms-10ms-32.DNG', 0);
figure, hold on
for i = 1:4
  plot(c30(:,i), c100p(:,i), ['.' colors(i)]);
end

• the knee point isn't as sharp as in the datasheet
• its location is around 400 (todo: find the relationship between this and vtfl)
• the second exposure segment appears a little noisy

Let's leave Vtfl constant and change exposure time for the second segment (1,5,10,20,50 ms).

Commands for plotting the "response curve" approximations are left as an exercise to the reader. Result:

1.1.2.1 First models of the response curve for 2-segment PLR exposures

Notice the knee point appears to move to the right with higher Exp_kp1 values. Why would this happen?

If we assume the knee point threshold (Vtfl2) is only monitored while we still have time to fully expose the second segment (Exp_kp1), we can imagine a model of the PLR exposure that looks like this:

function y = plr_model_0(x, exp_time, vtfl_thr, exp_kp1)
   % regular exposure time
   y = x .* exp_time;
   
   % time needed to reach the Vtfl threshold (lower input levels => longer times)
   t_reach_vtfl = vtfl_thr ./ x;
   
   % which pixels reach vtfl?
   % assume this threshold is only monitored while we still have time
   % to fully perform the exp_kp1 (that is, before exp_time - exp_kp1)
   reached_vtfl = t_reach_vtfl < exp_time - exp_kp1;
   
   % how the exposure behaves in the highlights
   y_highlight = vtfl_thr + x .* exp_kp1;
   
   % copy highlight values only for those pixels that reached Vtfl
   y(reached_vtfl) = y_highlight(reached_vtfl);
end

Let's see how it compares to our real data.

Getting closer.

Notice the slopes of the second segment seem to be higher in the real data, compared to our model, as if the real exposure time would be a little higher. Let's extend it like this: assume the circuits that keep the pixel in "reset" state (after reaching vtfl) are not perfect - there may be still some light accumulating in the sensels. Let's call this gain "reset_leak", and we expect it to be quite a bit lower than 1 (which is the gain used during actual exposures).

function y = plr_model_1(x, exp_time, vtfl_thr, exp_kp1, reset_leak)
  
  [ ... ]  
  
  % time while the pixel is kept in "reset" state, after reaching vtfl
  reset_time = exp_time - t_reach_vtfl - exp_kp1;
  
  % how the exposure behaves in the highlights
  y_highlight = vtfl_thr + x .* exp_kp1 + x .* reset_time * reset_leak;
  
  [ ... ]  
  
end

Tuning parameters (trial and error): vtfl_thr = 900, reset_leak = 1/40.

This model appears to explain the actual response curves pretty well, but there's still room for improvement.

At this point, I would try to find some real response curves and reduce the noise of the test images by averaging multiple frames, then repeat the experiment. This will take a while.

[to be continued]