PLR

Some attempts to reverse engineer the PLR high dynamic range mode from CMV12000.

1 Registers

• 79: Number_slopes: 1,2,3.
• 75-78: Exp_kp1, Exp_kp2: exposure times for highlights (same formula as Exp_time)
• 106: Vtfl2, Vtfl3: knee point locations (range: 0-63; units: unknown)

Let's check the effect.

1.1 Register effects

I'll use an IT8 chart, exposed at 30 ms (normal exposure) and 100 ms (overexposed). A little dark in the lab today, but shouldn't be a big problem.

To analyze the images, I'll use octave 4.0, compiled with 16-bit image support. The scripts should run in Matlab as well, with minimal changes.

1.1.1 Linear exposures

Let's check if the first image is really exposed to the right, in octave.

a = read_raw('30ms-lin.DNG');
prctile(a(:),99) - 128        % note: black level is forced to 128 in raw2dng
ans =  2269                   % clipping starts at about 2400-2500 above black

Let's check if the clipping point is, indeed, where I say:

figure, hold on
colors = 'rgcb';                                  % meaning: red, gren1, green2 (plotted as cyan), blue (raw data from Bayer channels)
[g30,c30]   = sample_it8('30ms-lin.DNG', 0);      % read median RGGB swatch values from IT8 chart
[g100,c100] = sample_it8('100ms-lin.DNG', 0);     % first output arg is grayscale, second is full color, each column is a Bayer channel
for i = 1:4
  plot(c30(:,i), c100(:,i), ['.' colors(i)]);     % plot the 100ms image vs the 30ms one (extremely rough guess for response curve)
end

• this approximates (but it's not equal to!) the response curve without PLR
• this plot assumes the sensor response in the 30ms image (which was not overexposed) is linear (but it's probably not)
• the sensor doesn't clip very harshly to white (could be the PLR circuits kicking in with a very low exposure time? to be checked)

1.1.2 2-segment PLR exposures

Let's start with a 2-segment PLR exposure, 100ms/10ms, vtfl2=32. That means, Number_slopes = 2, Exp_time = 8072, Exp_kp1 = 805, Vtfl=96.

Although the image may look a little overexposed at first sight (because of the magenta cast), the raw levels seem to be alright:

c = read_raw('100ms-10ms-32.DNG');
prctile(c(:),99)
ans =  1957

Let's check our "response curve" approximation:

[g100p,c100p] = sample_it8('100ms-10ms-32.DNG', 0);
figure, hold on
for i = 1:4
  plot(c30(:,i), c100p(:,i), ['.' colors(i)]);
end

• the knee point isn't as sharp as in the datasheet
• its location is around 400 (todo: find the relationship between this and vtfl)
• the second exposure segment appears a little noisy

1.1.2.1 Changing Exp_kp1

Let's leave Vtfl constant and change exposure time for the second segment (0,1,5,10,20,35,50 ms).

Commands for plotting the "response curve" approximations are left as an exercise to the reader. Result:

1.1.2.2 Changing Vtfl

Now let's leave exposure time constant (100ms/1ms) and change Vtfl (off,0,8,16,24,32,40,48,56,63):

1.1.2.3 Changing Exp_time

Now let's leave Exp_kp1 and Vtfl constant (0ms/32) and change base exposure time (20,50,100,150 ms):

• there is a noticeable light leak in the second segment, but it doesn't seem to change with base exposure time.
• most of the noise in these curves seems static (correctable with calibration frames)

1.1.2.4 Changing Exp_kp2 and Vtfl3

As expected, these settings do not seem to have any noticeable effect when num_slopes = 2.

1.2 Mathematical models

1.2.1 First models of the response curve for 2-segment PLR exposures

Let's go back to the scenario with constant Vtfl and variable exposure.

Notice the knee point appears to move to the right with higher Exp_kp1 values. Why would this happen?

If we assume the knee point threshold (Vtfl2) is only monitored while we still have time to fully expose the second segment (Exp_kp1), we can imagine a model of the PLR exposure that looks like this:

function y = plr_model_0(x, exp_time, vtfl_thr, exp_kp1)
   % regular exposure time
   y = x .* exp_time;
   
   % time needed to reach the Vtfl threshold (lower input levels => longer times)
   t_reach_vtfl = vtfl_thr ./ x;
   
   % which pixels reach vtfl?
   % assume this threshold is only monitored while we still have time
   % to fully perform the exp_kp1 (that is, before exp_time - exp_kp1)
   reached_vtfl = t_reach_vtfl < exp_time - exp_kp1;
   
   % how the exposure behaves in the highlights
   y_highlight = vtfl_thr + x .* exp_kp1;
   
   % copy highlight values only for those pixels that reached Vtfl
   y(reached_vtfl) = y_highlight(reached_vtfl);
end

Let's see how it compares to our real data.

Getting closer.

Notice the slopes of the second segment seem to be higher in the real data, compared to our model, as if the real exposure time would be a little higher.

Since the slope at exp_kp1=0 doesn't depend on total exposure time, let's assume there is some sort of exposure leak - that is, the actual exposure time is exp_kp1 + some constant (called exp_leak).

function y = plr_model_1(x, exp_time, vtfl_thr, exp_kp1, exp_leak)
  
   [ ... ]  
  
   % how the exposure behaves in the highlights
   y_highlight = vtfl_thr + x .* (exp_kp1 + exp_leak);
  
   [ ... ]  
  
end

Tuning parameters (trial and error): vtfl_thr = 850, exp_leak = 2.5ms.

Black level 128 (after subtracting black reference columns, dark frame and dark current nonuniformity). Reference frames obtained from fitting 256 dark frames taken at exposures from 1 to 64 ms, in 1 ms increments, 4 images at each exposure, with raw2dng --swap-lines --calc-dcnuframe *x1*.raw12.

This model appears to explain the actual response curves pretty well, but there's still room for improvement.

---

'At this point, I would try to find some real response curves and reduce the noise of the test images by averaging multiple frames, then repeat the experiment. This will take a while.

---

After averaging 50 frames at each setting, the noise in the curve remained the same, so it must be systematic error (correctable with some sort of calibration frame).

Finding response curves from bracketed images is difficult - we can't just scale exposure times hoping we'll get the same curve. We will need a dimmable light source with no ambient light (or some dedicated calibration hardware).

[to be continued]