Difference between revisions of "PLR"

From apertus wiki
Jump to: navigation, search
 
(8 intermediate revisions by one other user not shown)
Line 13: Line 13:
I'll use an IT8 chart, exposed at 30 ms (normal exposure) and 100 ms (overexposed). A little dark in the lab today, but shouldn't be a big problem.
I'll use an IT8 chart, exposed at 30 ms (normal exposure) and 100 ms (overexposed). A little dark in the lab today, but shouldn't be a big problem.


To analyze the images, I'll use octave 4.0, compiled [http://marcelojoeng.blogspot.com/2012/11/compile-octave-using-1632-bits-colour.html with 16-bit image support]. The scripts should run in Matlab as well, with minimal changes.
To analyze the images, I'll use octave 4.0, compiled [https://marcelojoeng.blogspot.com/2012/11/compile-octave-using-1632-bits-colour.html with 16-bit image support]. The scripts should run in Matlab as well, with minimal changes.


=====Linear exposures=====
=====Linear exposures=====


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/30ms-lin.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-lin.jpg
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/30ms-lin.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-lin.jpg


Let's check if the first image is really exposed to the right, in octave.
Let's check if the first image is really exposed to the right, in octave.
Line 34: Line 34:
  end
  end


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-vs-30ms-lin.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-vs-30ms-lin.png


* this approximates (but it's not equal to!) the response curve without PLR
* this approximates (but it's not equal to!) the response curve without PLR
Line 44: Line 44:
Let's start with a 2-segment PLR exposure, 100ms/10ms, vtfl2=32. That means, Number_slopes = 2, Exp_time = 8072, Exp_kp1 = 805, Vtfl=96.
Let's start with a 2-segment PLR exposure, 100ms/10ms, vtfl2=32. That means, Number_slopes = 2, Exp_time = 8072, Exp_kp1 = 805, Vtfl=96.


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-10ms-32.jpg
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-10ms-32.jpg


Although the image may look a little overexposed at first sight (because of the magenta cast), the raw levels seem to be alright:
Although the image may look a little overexposed at first sight (because of the magenta cast), the raw levels seem to be alright:
Line 59: Line 59:
  end
  end


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-10-32-plr-vs-30ms-lin.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-10-32-plr-vs-30ms-lin.png


* the knee point isn't as sharp as in the datasheet
* the knee point isn't as sharp as in the datasheet
Line 69: Line 69:
Let's leave Vtfl constant and change exposure time for the second segment (0,1,5,10,20,35,50 ms).
Let's leave Vtfl constant and change exposure time for the second segment (0,1,5,10,20,35,50 ms).


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-0ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-5ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-10ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-20ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-35ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-50ms-32-tiny.jpg
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-0ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-5ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-10ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-20ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-35ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-50ms-32-tiny.jpg


Commands for plotting the "response curve" approximations are left as an exercise to the reader. Result:
Commands for plotting the "response curve" approximations are left as an exercise to the reader. Result:


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin.png


======Changing Vtfl======
======Changing Vtfl======
Line 79: Line 79:
Now let's leave exposure time constant (100ms/1ms) and change Vtfl (off,0,8,16,24,32,40,48,56,63):
Now let's leave exposure time constant (100ms/1ms) and change Vtfl (off,0,8,16,24,32,40,48,56,63):


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-00-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-0-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-8-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-16-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-24-tiny.jpg
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-00-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-0-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-8-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-16-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-24-tiny.jpg


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-32-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-40-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-48-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-56-tiny.jpg http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-63-tiny.jpg
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-32-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-40-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-48-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-56-tiny.jpg https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100ms-1ms-63-tiny.jpg


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-1-x-plr-vs-30ms-lin.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-1-x-plr-vs-30ms-lin.png
 
======Changing Exp_time======
 
Now let's leave Exp_kp1 and Vtfl constant (0ms/32) and change base exposure time (20,50,100,150 ms):
 
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/x-0-32-plr-vs-30ms-lin.png
 
* there is a noticeable light leak in the second segment, but it doesn't seem to change with base exposure time.
* most of the noise in these curves seems static (correctable with calibration frames)
 
======Changing Exp_kp2 and Vtfl3======
 
As expected, these settings do not seem to have any noticeable effect when num_slopes = 2.


====Mathematical models====
====Mathematical models====
Line 91: Line 104:
Let's go back to the scenario with constant Vtfl and variable exposure.
Let's go back to the scenario with constant Vtfl and variable exposure.


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin.png




Line 118: Line 131:
Let's see how it compares to our real data.
Let's see how it compares to our real data.


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin-model0.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin-model0.png


Getting closer.
Getting closer.
Line 124: Line 137:
Notice the slopes of the second segment seem to be higher in the real data, compared to our model, as if the real exposure time would be a little higher.
Notice the slopes of the second segment seem to be higher in the real data, compared to our model, as if the real exposure time would be a little higher.


Let's extend the model like this: assume the circuits that keep the pixel in "reset" state (after reaching vtfl) are not perfect - there may be still some light accumulating in the sensels. Let's call this gain "reset_leak", and we expect it to be quite a bit lower than 1 (which is the gain used during actual exposures).
Since the slope at exp_kp1=0 doesn't depend on total exposure time, let's assume there is some sort of exposure leak - that is, the actual exposure time is exp_kp1 + some constant (called exp_leak).


  function y = plr_model_1(x, exp_time, vtfl_thr, exp_kp1, reset_leak)
  function y = plr_model_1(x, exp_time, vtfl_thr, exp_kp1, exp_leak)
    
    
  [ ... ]   
    [ ... ]   
    
    
  % time while the pixel is kept in "reset" state, after reaching vtfl
    % how the exposure behaves in the highlights
  reset_time = exp_time - t_reach_vtfl - exp_kp1;
    y_highlight = vtfl_thr + x .* (exp_kp1 + exp_leak);
    
    
  % how the exposure behaves in the highlights
    [ ... ]   
  y_highlight = vtfl_thr + x .* exp_kp1 + x .* reset_time * reset_leak;
 
  [ ... ]   
    
    
  end
  end


http://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin-model1.png
https://files.apertus.org/AXIOM-Beta/snapshots/PLRHDR/wiki/100-x-32-plr-vs-30ms-lin-model1.png
 
Tuning parameters (trial and error): vtfl_thr = 850, exp_leak = 2.5ms.


Tuning parameters (trial and error): vtfl_thr = 900, reset_leak = 1/40.
Black level 128 (after subtracting black reference columns, dark frame and dark current nonuniformity). Reference frames obtained from fitting 256 dark frames taken at exposures from 1 to 64 ms, in 1 ms increments, 4 images at each exposure, with raw2dng --swap-lines --calc-dcnuframe *x1*.raw12.


This model appears to explain the actual response curves pretty well, but there's still room for improvement.
This model appears to explain the actual response curves pretty well, but there's still room for improvement.


At this point, I would try to find some real response curves and reduce the noise of the test images by averaging multiple frames, then repeat the experiment. This will take a while.
-----
<blockquote>
'''At this point, I would try to find some real response curves and reduce the noise of the test images by averaging multiple frames, then repeat the experiment. This will take a while.''
</blockquote>
-----
 
After averaging 50 frames at each setting, the noise in the curve remained the same, so it must be systematic error (correctable with some sort of calibration frame).
 
Finding response curves from bracketed images is difficult - we can't just scale exposure times hoping we'll get the same curve. We will need a dimmable light source with no ambient light (or some dedicated calibration hardware).
 


[to be continued]
[to be continued]

Latest revision as of 16:01, 26 June 2018

Some attempts to reverse engineer the PLR high dynamic range mode from CMV12000.

1 Registers

  • 79: Number_slopes: 1,2,3.
  • 75-78: Exp_kp1, Exp_kp2: exposure times for highlights (same formula as Exp_time)
  • 106: Vtfl2, Vtfl3: knee point locations (range: 0-63; units: unknown)

Let's check the effect.

1.1 Register effects

I'll use an IT8 chart, exposed at 30 ms (normal exposure) and 100 ms (overexposed). A little dark in the lab today, but shouldn't be a big problem.

To analyze the images, I'll use octave 4.0, compiled with 16-bit image support. The scripts should run in Matlab as well, with minimal changes.

1.1.1 Linear exposures

30ms-lin.jpg 100ms-lin.jpg

Let's check if the first image is really exposed to the right, in octave.

a = read_raw('30ms-lin.DNG');
prctile(a(:),99) - 128        % note: black level is forced to 128 in raw2dng
ans =  2269                   % clipping starts at about 2400-2500 above black

Let's check if the clipping point is, indeed, where I say:

figure, hold on
colors = 'rgcb';                                  % meaning: red, gren1, green2 (plotted as cyan), blue (raw data from Bayer channels)
[g30,c30]   = sample_it8('30ms-lin.DNG', 0);      % read median RGGB swatch values from IT8 chart
[g100,c100] = sample_it8('100ms-lin.DNG', 0);     % first output arg is grayscale, second is full color, each column is a Bayer channel
for i = 1:4
  plot(c30(:,i), c100(:,i), ['.' colors(i)]);     % plot the 100ms image vs the 30ms one (extremely rough guess for response curve)
end

100ms-vs-30ms-lin.png

  • this approximates (but it's not equal to!) the response curve without PLR
  • this plot assumes the sensor response in the 30ms image (which was not overexposed) is linear (but it's probably not)
  • the sensor doesn't clip very harshly to white (could be the PLR circuits kicking in with a very low exposure time? to be checked)
1.1.2 2-segment PLR exposures

Let's start with a 2-segment PLR exposure, 100ms/10ms, vtfl2=32. That means, Number_slopes = 2, Exp_time = 8072, Exp_kp1 = 805, Vtfl=96.

100ms-10ms-32.jpg

Although the image may look a little overexposed at first sight (because of the magenta cast), the raw levels seem to be alright:

c = read_raw('100ms-10ms-32.DNG');
prctile(c(:),99)
ans =  1957

Let's check our "response curve" approximation:

[g100p,c100p] = sample_it8('100ms-10ms-32.DNG', 0);
figure, hold on
for i = 1:4
  plot(c30(:,i), c100p(:,i), ['.' colors(i)]);
end

100-10-32-plr-vs-30ms-lin.png

  • the knee point isn't as sharp as in the datasheet
  • its location is around 400 (todo: find the relationship between this and vtfl)
  • the second exposure segment appears a little noisy
1.1.2.1 Changing Exp_kp1

Let's leave Vtfl constant and change exposure time for the second segment (0,1,5,10,20,35,50 ms).

100ms-0ms-32-tiny.jpg 100ms-1ms-32-tiny.jpg 100ms-5ms-32-tiny.jpg 100ms-10ms-32-tiny.jpg 100ms-20ms-32-tiny.jpg 100ms-35ms-32-tiny.jpg 100ms-50ms-32-tiny.jpg

Commands for plotting the "response curve" approximations are left as an exercise to the reader. Result:

100-x-32-plr-vs-30ms-lin.png

1.1.2.2 Changing Vtfl

Now let's leave exposure time constant (100ms/1ms) and change Vtfl (off,0,8,16,24,32,40,48,56,63):

100ms-1ms-00-tiny.jpg 100ms-1ms-0-tiny.jpg 100ms-1ms-8-tiny.jpg 100ms-1ms-16-tiny.jpg 100ms-1ms-24-tiny.jpg

100ms-1ms-32-tiny.jpg 100ms-1ms-40-tiny.jpg 100ms-1ms-48-tiny.jpg 100ms-1ms-56-tiny.jpg 100ms-1ms-63-tiny.jpg

100-1-x-plr-vs-30ms-lin.png

1.1.2.3 Changing Exp_time

Now let's leave Exp_kp1 and Vtfl constant (0ms/32) and change base exposure time (20,50,100,150 ms):

x-0-32-plr-vs-30ms-lin.png

  • there is a noticeable light leak in the second segment, but it doesn't seem to change with base exposure time.
  • most of the noise in these curves seems static (correctable with calibration frames)
1.1.2.4 Changing Exp_kp2 and Vtfl3

As expected, these settings do not seem to have any noticeable effect when num_slopes = 2.

1.2 Mathematical models

1.2.1 First models of the response curve for 2-segment PLR exposures

Let's go back to the scenario with constant Vtfl and variable exposure.

100-x-32-plr-vs-30ms-lin.png


Notice the knee point appears to move to the right with higher Exp_kp1 values. Why would this happen?

If we assume the knee point threshold (Vtfl2) is only monitored while we still have time to fully expose the second segment (Exp_kp1), we can imagine a model of the PLR exposure that looks like this:

function y = plr_model_0(x, exp_time, vtfl_thr, exp_kp1)
   % regular exposure time
   y = x .* exp_time;
   
   % time needed to reach the Vtfl threshold (lower input levels => longer times)
   t_reach_vtfl = vtfl_thr ./ x;
   
   % which pixels reach vtfl?
   % assume this threshold is only monitored while we still have time
   % to fully perform the exp_kp1 (that is, before exp_time - exp_kp1)
   reached_vtfl = t_reach_vtfl < exp_time - exp_kp1;
   
   % how the exposure behaves in the highlights
   y_highlight = vtfl_thr + x .* exp_kp1;
   
   % copy highlight values only for those pixels that reached Vtfl
   y(reached_vtfl) = y_highlight(reached_vtfl);
end

Let's see how it compares to our real data.

100-x-32-plr-vs-30ms-lin-model0.png

Getting closer.

Notice the slopes of the second segment seem to be higher in the real data, compared to our model, as if the real exposure time would be a little higher.

Since the slope at exp_kp1=0 doesn't depend on total exposure time, let's assume there is some sort of exposure leak - that is, the actual exposure time is exp_kp1 + some constant (called exp_leak).

function y = plr_model_1(x, exp_time, vtfl_thr, exp_kp1, exp_leak)
  
   [ ... ]  
  
   % how the exposure behaves in the highlights
   y_highlight = vtfl_thr + x .* (exp_kp1 + exp_leak);
  
   [ ... ]  
  
end

100-x-32-plr-vs-30ms-lin-model1.png

Tuning parameters (trial and error): vtfl_thr = 850, exp_leak = 2.5ms.

Black level 128 (after subtracting black reference columns, dark frame and dark current nonuniformity). Reference frames obtained from fitting 256 dark frames taken at exposures from 1 to 64 ms, in 1 ms increments, 4 images at each exposure, with raw2dng --swap-lines --calc-dcnuframe *x1*.raw12.

This model appears to explain the actual response curves pretty well, but there's still room for improvement.


'At this point, I would try to find some real response curves and reduce the noise of the test images by averaging multiple frames, then repeat the experiment. This will take a while.


After averaging 50 frames at each setting, the noise in the curve remained the same, so it must be systematic error (correctable with some sort of calibration frame).

Finding response curves from bracketed images is difficult - we can't just scale exposure times hoping we'll get the same curve. We will need a dimmable light source with no ambient light (or some dedicated calibration hardware).


[to be continued]