Difference between revisions of "Black Calibration"

From apertus wiki
Jump to: navigation, search
m
 
(32 intermediate revisions by 3 users not shown)
Line 7: Line 7:
It covers:
It covers:


* dark frame subtraction
* Dark frame subtraction.
* dark current compensation
* dark current compensation.
* using black reference columns (called "optical black" by other manufacturers) to find the black level and fine-tune static offsets.
* using black reference columns (called "optical black" by other manufacturers) to find the black level and fine-tune static offsets.


Note: black reference columns can be used to reduce row noise as well. See [[Pattern Noise]].
'''Note:''' black reference columns can be used to reduce row noise as well. See [[Pattern Noise]].


=Calibration methods=
 
====Dark frame subtraction====
----
 
 
=Calibration Methods=
 
====Dark Frame Subtraction====


This is a basic technique: take a picture with the lens cap on, and subtract it from your image. To make really sure no light is reaching the sensor, also cover the entire camera with something.
This is a basic technique: take a picture with the lens cap on, and subtract it from your image. To make really sure no light is reaching the sensor, also cover the entire camera with something.
----


=====How many dark frames?=====
=====How many dark frames?=====
'''Problem:''' if you take only one dark frame, it will also contain read noise (assummed to be Gaussian and uncorrelated with the read noise from other frames). Therefore, subtracting only one image will actually increase the noise in the final output, by sqrt(2).
'''Problem:''' if you take only one dark frame, it will also contain read noise (assummed to be Gaussian and uncorrelated with the read noise from other frames). Therefore, subtracting only one image will actually increase the noise in the final output, by sqrt(2).


Line 29: Line 39:
So, 16 dark frames will reduce the read noise in the dark frame by 2 stops, 64 frames by 3 stops, and 256 frames by 4 stops.
So, 16 dark frames will reduce the read noise in the dark frame by 2 stops, 64 frames by 3 stops, and 256 frames by 4 stops.


Okay, but how much noise will be added in the output image?
Okay, but how much noise will be added to the output image?


Let's say the read noise stdev in one image is ''r'', so a dark frame averaged from N frames will have noise stdev = r/sqrt(N). Therefore, the noise in the output image will be: sqrt(r^2 + r^2/N) = r * sqrt(1 + 1/N).
Let's say the read noise stdev in one image is ''r'', so a dark frame averaged from N frames will have noise stdev = r/sqrt(N). Therefore, the noise in the output image will be: sqrt(r^2 + r^2/N) = r * sqrt(1 + 1/N).


Check it in octave:
Check it in octave:
  octave:1>  N = 4;
 
  <pre style="white-space: pre-wrap">octave:1>  N = 4;
  octave:2>  a = randn(1,1000000);    # one Gaussian noise sample, with mean=0 and stdev=1
  octave:2>  a = randn(1,1000000);    # one Gaussian noise sample, with mean=0 and stdev=1
  octave:3>  b = randn(N,1000000);    # N noise samples
  octave:3>  b = randn(N,1000000);    # N noise samples
  octave:4>  std(a + mean(b))        # add one noise sample to N averaged noise samples
  octave:4>  std(a + mean(b))        # add one noise sample to N averaged noise samples
ans =  1.1174
octave:5>  sqrt(1 + 1/4)            # compare with the theoretical result
  ans =  1.1180
  ans =  1.1180
</pre>


So, it seems that averaging a small of dark frames will not introduce significant noise in your images (4 should be enough if you are in a hurry, and 16 should give a very good result).
So, it seems that averaging a small number of dark frames will not introduce significant noise in your images (4 should be enough if you are in a hurry, and 16 should give a very good result).


Example: one dark frame at 6ms x1 (same image as above), vs 4 darkframes at 6ms, averaged.
Example: one dark frame at 6ms x1 (same image as above), vs 4 darkframes at 6ms, averaged.
Line 46: Line 60:
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/darkavg-5ms.jpg
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/darkavg-5ms.jpg


=====What about camera settings?=====
 
----
 
 
=====What About Camera Settings?=====


Unfortunately, dark frames depend on many camera settings: analog gain (ISO), exposure, other sensor settings like offset, black sun protection, PLR configuration and so on. Temperature is a variable as well.
Unfortunately, dark frames depend on many camera settings: analog gain (ISO), exposure, other sensor settings like offset, black sun protection, PLR configuration and so on. Temperature is a variable as well.
Line 52: Line 70:
Luckily, the dependence on exposure appears to be linear, so we can take calibration frames at various exposures, combine them into a single dark frame, adjust it for the dark current and use it for the entire range of exposure settings (hopefully). We'll discuss that in the next section.
Luckily, the dependence on exposure appears to be linear, so we can take calibration frames at various exposures, combine them into a single dark frame, adjust it for the dark current and use it for the entire range of exposure settings (hopefully). We'll discuss that in the next section.


====Dark current====
 
----
 
 
====Dark Current====


Let's look at some dark frames: gain x1, exposures 1.2ms, 6ms and 78ms. Notice they get brighter as exposure increase.
Let's look at some dark frames: gain x1, exposures 1.2ms, 6ms and 78ms. Notice they get brighter as exposure increase.


http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-1ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-64ms-01.jpg
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-1ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-64ms-01.jpg


The overall brightness in the dark frame changes with exposure in a linear fashion. We'll try to account for this in two ways: with a simple scalar value, and with a per-pixel correction.
The overall brightness in the dark frame changes with exposure in a linear fashion. We'll try to account for this in two ways: with a simple scalar value, and with a per-pixel correction.
Line 68: Line 91:
<blockquote>"''Dark current is temperature dependent and most modern CMOS digital cameras, circa 2008 and later have on sensor dark current subtraction, but while the dark current level is subtracted, the noise from the dark current still accumulates.''" [http://www.clarkvision.com/reviews/how-to-interpret-reviews/].</blockquote>
<blockquote>"''Dark current is temperature dependent and most modern CMOS digital cameras, circa 2008 and later have on sensor dark current subtraction, but while the dark current level is subtracted, the noise from the dark current still accumulates.''" [http://www.clarkvision.com/reviews/how-to-interpret-reviews/].</blockquote>


=====Simple correction=====
 
----
 
 
=====Simple Correction=====


Experimentally, we have found the dark frame changes with exposure at roughly 0.065 digital units for each ms. This value is multiplied by analog gain. To find this value, take the dark frames at different exposure times (say 1...50 ms), then do a linear fit for the frame average (or median).
Experimentally, we have found the dark frame changes with exposure at roughly 0.065 digital units for each ms. This value is multiplied by analog gain. To find this value, take the dark frames at different exposure times (say 1...50 ms), then do a linear fit for the frame average (or median).
Line 75: Line 102:
  raw2dng *x1*.raw12 --calc-darkframe
  raw2dng *x1*.raw12 --calc-darkframe


Example: a dark frame created from 256 exposures, between 1.2ms and 77ms, without using black reference columns. The image was adjusted (with a constant offset) to match a zero-length exposure, so calling it ''bias frame'' may be a good idea. If we use it to correct the individual dark frames, we should no longer see a variation in overall brightness.
Example: A dark frame created from 256 exposures, between 1.2ms and 77ms, without using black reference columns. The image was adjusted (with a constant offset) to match a zero-length exposure, so calling it ''bias frame'' may be a good idea. If we use it to correct the individual dark frames, we should no longer see a variation in overall brightness.


  Averaged 256 frames exposed from 1.19 to 76.79 ms.
  <pre style="white-space: pre-wrap">Averaged 256 frames exposed from 1.19 to 76.79 ms.
  Dark current: 0.0653 DN/ms
  Dark current: 0.0653 DN/ms
</pre>


Image sequence: bias frame, single dark frames at exposures 1.2, 6 and 77 ms, corrected with the bias frame and the (scalar) dark current average. All files scaled to show a range of 60 DN, but the master dark frame has a different offset than the others.
Image sequence: bias frame, single dark frames at exposures 1.2, 6 and 77 ms, corrected with the bias frame and the (scalar) dark current average. All files scaled to show a range of 60 DN, but the master dark frame has a different offset than the others.
Line 85: Line 113:




----


=====Dark current nonuniformity correction=====
 
=====Dark Current Non-uniformity Correction=====


<blockquote>"''One common use of bias frames is for scaling dark frames.  By subtracting a bias frame from a dark frame, you end up with a “thermal frame.”  A thermal frame contains pixel values showing just the effect of dark current.  Because dark current in any given pixel accumulates at a constant rate, a thermal frame allows you to predict with reasonable accuracy how much dark current there would be for different length exposures. However, given the opportunity, you’re always better off taking dark frames that match the exposure times of your light frames.''" [http://qsimaging.com/ccd_noise_measure.html]</blockquote>
<blockquote>"''One common use of bias frames is for scaling dark frames.  By subtracting a bias frame from a dark frame, you end up with a “thermal frame.”  A thermal frame contains pixel values showing just the effect of dark current.  Because dark current in any given pixel accumulates at a constant rate, a thermal frame allows you to predict with reasonable accuracy how much dark current there would be for different length exposures. However, given the opportunity, you’re always better off taking dark frames that match the exposure times of your light frames.''" [http://qsimaging.com/ccd_noise_measure.html]</blockquote>


<blockquote>"''Dark current non-uniformity is a noise that results from the fact that each pixel generates a slightly
<blockquote>"''Dark current non-uniformity is a noise that results from the fact that each pixel generates a slightlydifferent amount of dark current. This noise can be eliminated by subtracting a dark reference frame from each image. The dark reference frame should be taken at the same temperature and with the same integration time as the image.''" [http://www.digitaltechnologyart.com/dark-current-or-thermal-noise.html]</blockquote>
different amount of dark current. This noise can be eliminated by subtracting a dark reference frame
from each image. The dark reference frame should be taken at the same temperature and with the
same integration time as the image.''" [http://www.digitaltechnologyart.com/dark-current-or-thermal-noise.html]</blockquote>


<blockquote>"''Dark noise is not random; in fact, it is highly repeatable. A given photosite on a sensor will accumulate almost exactly the same amount of dark noise from one exposure to the next, as long as temperature and exposure duration do not vary.''" [http://photo.net/learn/dark_noise/]</blockquote>
<blockquote>"''Dark noise is not random; in fact, it is highly repeatable. A given photosite on a sensor will accumulate almost exactly the same amount of dark noise from one exposure to the next, as long as temperature and exposure duration do not vary.''" [http://photo.net/learn/dark_noise/]</blockquote>


So, rather than using a single scalar value (0.06 dn/ms/gain) for all pixels, we can try finding the individual dark current for each pixel. Instead of doing a linear fit on the overall dark frame brightness (vs exposure), we will do the linear fit per pixel. We'll have to acquire a lot more dark frames to compute a good result, but it might be worth the trouble.
So, rather than using a single scalar value (0.06 dn/ms/gain) for all pixels, we can try finding the individual dark current for each pixel. Instead of doing a linear fit on the overall dark frame brightness (vs exposure), we will do the linear fit per pixel. We'll have to acquire a lot more dark frames to compute a good result, but it might be worth the trouble.


Command-line:
Command-line:
  raw2dng *x1*.raw12 --calc-dcnuframe
  raw2dng *x1*.raw12 --calc-dcnuframe


Example: bias frame (static offset) and dark current frame (exposure-dependent offset). Notice the bias frame looks quite similar to the previous one, but a little darker. The median value of the dark current frame is, unsurprisingly, 0.0645 DN/ms.
 
Example: Bias frame (static offset) and dark current frame (exposure-dependent offset). Notice the bias frame looks quite similar to the previous one, but a little darker. The median value of the dark current frame is, unsurprisingly, 0.0645 DN/ms.


http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/darkframe-x1-no-blackcol-darkcurrent-256.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/dcnuframe-x1-no-blackcol-darkcurrent-256.jpg
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/darkframe-x1-no-blackcol-darkcurrent-256.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/dcnuframe-x1-no-blackcol-darkcurrent-256.jpg




 
Individual dark frames (1.2, 6 and 77 ms) adjusted with dark current non-uniformity:
Individual dark frames (1.2, 6 and 77 ms) adjusted with dark current nonuniformity:


http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-1ms-01-darkcurrent-no-blackcol.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01-darkcurrent-no-blackcol.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-64ms-01-darkcurrent-no-blackcol.jpg
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-1ms-01-darkcurrent-no-blackcol.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-5ms-01-darkcurrent-no-blackcol.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/blackframes-gainx1-offset2047-64ms-01-darkcurrent-no-blackcol.jpg
Line 119: Line 148:
* Dark frame + scalar dark current: [http://files.apertus.org/AXIOM-Beta/snapshots/Darkframes256-21.01.2016/darkframe-check-x1-256-simple.png]
* Dark frame + scalar dark current: [http://files.apertus.org/AXIOM-Beta/snapshots/Darkframes256-21.01.2016/darkframe-check-x1-256-simple.png]
* Dark frame + dark current frame: [http://files.apertus.org/AXIOM-Beta/snapshots/Darkframes256-21.01.2016/darkframe-check-x1-256-dcnu.png]
* Dark frame + dark current frame: [http://files.apertus.org/AXIOM-Beta/snapshots/Darkframes256-21.01.2016/darkframe-check-x1-256-dcnu.png]


You may notice:
You may notice:


* median (black level) variation: noticeable improvement with the second method
* Median (black level) variation: Noticeable improvement with the second method.
* stdev (overall noise): minor improvement at extreme settings
* stdev (overall noise): Minor improvement at extreme settings.
* row noise: identical with both methods
* Row noise: Identical with both methods.
* column noise: small improvement with the second method
* Column noise: Small improvement with the second method.


Indeed, the more complex method appears just a tiny bit better than the simpler one.
Indeed, the more complex method appears just a tiny bit better than the simpler one.
'''
'''>>>> TODO: Nicer graph'''


TODO: nicer graph


=====Dark current measurement from hot pixels=====
----
 
 
=====Dark Current Measurement from Hot Pixels=====


A very interesting idea can be found in [http://www.photonics.com/Article.aspx?AID=44298], where hot pixels can be used to measure the amount of dark current and scale it properly. This will probably account for changes in temperature, and may work at very long exposures without actually having to calibrate the camera in these conditions. Genius, if you ask me.
A very interesting idea can be found in [http://www.photonics.com/Article.aspx?AID=44298], where hot pixels can be used to measure the amount of dark current and scale it properly. This will probably account for changes in temperature, and may work at very long exposures without actually having to calibrate the camera in these conditions. Genius, if you ask me.
Line 137: Line 172:
To be studied.
To be studied.


====Black reference columns====
 
----
 
 
====Black Reference Columns====


This sensor has 8+8 columns that can be used for calibrating the black levels; they are also useful for reducing the dynamic row noise.
This sensor has 8+8 columns that can be used for calibrating the black levels; they are also useful for reducing the dynamic row noise.
Line 145: Line 184:
Therefore, it may be wise to process the black columns for odd and even rows separately, which should already fix some issues like static row noise, or the need for green equilibration.
Therefore, it may be wise to process the black columns for odd and even rows separately, which should already fix some issues like static row noise, or the need for green equilibration.


In raw2dng, this correction is enabled by default, as long as you use a dark frame. You can turn it off with --no-blackcol, if you want.
In <code>raw2dng</code> this correction is enabled by default, as long as you use a dark frame. You can turn it off with <code>--no-blackcol</code> if you want.


=====Black level=====
 
----
 
 
=====Black Level=====


The sensor has two registers that can be used to adjust the black level: one for odd rows, another for even rows. This confirms our finding about two parallel readout circuits.
The sensor has two registers that can be used to adjust the black level: one for odd rows, another for even rows. This confirms our finding about two parallel readout circuits.
Line 155: Line 198:
* If you adjust the offset until the black level becomes roughly 0, you will clip all the data below this level. Good luck subtracting a dark frame after that.
* If you adjust the offset until the black level becomes roughly 0, you will clip all the data below this level. Good luck subtracting a dark frame after that.
* Even if you change the level after doing all the black corrections, you may still have useful data below zero. You will need it when stacking multiple frames, or when doing noise reduction.
* Even if you change the level after doing all the black corrections, you may still have useful data below zero. You will need it when stacking multiple frames, or when doing noise reduction.
* Just FYI, there is a hack for Nikon cameras that moves the black level above 0 [http://landingfield.wordpress.com/2014/05/13/teaser-nikon-dslr-black-point-hack-for-astrophotography/]. Sample images taken with the hack: [http://www.cloudynights.com/topic/473696-rho-ophiuchi-with-nikon-hacked-black-level/] [http://nikonhacker.com/viewtopic.php?t=2548&p=18449#p17973]


I recommend setting the offset so that only a few isolated pixels (if any) reach the value of 0. A black offset of 2047 (registers 87/88) is a good choice. You won't lose any dynamic range by doing that.  
I recommend setting the offset so that only a few isolated pixels (if any) reach the value of 0. A black offset of 2047 (registers 87/88) is a good choice. You won't lose any dynamic range by doing that.  
Line 162: Line 206:
With the black offset of 2047 on the CMV12K, the black level ends up at around 150, so you lose a whooping 0.05 bits from the 12-bit range.
With the black offset of 2047 on the CMV12K, the black level ends up at around 150, so you lose a whooping 0.05 bits from the 12-bit range.


Note: for easier processing, raw2dng shifts the raw data in order to fix the black level at 128.
'''Note:''' for easier processing, raw2dng shifts the raw data in order to fix the black level at 128.


=====Row noise correction from black columns=====
 
----
 
 
=====Row Noise Correction from Black Columns=====


Discussed in detail on the [[Pattern Noise]] page.
Discussed in detail on the [[Pattern Noise]] page.


===Proposed calibration pipeline===


# use black reference columns to find the black levels for odd and even rows
----
# subtract dark offset and dark current
 
# use variations in black reference columns to reduce row noise
 
=Checking Black Level=
 
You may wonder: After all these corrections, did we get the right black level? Can we render shadow detail correctly?
 
A possible criteria for checking: if we have two images of the same scene, taken with different exposures, we should be able to match those in postprocessing by simply dragging the exposure slider. For example, if we have one image at 5ms and another one at 20ms, we would set the exposure to +2 EV on the first image - the results should be pretty much identical, except for noise (and clipped highlights, if we weren't careful with the exposure).
 
Of course, that will work once we know the sensor [[Response Curves]]. But it doesn't hurt checking, I think.
 
To check how well the images can be exposure-compensated, we could evaluate (and minimize) one of those error metrics:
 
<pre style="white-space: pre-wrap">a  = dark_image - black_level;
b  = bright_image - black_level;
e1 = norm(a*expo_ratio - b);
e2 = abs(median((b ./ a)(:)) - expo_ratio)
e3 = mad(log2((b ./ a)(:)));
</pre>
 
First metric checks the difference between the two images, where the darkest one was adjusted by scaling (gain) to match the brightest one. Second metric computes the ratio between the two images at each pixel, and checks its median value vs the expected value. Third one checks the variation of per-pixel ratios between the two images, in stops, ignoring the expected value - this metric could be useful if we suspect the exposure controls may not be accurate.
 
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/black_check.png
 
With all 3 methods, minimization indicates a black level of around 120-122 (expected 128), so there's still something missing with our calibration. This is confirmed by missing details in very dark areas (crushed blacks) on some sample images.
 
On the same test image, the response curve [[Response_Curves#Curves_from_grayscale_IT8_reference_data|estimated from the grayscale IT8 reference data]] indicates a black level of 129.
 
On the same test image, the response curve [[Response_Curves#Robertson02_results|estimated with the Robertson02 algorithm]] indicates a black level of 124.
 
Who is right?
 
The sensor also has a strange behavior - something we have called the [[Response_Curves#.22Black_Hole.22_anomaly|"Black Hole" anomaly]]: in very dark areas, sensor output decreases (!) with exposure time. This might give a clue for solving this mystery.
 
http://files.apertus.org/AXIOM-Beta/snapshots/response-curves/blackhole.jpg http://files.apertus.org/AXIOM-Beta/snapshots/response-curves/blackhole.png
 
 
----
 
 
=Proposed Calibration Pipeline=
 
# Use black reference columns to find the black levels for odd and even rows.
# Subtract dark offset and dark current.
# Use variations in black reference columns to reduce row noise.


These operations should be simple enough to be implemented in FPGA, as real-time corrections.
These operations should be simple enough to be implemented in FPGA, as real-time corrections.
Line 178: Line 267:
Detailed proposal on the [[Raw preprocessing]] page.
Detailed proposal on the [[Raw preprocessing]] page.


=Calibration procedure=


====Performing the calibration====
----
 


=====Quick calibration=====
=Calibration Procedure=
 
====Performing the Calibration====
 
 
----
 
 
=====Quick Calibration=====


Acquire 16 dark frames, gain x1, exposures between 1 and 50 ms, save them under the 'darkframes' subdirectory, then run:   
Acquire 16 dark frames, gain x1, exposures between 1 and 50 ms, save them under the 'darkframes' subdirectory, then run:   


  raw2dng darkframes/*x1*.raw12 --calc-darkframe --swap-lines
  raw2dng darkframes/*x1*.raw12 --calc-darkframe --swap-lines


Result: darkframe-x1.pgm
Result: darkframe-x1.pgm
Line 192: Line 290:
Repeat for other gains if needed.
Repeat for other gains if needed.


=====Accurate calibration, with dark current nonuniformity=====
 
----
 
 
=====Accurate Calibration, with Dark Current Non-uniformity=====


Acquire 256 dark frames, gain x1, exposures between 1 and 64 ms in linear increments (4 images at each setting), save them under the 'darkframes' subdirectory, then run:
Acquire 256 dark frames, gain x1, exposures between 1 and 64 ms in linear increments (4 images at each setting), save them under the 'darkframes' subdirectory, then run:


  raw2dng darkframes/*x1*.raw12 --calc-dcnuframe --swap-lines
  raw2dng darkframes/*x1*.raw12 --calc-dcnuframe --swap-lines


Result: darkframe-x1.pgm and dcnuframe-x1.pgm.
Result: darkframe-x1.pgm and dcnuframe-x1.pgm.
Line 202: Line 305:
Repeat for other gains if needed.
Repeat for other gains if needed.


====Using the reference frames to correct raw12 files====


# place the calibration files in the working directory
----
# make sure your .raw12 images contain a metadata block
 
# raw2dng will recognize the dark frames and use them for correcting your image


  cp /path/to/darkframes/*-x[1-4].pgm .
====Checking the Calibration====
 
You can verify the calibration by rendering the same individual dark frames, this time corrected, to check the residuals. Or, even better, acquire a new set of dark frames at the same settings, and check those instead:
 
  raw2dng darkframes/*x1*.raw12 --check-darkframe --swap-lines
 
 
You will see the average value, pixel noise and row/column noise levels (both absolute and relative to pixel noise) for each dark frame. Example:
 
<pre style="white-space: pre-wrap">Average    : 127.48
Pixel noise : 2.48
Row noise  : 0.61 (24.6%)
Col noise  : 0.05 (2.2%)
</pre>
 
 
----
 
 
====Using Reference Frames to Correct raw12 files====
 
# Place the calibration files in the working directory.
# Make sure your .raw12 images contain a metadata block.
# raw2dng will recognize the dark frames and use them for correcting your image.
 
<pre style="white-space: pre-wrap">cp /path/to/darkframes/*-x[1-4].pgm .
  raw2dng *.raw12 --swap-lines
  raw2dng *.raw12 --swap-lines
</pre>


(note: you need --swap-lines as an workaround for an old bug introduced in the Beta, but wasn't fixed yet in the FPGA, hint, hint)
'''Note:''' You need -<code>-swap-lines</code> as a workaround for an old bug introduced in the Beta, but wasn't fixed yet in the FPGA, hint, hint.


This is not exactly useful when dealing with multiple folders, so until we'll have a better way to organize the calibration frames, you may try an alternative workflow:
This is not exactly useful when dealing with multiple folders, so until we'll have a better way to organize the calibration frames, you may try an alternative workflow:


# place the calibration files in some directory (let's call it "calibration directory")
# Place the calibration files in some directory (let's call it "calibration directory").
# make sure your .raw12 images contain a metadata block
# Make sure your .raw12 images contain a metadata block.
# use paths when passing input files raw2dng (the output files will be saved in the same directory as the input file)
# Use paths when passing input files raw2dng (the output files will be saved in the same directory as the input file).


  cd /path/to/darkframes
  <pre style="white-space: pre-wrap">cd /path/to/darkframes
  ls *.pgm
  ls *.pgm
   darkframe-x1.pgm      dcnuframe-x1.pgm      ...
   darkframe-x1.pgm      dcnuframe-x1.pgm      ...
  raw2dng /path/to/images/*.raw12 --swap-lines
  raw2dng /path/to/images/*.raw12 --swap-lines
</pre>
----


=Example=
=Example=
Line 228: Line 361:
Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).
Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).


* top left: raw sensor data (adjusted black level manually).
* top right: corrected with dark frame, scalar dark current, no black columns.
* bottom left: corrected with dark frame, dark current frame, no black columns
* bottom right: corrected with dark frame, dark current frame, black columns enabled
* Note: in the raw data, even and odd rows have different black offsets; that's why we have wrong colors.


* Left: raw sensor data (adjusted black level manually).
http://files.apertus.org/AXIOM-Beta/snapshots/pattern-noise/10ms+4-totally-raw-crop.jpg  http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/10ms+4-no-blackcol-crop.jpg
* Right: after dark frame and dark current subtraction, but without correction from black columns.
* Note: in the raw data, even and odd rows have different black offsets; that's why we have wrong colors.


http://files.apertus.org/AXIOM-Beta/snapshots/pattern-noise/10ms+4-totally-raw-crop.jpg http://files.apertus.org/AXIOM-Beta/snapshots/pattern-noise/10ms+4-no-blackcol-crop.jpg
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/10ms+4-darkcurrent-no-blackcol-crop.jpg http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-tests/10ms+4-darkcurrent-crop.jpg

Latest revision as of 05:24, 19 March 2018

[WIP]

1 What does this mean?

Finding the sensor output value, per pixel, in the absence of any illumination.

It covers:

  • Dark frame subtraction.
  • dark current compensation.
  • using black reference columns (called "optical black" by other manufacturers) to find the black level and fine-tune static offsets.

Note: black reference columns can be used to reduce row noise as well. See Pattern Noise.




2 Calibration Methods

2.1 Dark Frame Subtraction

This is a basic technique: take a picture with the lens cap on, and subtract it from your image. To make really sure no light is reaching the sensor, also cover the entire camera with something.




2.1.1 How many dark frames?

Problem: if you take only one dark frame, it will also contain read noise (assummed to be Gaussian and uncorrelated with the read noise from other frames). Therefore, subtracting only one image will actually increase the noise in the final output, by sqrt(2).

Solution: use a master dark frame, averaged from many images.

How many?

If you take N images, the signal will be multiplied by N, and the noise will be multiplied by sqrt(N). Therefore, the SNR will increase by log2(sqrt(N)) stops.

So, 16 dark frames will reduce the read noise in the dark frame by 2 stops, 64 frames by 3 stops, and 256 frames by 4 stops.

Okay, but how much noise will be added to the output image?

Let's say the read noise stdev in one image is r, so a dark frame averaged from N frames will have noise stdev = r/sqrt(N). Therefore, the noise in the output image will be: sqrt(r^2 + r^2/N) = r * sqrt(1 + 1/N).

Check it in octave:

octave:1>  N = 4;
 octave:2>  a = randn(1,1000000);    # one Gaussian noise sample, with mean=0 and stdev=1
 octave:3>  b = randn(N,1000000);    # N noise samples
 octave:4>  std(a + mean(b))         # add one noise sample to N averaged noise samples
 ans =  1.1174
 octave:5>  sqrt(1 + 1/4)            # compare with the theoretical result
 ans =  1.1180

So, it seems that averaging a small number of dark frames will not introduce significant noise in your images (4 should be enough if you are in a hurry, and 16 should give a very good result).

Example: one dark frame at 6ms x1 (same image as above), vs 4 darkframes at 6ms, averaged.

blackframes-gainx1-offset2047-5ms-01.jpg darkavg-5ms.jpg




2.1.2 What About Camera Settings?

Unfortunately, dark frames depend on many camera settings: analog gain (ISO), exposure, other sensor settings like offset, black sun protection, PLR configuration and so on. Temperature is a variable as well.

Luckily, the dependence on exposure appears to be linear, so we can take calibration frames at various exposures, combine them into a single dark frame, adjust it for the dark current and use it for the entire range of exposure settings (hopefully). We'll discuss that in the next section.




2.2 Dark Current

Let's look at some dark frames: gain x1, exposures 1.2ms, 6ms and 78ms. Notice they get brighter as exposure increase.

blackframes-gainx1-offset2047-1ms-01.jpg blackframes-gainx1-offset2047-5ms-01.jpg blackframes-gainx1-offset2047-64ms-01.jpg


The overall brightness in the dark frame changes with exposure in a linear fashion. We'll try to account for this in two ways: with a simple scalar value, and with a per-pixel correction.

The values in the black reference columns do not appear to compensate for the dark current, so we'll need to do it ourselves.

By identifying the dark current, we will be able to compute a dark frame that is applicable to any usual exposure time, but we are going to store only one or two reference frames for each gain. First reference frame will be called a bias frame (a zero-length exposure, that would contain only static black offsets), and the second reference frame, if used, will be called a dark current frame.

Dark current cannot be fully corrected because, while its stationary value can be measured and subtracted, it also introduces photon noise. Roger Clark explains it better:

"Dark current is temperature dependent and most modern CMOS digital cameras, circa 2008 and later have on sensor dark current subtraction, but while the dark current level is subtracted, the noise from the dark current still accumulates." [1].




2.2.1 Simple Correction

Experimentally, we have found the dark frame changes with exposure at roughly 0.065 digital units for each ms. This value is multiplied by analog gain. To find this value, take the dark frames at different exposure times (say 1...50 ms), then do a linear fit for the frame average (or median).

Command-line:

raw2dng *x1*.raw12 --calc-darkframe

Example: A dark frame created from 256 exposures, between 1.2ms and 77ms, without using black reference columns. The image was adjusted (with a constant offset) to match a zero-length exposure, so calling it bias frame may be a good idea. If we use it to correct the individual dark frames, we should no longer see a variation in overall brightness.

Averaged 256 frames exposed from 1.19 to 76.79 ms.
 Dark current: 0.0653 DN/ms

Image sequence: bias frame, single dark frames at exposures 1.2, 6 and 77 ms, corrected with the bias frame and the (scalar) dark current average. All files scaled to show a range of 60 DN, but the master dark frame has a different offset than the others.

darkframe-x1-no-blackcol-256.jpg blackframes-gainx1-offset2047-1ms-01-simple-darkframe-no-blackcol.jpg blackframes-gainx1-offset2047-5ms-01-simple-darkframe-no-blackcol.jpg blackframes-gainx1-offset2047-64ms-01-simple-darkframe-no-blackcol.jpg




2.2.2 Dark Current Non-uniformity Correction

"One common use of bias frames is for scaling dark frames. By subtracting a bias frame from a dark frame, you end up with a “thermal frame.” A thermal frame contains pixel values showing just the effect of dark current. Because dark current in any given pixel accumulates at a constant rate, a thermal frame allows you to predict with reasonable accuracy how much dark current there would be for different length exposures. However, given the opportunity, you’re always better off taking dark frames that match the exposure times of your light frames." [2]

"Dark current non-uniformity is a noise that results from the fact that each pixel generates a slightlydifferent amount of dark current. This noise can be eliminated by subtracting a dark reference frame from each image. The dark reference frame should be taken at the same temperature and with the same integration time as the image." [3]

"Dark noise is not random; in fact, it is highly repeatable. A given photosite on a sensor will accumulate almost exactly the same amount of dark noise from one exposure to the next, as long as temperature and exposure duration do not vary." [4]

So, rather than using a single scalar value (0.06 dn/ms/gain) for all pixels, we can try finding the individual dark current for each pixel. Instead of doing a linear fit on the overall dark frame brightness (vs exposure), we will do the linear fit per pixel. We'll have to acquire a lot more dark frames to compute a good result, but it might be worth the trouble.


Command-line:

raw2dng *x1*.raw12 --calc-dcnuframe


Example: Bias frame (static offset) and dark current frame (exposure-dependent offset). Notice the bias frame looks quite similar to the previous one, but a little darker. The median value of the dark current frame is, unsurprisingly, 0.0645 DN/ms.

darkframe-x1-no-blackcol-darkcurrent-256.jpg dcnuframe-x1-no-blackcol-darkcurrent-256.jpg


Individual dark frames (1.2, 6 and 77 ms) adjusted with dark current non-uniformity:

blackframes-gainx1-offset2047-1ms-01-darkcurrent-no-blackcol.jpg blackframes-gainx1-offset2047-5ms-01-darkcurrent-no-blackcol.jpg blackframes-gainx1-offset2047-64ms-01-darkcurrent-no-blackcol.jpg


No obvious improvement in the test images, so why bother?

Let's correct all these 256 dark frames and check a few indicators: median, stdev, row noise and column noise.

  • Dark frame + scalar dark current: [5]
  • Dark frame + dark current frame: [6]


You may notice:

  • Median (black level) variation: Noticeable improvement with the second method.
  • stdev (overall noise): Minor improvement at extreme settings.
  • Row noise: Identical with both methods.
  • Column noise: Small improvement with the second method.

Indeed, the more complex method appears just a tiny bit better than the simpler one.

>>>> TODO: Nicer graph




2.2.3 Dark Current Measurement from Hot Pixels

A very interesting idea can be found in [7], where hot pixels can be used to measure the amount of dark current and scale it properly. This will probably account for changes in temperature, and may work at very long exposures without actually having to calibrate the camera in these conditions. Genius, if you ask me.

To be studied.




2.3 Black Reference Columns

This sensor has 8+8 columns that can be used for calibrating the black levels; they are also useful for reducing the dynamic row noise.

Experimentally, we have noticed that odd rows have slightly different statistics (noise level, offset, gain), compared to even rows. This happens in both the black columns and the active area, and it may indicate two parallel circuits used for readout, each having slightly different electrical response.

Therefore, it may be wise to process the black columns for odd and even rows separately, which should already fix some issues like static row noise, or the need for green equilibration.

In raw2dng this correction is enabled by default, as long as you use a dark frame. You can turn it off with --no-blackcol if you want.




2.3.1 Black Level

The sensor has two registers that can be used to adjust the black level: one for odd rows, another for even rows. This confirms our finding about two parallel readout circuits.

You might be tempted to adjust the black level to 0 (like Nikon does). Please don't. Here's why:

  • If you adjust the offset until the black level becomes roughly 0, you will clip all the data below this level. Good luck subtracting a dark frame after that.
  • Even if you change the level after doing all the black corrections, you may still have useful data below zero. You will need it when stacking multiple frames, or when doing noise reduction.
  • Just FYI, there is a hack for Nikon cameras that moves the black level above 0 [8]. Sample images taken with the hack: [9] [10]

I recommend setting the offset so that only a few isolated pixels (if any) reach the value of 0. A black offset of 2047 (registers 87/88) is a good choice. You won't lose any dynamic range by doing that.

On most recent Canon DSLRs, black level is 2048. That's a little on the large side, but it's a good thing. Some may argue that you may lose 2 stops or more of dynamic range by doing that [11], but this is wrong. On Canons, the raw output is 14-bit, so by setting the offset to 2048 instead of 0, the useful range will be "just" log2(16384-2048) = 13.8 bits, instead of 14. So, yeah, you lose 0.2 bits from the ADC range.

With the black offset of 2047 on the CMV12K, the black level ends up at around 150, so you lose a whooping 0.05 bits from the 12-bit range.

Note: for easier processing, raw2dng shifts the raw data in order to fix the black level at 128.




2.3.2 Row Noise Correction from Black Columns

Discussed in detail on the Pattern Noise page.




3 Checking Black Level

You may wonder: After all these corrections, did we get the right black level? Can we render shadow detail correctly?

A possible criteria for checking: if we have two images of the same scene, taken with different exposures, we should be able to match those in postprocessing by simply dragging the exposure slider. For example, if we have one image at 5ms and another one at 20ms, we would set the exposure to +2 EV on the first image - the results should be pretty much identical, except for noise (and clipped highlights, if we weren't careful with the exposure).

Of course, that will work once we know the sensor Response Curves. But it doesn't hurt checking, I think.

To check how well the images can be exposure-compensated, we could evaluate (and minimize) one of those error metrics:

a  = dark_image - black_level;
 b  = bright_image - black_level;
 e1 = norm(a*expo_ratio - b);
 e2 = abs(median((b ./ a)(:)) - expo_ratio)
 e3 = mad(log2((b ./ a)(:)));

First metric checks the difference between the two images, where the darkest one was adjusted by scaling (gain) to match the brightest one. Second metric computes the ratio between the two images at each pixel, and checks its median value vs the expected value. Third one checks the variation of per-pixel ratios between the two images, in stops, ignoring the expected value - this metric could be useful if we suspect the exposure controls may not be accurate.

black_check.png

With all 3 methods, minimization indicates a black level of around 120-122 (expected 128), so there's still something missing with our calibration. This is confirmed by missing details in very dark areas (crushed blacks) on some sample images.

On the same test image, the response curve estimated from the grayscale IT8 reference data indicates a black level of 129.

On the same test image, the response curve estimated with the Robertson02 algorithm indicates a black level of 124.

Who is right?

The sensor also has a strange behavior - something we have called the "Black Hole" anomaly: in very dark areas, sensor output decreases (!) with exposure time. This might give a clue for solving this mystery.

blackhole.jpg blackhole.png




4 Proposed Calibration Pipeline

  1. Use black reference columns to find the black levels for odd and even rows.
  2. Subtract dark offset and dark current.
  3. Use variations in black reference columns to reduce row noise.

These operations should be simple enough to be implemented in FPGA, as real-time corrections.

Detailed proposal on the Raw preprocessing page.




5 Calibration Procedure

5.1 Performing the Calibration



5.1.1 Quick Calibration

Acquire 16 dark frames, gain x1, exposures between 1 and 50 ms, save them under the 'darkframes' subdirectory, then run:

raw2dng darkframes/*x1*.raw12 --calc-darkframe --swap-lines


Result: darkframe-x1.pgm

Repeat for other gains if needed.




5.1.2 Accurate Calibration, with Dark Current Non-uniformity

Acquire 256 dark frames, gain x1, exposures between 1 and 64 ms in linear increments (4 images at each setting), save them under the 'darkframes' subdirectory, then run:

raw2dng darkframes/*x1*.raw12 --calc-dcnuframe --swap-lines


Result: darkframe-x1.pgm and dcnuframe-x1.pgm.

Repeat for other gains if needed.




5.2 Checking the Calibration

You can verify the calibration by rendering the same individual dark frames, this time corrected, to check the residuals. Or, even better, acquire a new set of dark frames at the same settings, and check those instead:

raw2dng darkframes/*x1*.raw12 --check-darkframe --swap-lines


You will see the average value, pixel noise and row/column noise levels (both absolute and relative to pixel noise) for each dark frame. Example:

Average     : 127.48
 Pixel noise : 2.48
 Row noise   : 0.61 (24.6%)
 Col noise   : 0.05 (2.2%)




5.3 Using Reference Frames to Correct raw12 files

  1. Place the calibration files in the working directory.
  2. Make sure your .raw12 images contain a metadata block.
  3. raw2dng will recognize the dark frames and use them for correcting your image.
cp /path/to/darkframes/*-x[1-4].pgm .
 raw2dng *.raw12 --swap-lines


Note: You need --swap-lines as a workaround for an old bug introduced in the Beta, but wasn't fixed yet in the FPGA, hint, hint.

This is not exactly useful when dealing with multiple folders, so until we'll have a better way to organize the calibration frames, you may try an alternative workflow:

  1. Place the calibration files in some directory (let's call it "calibration directory").
  2. Make sure your .raw12 images contain a metadata block.
  3. Use paths when passing input files raw2dng (the output files will be saved in the same directory as the input file).
cd /path/to/darkframes
 ls *.pgm
   darkframe-x1.pgm       dcnuframe-x1.pgm       ...
 raw2dng /path/to/images/*.raw12 --swap-lines




6 Example

Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).

  • top left: raw sensor data (adjusted black level manually).
  • top right: corrected with dark frame, scalar dark current, no black columns.
  • bottom left: corrected with dark frame, dark current frame, no black columns
  • bottom right: corrected with dark frame, dark current frame, black columns enabled
  • Note: in the raw data, even and odd rows have different black offsets; that's why we have wrong colors.

10ms+4-totally-raw-crop.jpg 10ms+4-no-blackcol-crop.jpg

10ms+4-darkcurrent-no-blackcol-crop.jpg 10ms+4-darkcurrent-crop.jpg