Black Calibration

[WIP]

1 What does this mean?

Finding the sensor output value, per pixel, in the absence of any illumination.

It covers:

• Dark frame subtraction.
• dark current compensation.
• using black reference columns (called "optical black" by other manufacturers) to find the black level and fine-tune static offsets.

Note: black reference columns can be used to reduce row noise as well. See Pattern Noise.

---

2 Calibration Methods

2.1 Dark Frame Subtraction

This is a basic technique: take a picture with the lens cap on, and subtract it from your image. To make really sure no light is reaching the sensor, also cover the entire camera with something.

---

2.1.1 How many dark frames?

Problem: if you take only one dark frame, it will also contain read noise (assummed to be Gaussian and uncorrelated with the read noise from other frames). Therefore, subtracting only one image will actually increase the noise in the final output, by sqrt(2).

Solution: use a master dark frame, averaged from many images.

How many?

If you take N images, the signal will be multiplied by N, and the noise will be multiplied by sqrt(N). Therefore, the SNR will increase by log2(sqrt(N)) stops.

So, 16 dark frames will reduce the read noise in the dark frame by 2 stops, 64 frames by 3 stops, and 256 frames by 4 stops.

Okay, but how much noise will be added to the output image?

Let's say the read noise stdev in one image is r, so a dark frame averaged from N frames will have noise stdev = r/sqrt(N). Therefore, the noise in the output image will be: sqrt(r^2 + r^2/N) = r * sqrt(1 + 1/N).

Check it in octave:

octave:1>  N = 4;
 octave:2>  a = randn(1,1000000);    # one Gaussian noise sample, with mean=0 and stdev=1
 octave:3>  b = randn(N,1000000);    # N noise samples
 octave:4>  std(a + mean(b))         # add one noise sample to N averaged noise samples
 ans =  1.1174
 octave:5>  sqrt(1 + 1/4)            # compare with the theoretical result
 ans =  1.1180

So, it seems that averaging a small number of dark frames will not introduce significant noise in your images (4 should be enough if you are in a hurry, and 16 should give a very good result).

Example: one dark frame at 6ms x1 (same image as above), vs 4 darkframes at 6ms, averaged.

---

2.1.2 What About Camera Settings?

Unfortunately, dark frames depend on many camera settings: analog gain (ISO), exposure, other sensor settings like offset, black sun protection, PLR configuration and so on. Temperature is a variable as well.

Luckily, the dependence on exposure appears to be linear, so we can take calibration frames at various exposures, combine them into a single dark frame, adjust it for the dark current and use it for the entire range of exposure settings (hopefully). We'll discuss that in the next section.

---

2.2 Dark Current

Let's look at some dark frames: gain x1, exposures 1.2ms, 6ms and 78ms. Notice they get brighter as exposure increase.

The overall brightness in the dark frame changes with exposure in a linear fashion. We'll try to account for this in two ways: with a simple scalar value, and with a per-pixel correction.

The values in the black reference columns do not appear to compensate for the dark current, so we'll need to do it ourselves.

By identifying the dark current, we will be able to compute a dark frame that is applicable to any usual exposure time, but we are going to store only one or two reference frames for each gain. First reference frame will be called a bias frame (a zero-length exposure, that would contain only static black offsets), and the second reference frame, if used, will be called a dark current frame.

Dark current cannot be fully corrected because, while its stationary value can be measured and subtracted, it also introduces photon noise. Roger Clark explains it better:

"Dark current is temperature dependent and most modern CMOS digital cameras, circa 2008 and later have on sensor dark current subtraction, but while the dark current level is subtracted, the noise from the dark current still accumulates." [1].

---

2.2.1 Simple Correction

Experimentally, we have found the dark frame changes with exposure at roughly 0.065 digital units for each ms. This value is multiplied by analog gain. To find this value, take the dark frames at different exposure times (say 1...50 ms), then do a linear fit for the frame average (or median).

Command-line:

raw2dng *x1*.raw12 --calc-darkframe

Example: A dark frame created from 256 exposures, between 1.2ms and 77ms, without using black reference columns. The image was adjusted (with a constant offset) to match a zero-length exposure, so calling it bias frame may be a good idea. If we use it to correct the individual dark frames, we should no longer see a variation in overall brightness.

Averaged 256 frames exposed from 1.19 to 76.79 ms.
 Dark current: 0.0653 DN/ms

Image sequence: bias frame, single dark frames at exposures 1.2, 6 and 77 ms, corrected with the bias frame and the (scalar) dark current average. All files scaled to show a range of 60 DN, but the master dark frame has a different offset than the others.

---

2.2.2 Dark Current Non-uniformity Correction

"One common use of bias frames is for scaling dark frames. By subtracting a bias frame from a dark frame, you end up with a “thermal frame.” A thermal frame contains pixel values showing just the effect of dark current. Because dark current in any given pixel accumulates at a constant rate, a thermal frame allows you to predict with reasonable accuracy how much dark current there would be for different length exposures. However, given the opportunity, you’re always better off taking dark frames that match the exposure times of your light frames." [2]

"Dark current non-uniformity is a noise that results from the fact that each pixel generates a slightlydifferent amount of dark current. This noise can be eliminated by subtracting a dark reference frame from each image. The dark reference frame should be taken at the same temperature and with the same integration time as the image." [3]

"Dark noise is not random; in fact, it is highly repeatable. A given photosite on a sensor will accumulate almost exactly the same amount of dark noise from one exposure to the next, as long as temperature and exposure duration do not vary." [4]

So, rather than using a single scalar value (0.06 dn/ms/gain) for all pixels, we can try finding the individual dark current for each pixel. Instead of doing a linear fit on the overall dark frame brightness (vs exposure), we will do the linear fit per pixel. We'll have to acquire a lot more dark frames to compute a good result, but it might be worth the trouble.

Command-line:

raw2dng *x1*.raw12 --calc-dcnuframe

Example: Bias frame (static offset) and dark current frame (exposure-dependent offset). Notice the bias frame looks quite similar to the previous one, but a little darker. The median value of the dark current frame is, unsurprisingly, 0.0645 DN/ms.

Individual dark frames (1.2, 6 and 77 ms) adjusted with dark current non-uniformity:

No obvious improvement in the test images, so why bother?

Let's correct all these 256 dark frames and check a few indicators: median, stdev, row noise and column noise.

• Dark frame + scalar dark current: [5]
• Dark frame + dark current frame: [6]

You may notice:

• Median (black level) variation: Noticeable improvement with the second method.
• stdev (overall noise): Minor improvement at extreme settings.
• Row noise: Identical with both methods.
• Column noise: Small improvement with the second method.

Indeed, the more complex method appears just a tiny bit better than the simpler one.

>>>> TODO: Nicer graph

---

2.2.3 Dark Current Measurement from Hot Pixels

A very interesting idea can be found in [7], where hot pixels can be used to measure the amount of dark current and scale it properly. This will probably account for changes in temperature, and may work at very long exposures without actually having to calibrate the camera in these conditions. Genius, if you ask me.

To be studied.

---

2.3 Black Reference Columns

This sensor has 8+8 columns that can be used for calibrating the black levels; they are also useful for reducing the dynamic row noise.

Experimentally, we have noticed that odd rows have slightly different statistics (noise level, offset, gain), compared to even rows. This happens in both the black columns and the active area, and it may indicate two parallel circuits used for readout, each having slightly different electrical response.

Therefore, it may be wise to process the black columns for odd and even rows separately, which should already fix some issues like static row noise, or the need for green equilibration.

In raw2dng this correction is enabled by default, as long as you use a dark frame. You can turn it off with --no-blackcol if you want.

---

2.3.1 Black Level

The sensor has two registers that can be used to adjust the black level: one for odd rows, another for even rows. This confirms our finding about two parallel readout circuits.

You might be tempted to adjust the black level to 0 (like Nikon does). Please don't. Here's why:

• If you adjust the offset until the black level becomes roughly 0, you will clip all the data below this level. Good luck subtracting a dark frame after that.
• Even if you change the level after doing all the black corrections, you may still have useful data below zero. You will need it when stacking multiple frames, or when doing noise reduction.
• Just FYI, there is a hack for Nikon cameras that moves the black level above 0 [8]. Sample images taken with the hack: [9] [10]

I recommend setting the offset so that only a few isolated pixels (if any) reach the value of 0. A black offset of 2047 (registers 87/88) is a good choice. You won't lose any dynamic range by doing that.

On most recent Canon DSLRs, black level is 2048. That's a little on the large side, but it's a good thing. Some may argue that you may lose 2 stops or more of dynamic range by doing that [11], but this is wrong. On Canons, the raw output is 14-bit, so by setting the offset to 2048 instead of 0, the useful range will be "just" log2(16384-2048) = 13.8 bits, instead of 14. So, yeah, you lose 0.2 bits from the ADC range.

With the black offset of 2047 on the CMV12K, the black level ends up at around 150, so you lose a whooping 0.05 bits from the 12-bit range.

Note: for easier processing, raw2dng shifts the raw data in order to fix the black level at 128.

---

2.3.2 Row Noise Correction from Black Columns

Discussed in detail on the Pattern Noise page.

---

3 Checking Black Level

You may wonder: After all these corrections, did we get the right black level? Can we render shadow detail correctly?

A possible criteria for checking: if we have two images of the same scene, taken with different exposures, we should be able to match those in postprocessing by simply dragging the exposure slider. For example, if we have one image at 5ms and another one at 20ms, we would set the exposure to +2 EV on the first image - the results should be pretty much identical, except for noise (and clipped highlights, if we weren't careful with the exposure).

Of course, that will work once we know the sensor Response Curves. But it doesn't hurt checking, I think.

To check how well the images can be exposure-compensated, we could evaluate (and minimize) one of those error metrics:

a  = dark_image - black_level;
 b  = bright_image - black_level;
 e1 = norm(a*expo_ratio - b);
 e2 = abs(median((b ./ a)(:)) - expo_ratio)
 e3 = mad(log2((b ./ a)(:)));

First metric checks the difference between the two images, where the darkest one was adjusted by scaling (gain) to match the brightest one. Second metric computes the ratio between the two images at each pixel, and checks its median value vs the expected value. Third one checks the variation of per-pixel ratios between the two images, in stops, ignoring the expected value - this metric could be useful if we suspect the exposure controls may not be accurate.

With all 3 methods, minimization indicates a black level of around 120-122 (expected 128), so there's still something missing with our calibration. This is confirmed by missing details in very dark areas (crushed blacks) on some sample images.

On the same test image, the response curve estimated from the grayscale IT8 reference data indicates a black level of 129.

On the same test image, the response curve estimated with the Robertson02 algorithm indicates a black level of 124.

Who is right?

The sensor also has a strange behavior - something we have called the "Black Hole" anomaly: in very dark areas, sensor output decreases (!) with exposure time. This might give a clue for solving this mystery.

---

4 Proposed Calibration Pipeline

1. Use black reference columns to find the black levels for odd and even rows.
2. Subtract dark offset and dark current.
3. Use variations in black reference columns to reduce row noise.

These operations should be simple enough to be implemented in FPGA, as real-time corrections.

Detailed proposal on the Raw preprocessing page.

---

5 Calibration Procedure

5.1 Performing the Calibration

---

5.1.1 Quick Calibration

Acquire 16 dark frames, gain x1, exposures between 1 and 50 ms, save them under the 'darkframes' subdirectory, then run:

raw2dng darkframes/*x1*.raw12 --calc-darkframe --swap-lines

Result: darkframe-x1.pgm

Repeat for other gains if needed.

---

5.1.2 Accurate Calibration, with Dark Current Non-uniformity

Acquire 256 dark frames, gain x1, exposures between 1 and 64 ms in linear increments (4 images at each setting), save them under the 'darkframes' subdirectory, then run:

raw2dng darkframes/*x1*.raw12 --calc-dcnuframe --swap-lines

Result: darkframe-x1.pgm and dcnuframe-x1.pgm.

Repeat for other gains if needed.

---

5.2 Checking the Calibration

You can verify the calibration by rendering the same individual dark frames, this time corrected, to check the residuals. Or, even better, acquire a new set of dark frames at the same settings, and check those instead:

raw2dng darkframes/*x1*.raw12 --check-darkframe --swap-lines

You will see the average value, pixel noise and row/column noise levels (both absolute and relative to pixel noise) for each dark frame. Example:

Average     : 127.48
 Pixel noise : 2.48
 Row noise   : 0.61 (24.6%)
 Col noise   : 0.05 (2.2%)

---

5.3 Using Reference Frames to Correct raw12 files

1. Place the calibration files in the working directory.
2. Make sure your .raw12 images contain a metadata block.
3. raw2dng will recognize the dark frames and use them for correcting your image.

cp /path/to/darkframes/*-x[1-4].pgm .
 raw2dng *.raw12 --swap-lines

Note: You need --swap-lines as a workaround for an old bug introduced in the Beta, but wasn't fixed yet in the FPGA, hint, hint.

This is not exactly useful when dealing with multiple folders, so until we'll have a better way to organize the calibration frames, you may try an alternative workflow:

1. Place the calibration files in some directory (let's call it "calibration directory").
2. Make sure your .raw12 images contain a metadata block.
3. Use paths when passing input files raw2dng (the output files will be saved in the same directory as the input file).

cd /path/to/darkframes
 ls *.pgm
   darkframe-x1.pgm       dcnuframe-x1.pgm       ...
 raw2dng /path/to/images/*.raw12 --swap-lines

---

6 Example

Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).

• top left: raw sensor data (adjusted black level manually).
• top right: corrected with dark frame, scalar dark current, no black columns.
• bottom left: corrected with dark frame, dark current frame, no black columns
• bottom right: corrected with dark frame, dark current frame, black columns enabled
• Note: in the raw data, even and odd rows have different black offsets; that's why we have wrong colors.