Finding the sensor output value, per pixel, in the absence of any illumination.
Note: black reference columns can be used to reduce row noise as well. See Pattern Noise.
This is a basic technique: take a picture with the lens cap on, and subtract it from your image. To make really sure no light is reaching the sensor, also cover the entire camera with something.
Problem: if you take only one dark frame, it will also contain read noise (assummed to be Gaussian and uncorrelated with the read noise from other frames). Therefore, subtracting only one image will actually increase the noise in the final output, by sqrt(2).
Solution: use a master dark frame, averaged from many images.
If you take N images, the signal will be multiplied by N, and the noise will be multiplied by sqrt(N). Therefore, the SNR will increase by log2(sqrt(N)) stops.
So, 16 dark frames will reduce the read noise in the dark frame by 2 stops, 64 frames by 3 stops, and 256 frames by 4 stops.
Okay, but how much noise will be added to the output image?
Let's say the read noise stdev in one image is r, so a dark frame averaged from N frames will have noise stdev = r/sqrt(N). Therefore, the noise in the output image will be: sqrt(r^2 + r^2/N) = r * sqrt(1 + 1/N).
Check it in octave:
octave:1> N = 4; octave:2> a = randn(1,1000000); # one Gaussian noise sample, with mean=0 and stdev=1 octave:3> b = randn(N,1000000); # N noise samples octave:4> std(a + mean(b)) # add one noise sample to N averaged noise samples ans = 1.1174 octave:5> sqrt(1 + 1/4) # compare with the theoretical result ans = 1.1180
So, it seems that averaging a small number of dark frames will not introduce significant noise in your images (4 should be enough if you are in a hurry, and 16 should give a very good result).
Example: one dark frame at 6ms x1 (same image as above), vs 4 darkframes at 6ms, averaged.
Unfortunately, dark frames depend on many camera settings: analog gain (ISO), exposure, other sensor settings like offset, black sun protection, PLR configuration and so on. Temperature is a variable as well.
Luckily, the dependence on exposure appears to be linear, so we can take calibration frames at various exposures, combine them into a single dark frame, adjust it for the dark current and use it for the entire range of exposure settings (hopefully). We'll discuss that in the next section.
Let's look at some dark frames: gain x1, exposures 1.2ms, 6ms and 78ms. Notice they get brighter as exposure increase.
The overall brightness in the dark frame changes with exposure in a linear fashion. We'll try to account for this in two ways: with a simple scalar value, and with a per-pixel correction.
The values in the black reference columns do not appear to compensate for the dark current, so we'll need to do it ourselves.
By identifying the dark current, we will be able to compute a dark frame that is applicable to any usual exposure time, but we are going to store only one or two reference frames for each gain. First reference frame will be called a bias frame (a zero-length exposure, that would contain only static black offsets), and the second reference frame, if used, will be called a dark current frame.
Dark current cannot be fully corrected because, while its stationary value can be measured and subtracted, it also introduces photon noise. Roger Clark explains it better:
"Dark current is temperature dependent and most modern CMOS digital cameras, circa 2008 and later have on sensor dark current subtraction, but while the dark current level is subtracted, the noise from the dark current still accumulates." .
Experimentally, we have found the dark frame changes with exposure at roughly 0.065 digital units for each ms. This value is multiplied by analog gain. To find this value, take the dark frames at different exposure times (say 1...50 ms), then do a linear fit for the frame average (or median).
raw2dng *x1*.raw12 --calc-darkframe
Example: A dark frame created from 256 exposures, between 1.2ms and 77ms, without using black reference columns. The image was adjusted (with a constant offset) to match a zero-length exposure, so calling it bias frame may be a good idea. If we use it to correct the individual dark frames, we should no longer see a variation in overall brightness.
Averaged 256 frames exposed from 1.19 to 76.79 ms. Dark current: 0.0653 DN/ms
Image sequence: bias frame, single dark frames at exposures 1.2, 6 and 77 ms, corrected with the bias frame and the (scalar) dark current average. All files scaled to show a range of 60 DN, but the master dark frame has a different offset than the others.
"One common use of bias frames is for scaling dark frames. By subtracting a bias frame from a dark frame, you end up with a “thermal frame.” A thermal frame contains pixel values showing just the effect of dark current. Because dark current in any given pixel accumulates at a constant rate, a thermal frame allows you to predict with reasonable accuracy how much dark current there would be for different length exposures. However, given the opportunity, you’re always better off taking dark frames that match the exposure times of your light frames." 
"Dark current non-uniformity is a noise that results from the fact that each pixel generates a slightlydifferent amount of dark current. This noise can be eliminated by subtracting a dark reference frame from each image. The dark reference frame should be taken at the same temperature and with the same integration time as the image." 
"Dark noise is not random; in fact, it is highly repeatable. A given photosite on a sensor will accumulate almost exactly the same amount of dark noise from one exposure to the next, as long as temperature and exposure duration do not vary." 
So, rather than using a single scalar value (0.06 dn/ms/gain) for all pixels, we can try finding the individual dark current for each pixel. Instead of doing a linear fit on the overall dark frame brightness (vs exposure), we will do the linear fit per pixel. We'll have to acquire a lot more dark frames to compute a good result, but it might be worth the trouble.
raw2dng *x1*.raw12 --calc-dcnuframe
Example: Bias frame (static offset) and dark current frame (exposure-dependent offset). Notice the bias frame looks quite similar to the previous one, but a little darker. The median value of the dark current frame is, unsurprisingly, 0.0645 DN/ms.
Individual dark frames (1.2, 6 and 77 ms) adjusted with dark current non-uniformity:
No obvious improvement in the test images, so why bother?
Let's correct all these 256 dark frames and check a few indicators: median, stdev, row noise and column noise.
You may notice:
Indeed, the more complex method appears just a tiny bit better than the simpler one.
>>>> TODO: Nicer graph
A very interesting idea can be found in , where hot pixels can be used to measure the amount of dark current and scale it properly. This will probably account for changes in temperature, and may work at very long exposures without actually having to calibrate the camera in these conditions. Genius, if you ask me.
To be studied.
This sensor has 8+8 columns that can be used for calibrating the black levels; they are also useful for reducing the dynamic row noise.
Experimentally, we have noticed that odd rows have slightly different statistics (noise level, offset, gain), compared to even rows. This happens in both the black columns and the active area, and it may indicate two parallel circuits used for readout, each having slightly different electrical response.
Therefore, it may be wise to process the black columns for odd and even rows separately, which should already fix some issues like static row noise, or the need for green equilibration.
raw2dng this correction is enabled by default, as long as you use a dark frame. You can turn it off with
--no-blackcol if you want.
The sensor has two registers that can be used to adjust the black level: one for odd rows, another for even rows. This confirms our finding about two parallel readout circuits.
You might be tempted to adjust the black level to 0 (like Nikon does). Please don't. Here's why:
I recommend setting the offset so that only a few isolated pixels (if any) reach the value of 0. A black offset of 2047 (registers 87/88) is a good choice. You won't lose any dynamic range by doing that.
On most recent Canon DSLRs, black level is 2048. That's a little on the large side, but it's a good thing. Some may argue that you may lose 2 stops or more of dynamic range by doing that , but this is wrong. On Canons, the raw output is 14-bit, so by setting the offset to 2048 instead of 0, the useful range will be "just" log2(16384-2048) = 13.8 bits, instead of 14. So, yeah, you lose 0.2 bits from the ADC range.
With the black offset of 2047 on the CMV12K, the black level ends up at around 150, so you lose a whooping 0.05 bits from the 12-bit range.
Note: for easier processing, raw2dng shifts the raw data in order to fix the black level at 128.
Discussed in detail on the Pattern Noise page.
You may wonder: After all these corrections, did we get the right black level? Can we render shadow detail correctly?
A possible criteria for checking: if we have two images of the same scene, taken with different exposures, we should be able to match those in postprocessing by simply dragging the exposure slider. For example, if we have one image at 5ms and another one at 20ms, we would set the exposure to +2 EV on the first image - the results should be pretty much identical, except for noise (and clipped highlights, if we weren't careful with the exposure).
Of course, that will work once we know the sensor Response Curves. But it doesn't hurt checking, I think.
To check how well the images can be exposure-compensated, we could evaluate (and minimize) one of those error metrics:
a = dark_image - black_level; b = bright_image - black_level; e1 = norm(a*expo_ratio - b); e2 = abs(median((b ./ a)(:)) - expo_ratio) e3 = mad(log2((b ./ a)(:)));
First metric checks the difference between the two images, where the darkest one was adjusted by scaling (gain) to match the brightest one. Second metric computes the ratio between the two images at each pixel, and checks its median value vs the expected value. Third one checks the variation of per-pixel ratios between the two images, in stops, ignoring the expected value - this metric could be useful if we suspect the exposure controls may not be accurate.
With all 3 methods, minimization indicates a black level of around 120-122 (expected 128), so there's still something missing with our calibration. This is confirmed by missing details in very dark areas (crushed blacks) on some sample images.
On the same test image, the response curve estimated from the grayscale IT8 reference data indicates a black level of 129.
On the same test image, the response curve estimated with the Robertson02 algorithm indicates a black level of 124.
Who is right?
The sensor also has a strange behavior - something we have called the "Black Hole" anomaly: in very dark areas, sensor output decreases (!) with exposure time. This might give a clue for solving this mystery.
These operations should be simple enough to be implemented in FPGA, as real-time corrections.
Detailed proposal on the Raw preprocessing page.
Acquire 16 dark frames, gain x1, exposures between 1 and 50 ms, save them under the 'darkframes' subdirectory, then run:
raw2dng darkframes/*x1*.raw12 --calc-darkframe --swap-lines
Repeat for other gains if needed.
Acquire 256 dark frames, gain x1, exposures between 1 and 64 ms in linear increments (4 images at each setting), save them under the 'darkframes' subdirectory, then run:
raw2dng darkframes/*x1*.raw12 --calc-dcnuframe --swap-lines
Result: darkframe-x1.pgm and dcnuframe-x1.pgm.
Repeat for other gains if needed.
You can verify the calibration by rendering the same individual dark frames, this time corrected, to check the residuals. Or, even better, acquire a new set of dark frames at the same settings, and check those instead:
raw2dng darkframes/*x1*.raw12 --check-darkframe --swap-lines
You will see the average value, pixel noise and row/column noise levels (both absolute and relative to pixel noise) for each dark frame. Example:
Average : 127.48 Pixel noise : 2.48 Row noise : 0.61 (24.6%) Col noise : 0.05 (2.2%)
cp /path/to/darkframes/*-x[1-4].pgm . raw2dng *.raw12 --swap-lines
Note: You need -
-swap-lines as a workaround for an old bug introduced in the Beta, but wasn't fixed yet in the FPGA, hint, hint.
This is not exactly useful when dealing with multiple folders, so until we'll have a better way to organize the calibration frames, you may try an alternative workflow:
cd /path/to/darkframes ls *.pgm darkframe-x1.pgm dcnuframe-x1.pgm ... raw2dng /path/to/images/*.raw12 --swap-lines
Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).