Pattern Noise

General info about pattern noise: http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/#patternnoise

The CMV12000 sensor suffers from dynamic row noise.

That means, a scalar offset gets added to each row. The offset is not correlated between different frames, so we can't remove it using a calibration frame (dark frame or whatever).

One can observe this noise by looking at the difference between two images taken at identical settings. There are two main components that appear obvious in such a difference frame: random noise (per pixel, increases on brighter pixels) and row noise (per line).

There are two ways to deal with this noise, after performing Black Calibration:

• use info from black reference columns to reduce dynamic row noise without guessing anything (fast, can be implemented in real-time, see Raw_preprocessing)
• use denoising techniques to reduce the remaining row noise (the guesswork part, slow)

1 Reducing row noise using black reference columns

http://github.com/apertus-open-source-cinema/misc-tools-utilities/commit/48de47b2a544dc32bbd5a8fd7701bb44a31ea850#diff-624053a553f49c0036b4d31282e58b2fR301

The application note AN01 from CMOSIS says:

"The noise is also present in the black reference columns (8 left and 8 right), so when enabled (reg 89[15] = 1), these can be used for row noise correction by for example making a relative row profile of these black columns and subtract this from the image."

However, simply subtracting each row average of the black columns from our image is not going to work. Here's why:

Kalman filter theory: http://robocup.mi.fu-berlin.de/buch/kalman.pdf

From page 3, if we know how noisy our estimations are, the optimal weights are inversely proportional with the noise variances:

x_optimal = (x1 * var(x2) + x2 * var(x1)) / (var(x1) + var(x2))

Here, let's say R = x1 is row noise (stdev = 1.6 at gain=x1) and x2 is black column noise.

R = x1
B = mean(black_col') = R + x2 =>  x2 = B - R
x2 can be estimated as mean(black_col') - mean(active_area')
stdev(x2) = 1.3.

We want to find k that minimizes var(R - k*B).

var(R - k*B) = var(x1 * (1-k) - x2 * k),
=> k = var(x1)) / (var(x1) + var(x2).

In particular, for gain = x1, k = 1.6^2 / (1.6^2 + 1.3^2) = 0.6.

So, we don't have to simply subtract the black columns. Rather, we'll subtract the static offset (median value) first, and then, we'll subtract the remaining variations multiplied by 0.6 at gain=x1.

Things get a little more complex because the static offset is different on odd and even rows, and it also appears to change from the left side to right side of the frame. More details on the Raw preprocessing page.

2 Reducing the remaining row noise by image filtering

Basic algorithm:

1. Filter the image with an edge-aware vertical blur (bilateral filter on pixels from the same column)
2. Subtract the blurred image; the residuals will reveal the row noise (see example)
3. Mask out highlights and strong edges
4. Take the median value from each row of the residuals image
5. Subtract these values from each row of the original image

Source: patternnoise.c

3 Usage

The methods discussed here are implemented in raw2dng.

• black reference columns are used by default, as long as you use a dark frame (since this method is fast and has no side effects)
• to reduce the remaining row noise, use raw2dng --fixrn
• if the image also suffers from column noise, use raw2dng --fixpn

Tip: the algorithm for filtering row noise is also available in mlvfs, so you can use it on MLV videos (recorded with Magic Lantern) as well.

4 Example

Showing half-res image crops pushed by 4 stops (ufraw-batch --wb=auto --exposure=4 --shrink=2).

• Left: raw sensor data (adjusted black level manually).
• Right: after dark frame and dark current subtraction, but without correction from black columns.
• Note: in the raw data, even and odd rows have different black offsets; that's why we have wrong colors.

---

• Left: after dark frame, dark current and black reference columns correction.
• Right: after row noise reduction (--fixrn).

---

Algorithm internals:

• Left: filtered image (vertical blur using a bilateral filter)
• Right: noise image, revealing row noise. Black regions are from edges that were masked.

---

Downsized images:

Corrected with dark frame and dark current only:

Also corrected with black columns:

Also corrected with --fixrn:

Larger images (half-res):

• 10ms+4-no-blackcol.jpg
• 10ms+4.jpg
• 10ms+4-fixrn.jpg

All files used for this test, including scripts, calibration frames and uncompressed images, can be found here: http://files.apertus.org/AXIOM-Beta/snapshots/pattern-noise/

In particular, you might be interested in:

• Raw12 image: 10ms.raw12
• Calibration frames: darkframe-x1.pgm and dcnuframe-x1.pgm
• Final image: 10ms.DNG
• Testing script: fpntest.sh
• Script output: fpntest.log