OpenCine.Nearest Neighbor and Bilinear Interpolation

Bilinear Interpolation and Nearest Neighbor Interpolation are two of the most basic demosaicing algorithms. Nearest Neighbor fills the “missing” pixels by using the value of a neighbor sensel. Bilinear Interpolation, on other hand, fills the “missing” pixels by using the average of two or four neighbor sensels. The aforementioned algorithms have a lot of artifacts, especially in edges.

A single class was used for Nearest Neighbor and for Bilinear Interpolation, due to their similarities.

Associated Files
BilinearDebayer.cpp
BilinearDebayer.h

The class contains two methods that we can choose to run the desired interpolation.

1 Constructor

The constructor takes an OCImage pointer and gets its width and height to calculate the image size, which will be important for the debayering methods of the class. Afterwards, it initializes the Red, Green, and Blue channel pointers, which will point to the OCImage’s respective channels. Finally, the constructor will get the Bayer pattern from OCImage, which will be used to call the SetPatternOffsets method.

2 Pattern Offsets

In order to correctly debayer the image, we need to set offsets for each color channel, taking in account the image’s Bayer pattern. The offsets are used to indicate in which pixels will the debayer methods work.

For the first color channel (_patternOffsets[0], Red channel for RGGB and GRBG patterns; Blue channel for the other patterns), the offset is the position of the first “missing” pixel from the color channel that is not part of the image border and is on a odd row.

For the Green channel, we have two offsets, due to the presence of two Green sensels per 2x2 blocks in the raw image. For the first offset, we use the position of the first “missing” pixel from the Green channel that is not part of the image border and is on a odd row. For the second offset, we use the position of the first “missing” pixel from the Green channel that is not part of the image border and is on a even row.

Lastly, for the last color channel (_patternOffsets[3], Blue channel for RGGB and GRBG patterns; Red channel for the other patterns), the offset is the position of the first “missing” pixel from the color channel that is not part of the image border and is on a even row.

Illustration of the chosen pattern offsets for a RGGB pattern image
_patternOffsets[0] is 9 ( _width + 1 ) _patternOffsets[1] is 9 ( _width + 1 ) _patternOffsets[2] is 18 ( (2 * _width) + 2 ) _patternOffsets[3] is 18 ( (2 * _width) + 2 )

3 Bilinear Interpolation

The Process method executes the Bilinear Interpolation. It starts by running two of the following four methods for demosaicing: DebayerBottomRight, DebayerBottomLeft, DebayerTopLeft, and DebayerTopRight, in accordance to the image’s Bayer pattern. Those methods are used to demosaic the Red and Blue channels of the image.

Red and Blue channels demosaicing

For each block of 2x2 pixels, the algorithm tries to fill in the “missing” pixels. In the case of Red and Blue channels, there are three “missing” pixels per block. As such, there are three different calculations, one for each “missing” pixel: Diagonal Cross Interpolation In this case, the “missing” pixel has four neighbors sensels around it. The average value of those four sensels will be the new value for the “missing” pixel. Horizontal Interpolation The “missing” pixel has a sensel at its left and a sensel at its right. Similarly to the previous case, the average value of those two sensels will be the new value for the “missing” pixel. Vertical Interpolation For the last case, the “missing” pixel has two sensels, one at its top and one at its bottom. The average value of those two sensels will be the new value for the “missing” pixel.

Afterwards, the Green channel is demosaiced with DebayerGreen.

Green channel demosaicing
In the case of Green channel, there are two “missing” pixels per block, with a unique calculation case: Cross Interpolation In this case, the “missing” pixel has four neighbors sensels around it. The average value of those four sensels will be the new value for the “missing” pixel.

Finally, the last step is to demosaic the borders, using DemosaicBorders for each color channel. This method simply copies the nearest pixel that is not in the image border.

4 Nearest Neighbor Interpolation

In a similar way as Bilinear Interpolation, Nearest Neighbor Interpolation is executed by the ProcessNearest method. The method calls the DebayerNearest method, with the correct color offsets, according to the image’s Bayer pattern. DebayerNearest works by interpolating the “missing” pixels from each color channel with the nearest sensel.

Red and Blue channels demosaicing
The method, for each block of 2x2 pixels, fills the three “missing” pixels with the same value as the only sensel in the block.

Green Channel demosaicing
In the Green channel the algorithm is a bit different, as there are only two missing colors. For each block of 2x2 pixels, it fills the first “missing” pixel with the same value as the first sensel and the second “missing” pixel with the same value as the second sensel.

---