Camera Sensor Data

We have to distinguish between grayscale and color cameras, although, as we will see, there is only a minor difference between the two. The simplest available sensor chips provide a grayscale image of 120 lines by 160 columns with 1 byte per pixel (for example VLSI Vision VV5301 in grayscale or VV6301 in color). A value of zero represents a black pixel, a value of 255 is a white pixel, everything in between is a shade of gray. Figure below illustrates such an image. The camera transmits the image data in row-major order, usually after a certain frame-start sequence.

Creating a color camera sensor chip from a grayscale camera sensor chip is
very simple. All it needs is a layer of paint over the pixel mask. The standard
technique for pixels arranged in a grid is the Bayer pattern (Figure 2.17). Pixels
in odd rows (1, 3, 5, etc.) are colored alternately in green and red, while
pixels in even rows (2, 4, 6, etc.) are colored alternately in blue and green.



With this colored filter over the pixel array, each pixel only records the intensity of a certain color component. For example, a pixel with a red filter will
only record the red intensity at its position. At first glance, this requires 4 bytes
per color pixel: green and red from one line, and blue and green (again) from the line below. This would result effectively in a 60􀁵80 color image with an additional, redundant green byte per pixel. However, there is one thing that is easily overlooked. The four components red, green1, blue, and green2 are not sampled at the same position. For example, the blue sensor pixel is below and to the right of the red pixel. So by treating the four components as one pixel, we have already applied some sort of filtering and lost information.



A technique called “demosaicing” can be used to restore the image in full 120􀁵160 resolution and in full color. This technique basically recalculates the three color component values (R, G, B) for each pixel position, for example by averaging the four closest component neighbors of the same color. Figure below shows the three times four pixels used for demosaicing the red, green, and blue components of the pixel at position [3,2] (assuming the image starts in the top left corner with [0,0]).



Averaging, however, is only the simplest method of image value restoration
and does not produce the best results. A number of articles have researched better algorithms for demosaicing [Kimmel 1999], [Muresan, Parks 2002].