CSCI935-Assignment 1 Solved

Design a C or C++ (ANSI standard) program that implements the colour image processing chain converting images captured by CMOS image sensors into true colour RGB images.

 

BACKGROUND 

To satisfy quality requirements, images captured by semiconductor sensors have to be processed before they are passed to higher stages of computer vision systems.  

You are provided with two bmp files test1.bmp and test2.bmp, which contain Bayer Pattern CFA data directly captured by CMOS image sensors. The images have resolution 640x480 pixels. The Bayer pattern used in the CMOS sensor is shown below:

 

 

Fig. 1. Mosaic of the Bayer pattern

 

Although basic operations with BMP files is not the focus of this assignment and BMP file read/write functions have been implemented for you in the program template assignment1.c, it may be a good idea to get familiar with the BMP file format that is explained below for your information.

 

The bmp file format stores image data together with supplementary information in the structured binary file. The BMP format uses Litte Endian byte order for short int, int and unsigned int values. It contains three major segments as shown in Fig. 2. 

 

 

  

       imageDataOffset 

 

 

 

 

 

        Fig. 2. The structure of BMP files. 

 

The File Header provides general information that helps file type verification and also indicates the starting position of image data in the file. The File Header structure can be described as follows: 

typedef struct                        

{ 

    char fileMarker1;       /* 'B' */                           char fileMarker2;       /* 'M' */     unsigned int   bfSize;                 unsigned short unused1;                  unsigned short unused2;              

    unsigned int   imageDataOffset;  /* Offset to the start of image data */ 

}FileHeader; 

 

The first two characters 'B' and 'M' indicate that this is a BMP file and their presence must be verified before any further analysis takes place.  If 'B''M' file markers are correct, this is a BMP file and thus, the value of imageDataOffset  indicates the start of image data in the file ( see Fig. 2 ). However, before reading the image data, it is important to know the image width and its height. This information is stored in the Info Header. 

 

typedef struct                       

{ 

    unsigned int   biSize;            

    int            width;            /* Width of the image */     int            height;           /* Height of the image */     unsigned short planes;            

    unsigned short bitPix;          /* 24 for true colour */     unsigned int   biCompression;          unsigned int   biSizeImage;            int            biXPelsPerMeter;        int            biYPelsPerMeter;        unsigned int   biClrUsed;              unsigned int   biClrImportant;     

}InfoHeader; 

 

As images can be considered as 2D arrays, height indicates the number of rows and width indicates the number of columns. Each element of the array contains information about three color components blue, green and red which together can represent any complex color. Thus, each image data element can be described as the following structure: 

typedef struct                        

{ 

    unsigned char  b;         /* Blue value */     unsigned char  g;         /* Green value */     unsigned char  r;         /* Red value */ 

}Pixel; 

 

Note: All these structures need to be packed when they are defined  

( #pragma pack(push, 1) ). Otherwise, your program may insert slack bytes. 

Note: BMP files have Little Endian byte order. Test your computer platform first (compile platform platform_test.c and run it) to make sure it is also Little Endian.  

 

Picture elements of true colour images are stored in the Image Data segment sequentially ( see Fig. 3). The total number of Pixel elements is width*height. 

 

      

                                              imageDataOffset                                                   width*height*3

      Fig. 3. The starting position and the internal structure of Image Data segment. 

Picture elements of CFA images are stored in the Image Data segment sequentially too ( see Fig. 4). The total number of unsigned char elements is width*height (the total number of bytes is 3 times less than for true colour images) 

 

       

                                              imageDataOffset                             width*height

      Fig. 4. The internal structure of Bayer pattern images. 

 

 

DESIGN SPECIFICATION  

Bayer pattern picture elements captured by the sensor have been stored in BMP files test1.bmp and test2.bmp. In this assignment you are provided with a template C file assignment1.c that:  

-  prompts the user to enter the input file name ( test1.bmp or test2.bmp )

-  reads content of a specified bmp file with CFA data into a 2D array

-  calls the function  processCFAImage(…)  that you have to complete - stores the produced true colour RGB image in the output bmb file.  

 

There is no need to use OpenCV library for this assignment. You need to modify assignment1.c (or use it as a basis for your C/C++ solution ) to implement a colour processing chain that includes the following stages:  

-  CFA Interpolation 

-  Colour Correction  

-  Gamma Correction  

These stages may be implemented in the function  processCFAImage(…). However,  

it may be better to implement them as three separate functions called from processCFAImage(…) in the required sequence.

 

You can select any CFA interpolation algorithms for your implementation such as Nearest Neighbor, Bilinear, etc. The requirement is that it must produce an RGB image with the same width and height. The algorithms are not complex, but you need to take into account boundary conditions, as pixels around the image boundary may not have neighboring pixels on one or two sides. Try to find the most efficient and simple solution to interpolate boundary pixels.

 

Colour Correction is a matrix operation. You should use the following matrix that was optimized for this CMOS image sensor:

 

    1.18  -0.09  -0.13 

 -0.24   1.29  -0.05 

 -0.18  -0.44   1.71 

 

Gamma correction should be implemented using a look-up table as described in the lecture notes. The only difference is that the table must map 8-bit values onto 8-bit values. The table entries must be calculated using γ = 0.5 and implement the following transfer function:

 

               0                                255     input

  

You need to find the expression how to calculate and fill the table.

 

The function processCFAImage(…) has the following prototype:

bool processCFAImage( unsigned char **cfaImage,    Pix ** rgbImage,   int width,   int height); 

 

Where width and height are cfa (and also rgb) image sizes in pixels. cfaImage is a two dimensional array with CFA Bayer pattern values. rgbImage is a 2D array of RGB image samples that must be produced by your image processing chain. If there are no errors in the function, it returns true. The resulting rgb samples are stored in a bmp file. The produced bmp file can be displayed by any image viewer. For test1.bmp it should be similar to the image shown below