ECSE415 - Introduction to Computer Vision  - Assignment 4 - Solved

1           Segmentation (65 Points)
Use image ‘flower.jpg’ (Fig.1) for this question.

 

Figure 1: Image to be used for segmentation questions.

1.1         K-means clustering and Expectation Maximization
In this question, you will implement K-means and EM from scratch. The algorithms should be implemented using only the numpy library. You can use opencv and matplotlib libraries only to read and display images but not for clustering.

1.1.1         K-means clustering
•    Implement K-means clustering from scratch. (10 points)

•    Apply your implementation of K-means to the provided image with K=2 and K=3. Use R, G, B color channels of the image as three features. Display (a) the segmentation evolving during the first 5 iterations and (b) the final segmentation. (5 points)

•    Convert the given image into gray-scale. Apply K-means to the provided image with K=2 and K=3. For every pixel, use gray-scale intensity as feature. Display (a) the segmentation evolving during the first 5 iterations and (b) the final segmentation. (4 points)

•    Compare final segmentation maps of color and gray-scale images. Which feature results in better segmentation? (1 points)

1.1.2         Expectation Maximization – Gaussian Mixture Model
•    Implement EM from scratch. (15 points)

•    Apply your implementation of EM to the provided image with 2 and 3 Gaussian components. Use R, G, B color channels of the image as three features. Display (a) the segmentation evolving during the first 5 iterations and (b) the final segmentation. (5 points)

•    Convert the given image into gray-scale. Apply EM to the provided image with 2 and 3 Gaussian components. For every pixel, use gray-scale intensity as feature. Display (a) the segmentation evolving during the first 5 iterations and (b) the final segmentation. (4 points) • Compare final segmentation maps of color and gray-scale images. Which feature results in better segmentation? (1 points)

1.2         Normalized Graph-cut and Mean-shift Segmentation
You can use functions from OpenCV or skimage libraries for this question.

•    Segment the given image using normalized graph-cuts. Vary the following parameters (try several values of each parameter): compactness and n segments (slic function), thresh (cut normalized function). Display segmentation results for several parameters and state their effect on the output (10 Points).

•    Segment the given image using mean-shift. Vary the following parameters (try several values of each parameter): ratio, kernel size, max dist. Display segmentation results for several parameters and state their effect on the output. (10 points)

2           Image Classification with Convolutional Neural Network (20 Points)
In this part, you will classify MNIST digits into 10 categories using a CNN. You may chose to run the code on GPU.

1.    Use Pytorch class torchvision.datasets.MNIST to (down)load the dataset. Use batch size of 32. (3 points)

2.    Implement a CNN with the layers mentioned below. (6 points)

•   A convolution layer with 32 kernels of size 3×3.

•   A ReLU activation.

•   A convolution layer with 64 kernels of size 3×3.

•   A ReLU activation.

•   A maxpool layer with kernels of size 2×2.

•   A convolution layer with 64 kernels of size 3×3.

•   A ReLU activation.

•   A convolution layer with 64 kernels of size 3×3.

•   A ReLU activation.

•   A flattening layer. (This layer resizes 2D feature map to a feature vector. The length of this feature vector should be 4096.)

•   A Linear layer with output size of 10.

(Suggestion: You can start with the code from the Tutorial and adapt it for the current problem.)

3.    Create an instance of SGD optimizer with learning rate of 0.001. Use the default setting for rest of the hyperparameters. Create an instance of categorical cross entropy criterion. (2 point)

4.    Train the CNN for 12 epochs. (6 points)

5.    Predicts labels of the test images using the above trained CNN. Measure and display classification accuracy. (3 points)