CS 4476/6476: Computer Vision, PS1 Solved

CS 4476/6476: Computer Vision,  PS1 

Instructions

1.   The assignment must be done in Python3. No other programming languages are allowed.

2.   Fill your answers in the answer sheet PPT provided and submit the file under the name: FirstName_LastName_PS1.pdf on Gradescope. Please do not modify the layout of the boxes provided in the answer sheet and fit your answers within the space provided.

3.   Please enter your code in the designated areas of the template Python files. Please do not add additional functions/imports to the files. Points will be deducted for any changes to code/file names, use of static paths and anything else that needs manual intervention to fix.

4.   Please submit your code and output files in a zipped format, using the helper script zip_submission.py with your GT username as a command line argument (using --gt_username), to Gradescope. Please do not create subdirectories within the main directory. The .zip_dir_list.yml file contains the required deliverable files, and zip_submission.py will fail if all the deliverables are not present in the root directory. Feel free to comment and uncomment them as you complete your solutions.

5.   For the implementation questions, make sure your code is bug-free and works out of the box. Please be sure to submit all main and helper functions. Be sure to not include absolute paths. Points will be deducted if your code does not run out of the box.

6.   If plots are required, you must include them in your Gradescope report and your code must display them when run. Points will be deducted for not following this protocol.

7.   The only libraries allowed for this assignment are numpy, matplotlib, pillow, opencv and skimage. Use pip install -r requirements.txt in a virtual environment to install necessary packages.

8.   Ensure that you follow the instructions very carefully.

Note: There is no extra credit for this first assignment. Students in both sections are expected to complete the entire assignment below (total 100 points).

Short answer problems [45 points]
Read through the provided Python, NumPy and Matplotlib introduction code and comments:

http://cs231n.github.io/python-numpy-tutorial/ or

https://filebox.ece.vt.edu/~F15ECE5554ECE4984/resources/numpy.pdf. Open an interactive session in Python and test the commands by typing them at the prompt. (Skip this step if you are already familiar with Python and NumPy.)

After reading the required documentation in the above section, to test your knowledge ensure that you know the outputs of the following commands without actually running them.

import numpy as np

(a)     x = np.random.permutation(1000)

(b)     a = np.array([[11,22,33],[40,50,60],[77,88,99]])

b = a[2,:]

(c)     a = np.array([[11,22,33],[40,50,60],[77,88,99])

b = a.reshape(-1)

(d)     f = np.random.randn(5,1)

g = f[f0]

(e)     x = np.zeros(10)+0.5

y = 0.5*np.ones(len(x))

z = x + y

(f)     a = np.arange(1,100)

b = a[::-1]

1.   Write a few lines of code to do each of the following. Copy and paste your code into the answer sheet.

[20 points]

1.1   Use numpy.random.rand to return the roll of a six-sided die over N trials.

1.2   Let y be the vector: y = np.array([11, 22, 33, 44, 55, 66]). Use the reshape command to form a new matrix z that looks like this: [[11,22],[33,44],[55,66]]

1.3   Use the numpy.max and numpy.where functions to set x to the maximum value that occurs in z (above), and set r to the row number (0-indexed) it occurs in and c to the column number (0-indexed) it occurs in.

1.4   Let v be the vector: v = np.array([1, 4, 7, 1, 2, 6, 8, 1, 9]). Set a new variable x to be the number of 1’s in the vector v.

2.   Load the 100x100 matrix inputAPS1Q2.npy which is the matrix A. Fill the template functions in the script PS1Q2.py to load inputAPS1Q2.npy and perform each of the following actions on A. Submit the file PS1Q2.py. [25 points]

2.1   Plot all the intensities in A, sorted in decreasing value. Provide the plot in your answer sheet. (Note, in this case we don’t care about the 2D structure of A, we only want to sort the list of all intensities.) To ensure consistency, you may use the gray colormap option.

2.2   Display a histogram of A’s intensities with 20 bins. Again, we do not care about the 2D structure. Provide the histogram in your answer sheet.

2.3   Create and return a new matrix X that consists of the bottom left quadrant of A. (Use function prob_2_3 and return X.)

2.4   Create and return a new matrix Y, which is the same as A, but with A’s mean intensity value subtracted from each pixel. (Use function prob_2_4 and return Y.)

2.5   Create and return a new matrix Z that represents a color image the same size as A, but with 3 channels to represent R, G and B values. Set the values to be red (i.e., R = 1, G = 0, B = 0) wherever the intensity in A is greater than a threshold t = the average intensity in A, and black everywhere else. Save Z as outputZPS1Q2.png and be sure to view it in an image viewer to make sure it looks right. (Use function prob_2_5 and return Z.)

Short programming example [36 points]
The input color image inputPS1Q3.jpg has been provided. Fill the template functions in the script PS1Q3.py to perform the following transformations. Avoid using loops. Submit the file PS1Q3.py. Display all the resultant images in your answer sheet.

Note: In every part, the image that is returned from your function must have integer values in the range [0, 255] i.e uint8 format.

3.1 Load the input color image and swap its red and green color channels. Return the output image. (Use function prob_3_1 and return swapImg.)

3.2 Convert the input color image to a grayscale image. Return the grayscale image.

(Use function prob_3_2 and return grayImg.)

Perform each of the below transformations on the grayscale image produced in part 2 above.

3.3 Convert the grayscale image to its negative image, in which the lightest values appear dark and vice versa. Return the negative image. (Use function prob_3_3 and return negativeImg.)

3.4 Map the grayscale image to its mirror image (flipping it left to right). Return the mirror image. (Use function prob_3_4 and return mirrorImg.)

3.5 Average the grayscale image with its mirror image (use typecasting). Return the averaged image. (Use function prob_3_5 and return avgImg.)

3.6 Create a matrix N whose size is same as the grayscale image, containing random numbers in the range [0, 255]. Save this matrix in a file called noise.npy. Add N to the grayscale image, then clip the resulting image to have a maximum value of 255. Return the clipped image. (Use function prob_3_6 and return addedNoiseImg.)

Be sure to submit noise.npy.

Tips: Do the necessary typecasting (uint8 and double) when working with or displaying the images. If you can’t find some functions in numpy (such as rgb2gray), you can write your own function. For example:

def rgb2gray(rgb): return np.dot(rgb[...,:3], [0.2989, 0.5870, 0.1140])

Understanding Color [19 points]
4.1 The same color may look different under different lighting conditions. Images indoor.png and outdoor.png are two photos of a same Rubik’s cube under different illuminances. Load the images and plot their R, G, B channels separately as grayscale images using plt.imshow() (beware of normalization). Then convert them into LAB color space using cv2.cvtColor() and plot the three channels again. Include the plots in your report. (Use function prob_4_1) (5 pts)

4.2 How do you know the illuminance change is better separated in LAB color space? (4 pts)

4.3 In the section below, we have explained the process behind translating the cubical colorspace of RGB to the cylinder of hue, saturation, and value. Read through the explanation and fill in the template function in PS1Q4.py to load inputPS1Q4.jpg, convert the image from RGB to HSV and return the final HSV image. You may use for loops for this question. Submit the core logic and outputPS1Q4.jpg in the answer sheet. (10pts)

Note: To ensure consistency, do the necessary typecasting (double) to transform the image to the [0,1] scale before performing the below operations.

So far we’ve been focussing on RGB and grayscale images. But there are other colorspaces out there too we may want to play around with. Like Hue, Saturation, and Value (HSV).

Hue can be thought of as the base color of a pixel. Saturation is the intensity of the color compared to white (the least saturated color). The Value is the perception of brightness of a pixel compared to black. You can try out this demo to get a better feel for the differences between these two colorspaces.

Now, to be sure, there are lots of issues with this colorspace. But it’s still fun to play around with and relatively easy to implement. The easiest component to calculate is the Value, it’s just the largest of the 3 RGB components:

V = max(R,G,B)

Next we can calculate Saturation. This is a measure of how much color is in the pixel compared to neutral white/gray. Neutral colors have the same amount of each three color components, so to calculate saturation we see how far the color is from being even across each component. First we find the minimum value

m = min(R,G,B) Then we see how far apart the min and max are:

C = V − m

and the Saturation will be the ratio between the difference and how large the max is:

S = C/V

Except if R, G, and B are all 0. Because then V would be 0 and we don’t want to divide by that, so just set the saturation 0 if that’s the case.

Finally, to calculate Hue we want to calculate how far around the color hexagon our target color is.

 

Figure 1: Color Hexagon

We start counting at Red. Each step to a point on the hexagon counts as 1 unit distance. The distance between points is given by the relative ratios of the secondary colors. We can use the following formula from Wikipedia:

 undefined     C = 0  if V = R    if V = G                   if V = B

 if H’ < 0

                                                                                     otherwise

There is no "correct" Hue if C = 0 because all of the channels are equal so the color is a shade of gray, right in the center of the cylinder. However, for now let’s just set H = 0 if C = 0 because then your implementation will match ours. (Use function prob_4_3 and return HSV.)

Deliverable Checklist

1.   1.1-1.4 (answer sheet) code snippets for each question.

2.   2.1-2.5 (code/files) PS1Q2.py, (answer sheet) corresponding plot for 2.1 & 2.2.

3.   3.1-3.6 (code/files) PS1Q3.py, noise.npy, (answer sheet) 6 images.

4.   4.1 (answer sheet) display the 2 plots (RGB, LAB)

5.   4.2 (answer sheet) explanation

6.   4.3 (code/files) PS1Q4.py, outputPS1Q4.png