MachineLearning -  Homework 7  -Solved

 I.    Kernel Eigenfaces


●     Data

o        The Yale Face Database.zip​     contains 165 images of 15 subjects​    (subject01, subject02, etc.).  There are 11 images per subject, one for each of the following facial expressions or configurations: center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink.  

o        These data are separated into training dataset(135 images) and testing dataset(30 images). You can resize the images for easier implementation.  

●     What you are going to do o    Part1: Use PCA and LDA to show the first 25 eigenfaces and fisherfaces, and randomly pick 10 images to show their reconstruction. (please refer to the lecture slides).

o        Part2: Use PCA and LDA to do face recognition, and compute the performance. You should use k nearest neighbor to classify which subject the testing image belongs to.  

o        Part3: Use kernel PCA and kernel LDA to do face recognition, and compute the performance. (You can choose whatever kernel you want, but you should try different kernels in your implementation.) Then compare the difference between simple LDA/PCA and kernel LDA/PCA, and the difference between different kernels.  

 

             

t-SNE  

Here are nice implementations of t-SNE in different programming languages: https://lvdmaaten.github.io/tsne/ 

●     Data & reference code o         Download link:

https://lvdmaaten.github.io/tsne/code/tsne_python.zip,​  

o        mnist2500_X.txt:​ contains 2500 feature vectors with length 784, for describing 2500 mnist images.  

o        mnist2500_labels.txt:​ provides corresponding labels o tsne.py: reference code​     

●     What you are going to do o    Part1: Try to modify the code a little bit and make it back to symmetric SNE. You need to first understand how to implement t-SNE and find out the specific code piece to modify. You have to explain the difference between symmetric SNE and t-SNE in the report (e.g. point out the crowded problem of symmetric SNE).  

o        Part2: Visualize the embedding of both t-SNE and symmetric SNE. Details of the visualization:

▪ Project all your data onto 2D space and mark the data points into different colors respectively. The color of the data points depends on the label.      

▪ Use videos or GIF images to show the optimize procedure.  o Part3: Visualize the distribution of pairwise similarities in both high-dimensional space and low-dimensional space, based on both t-SNE and symmetric SNE.

o        Part4: Try to play with different perplexity values. Observe the change in visualization and explain it in the report.  

■ Anything you want to discuss, such as the meaning of eigenfaces or trying different dimension reduction methods, comparing the advantages and disadvantages of them.