ISYE6740 Homework 3 Solution

1. Order of faces using ISOMAP
The objective of this question is to reproduce the ISOMAP algorithm results that we have seen discussed in lecture as an excercise. The file isomap.mat (or isomap.dat) contains 698 images, corresponding to different poses of the same face. Each image is given as a 64 × 64 luminosity map, hence represented as a vector in R4096. This vector is stored as a row in the file. [This is one of the datasets used in the original paper for ISOMAP, J.B. Tenenbaum, V. de Silva, and J.C. Langford, Science 290 (2000) 2319-2323.]
(a) (20 points) Choose the Euclidean distance between images (i.e., in this case a distancein R4096). Construct a similarity graph with vertices corresponding to the images, and tune the threshold so that each node has at least 100 neighbors. Visualize the similarity graph (e.g., plot the adjacency matrix, or visualize the graph and illustrate a few images corresponds to nodes at different parts of the graph; you can be a bit creative here).
(b) (20 points) Implement the ISOMAP algorithm and apply it to this graph to obtain ad = 2-dimensional embedding. Present a plot of this embedding. Find three points that are “close” to each other in the embedding space, and show what they look like. Do you see any visual similarity among them?
(c) (10 points) Now choose `1 distance (or Manhattan distance) between images (recall the definition from “Clustering” lecture)). Repeat the steps above. Again construct a similarity graph with vertices corresponding to the images, and tune the threshold so that each node has at least 100 neighbors. Implement the ISOMAP algorithm and apply it to this graph to obtain a d = 2-dimensional embedding. Present a plot of this embedding. Do you see any difference by choosing a different similarity measure?
1
2. Density estimation: Psychological experiments. 
(a)  Form 2-dimensional histogram for the pairs of variables (amygdala, acc). Decide on a suitable number of bins so you can see the shape of the distribution clearly.
(b) (20 points) Now implement kernel-density-estimation (KDE) to estimate the 2-dimensionalwith a two-dimensional density function of (amygdala, acc). Use a simple multidimensional Gaussian kernel, for
,
where x1 and x2 are the two dimensions respectively
.
Recall in this case, the kernel density estimator (KDE) for a density is given by
,
where xi are two-dimensional vectors, h > 0 is the kernel bandwidth. Set an appropriate h so you can see the shape of the distribution clearly. Plot of contour plot (like the ones in slides) for your estimated density.
2