Lab 7
Learning in Reproducing Kernel Hilbert Spaces
Advanced Machine Learning DATA 442/642
Exercise 1
For this exercise, the performance of the SVM is tested in the context of a two-class twodimensional classification task. The data set comprises N = 150 points uniformly distributed in the region [−5,5] × [−5,5]. For each point xn = [xn,1,xn,2]>, we compute
where η stands for zero mean Gaussian noise of variance = 4. The point is assigned to either of the two classes, depending on the value of the noise as well as its position with respect to the graph of the function
f(x) = 0.05x3 + 0.05x2 + 0.05x + 0.05
in the two-dimensional space. That is, if xn,2 ≥ yn, the point is assigned to class ω1; otherwise, it is assigned to class ω2.
(a) Plot the points [xn,1,xn,2] using different colors for each class.
(b) Use SVM with the Gaussian kernel for σ = 20 and set C = 1. Plot the classifier and the margin. Moreover, find the support vectors (i.e., the points with nonzero Lagrange multipliers that contribute to the expansion of the classifier) and plot them as circled points.
(c) Repeat step (b) using C = 0.5,0.1,0.05.
(d) Repeat step (b) using C = 5,10,50,100.
(e) Comment on the results.
