000 ece, Selected Topics in Machine Learning – Assignment 1
001
002
003
004 tldr: Perform linear regression of a noisy sinewave using a set of gaussian basis
005 functions with learned location and scale parameters. Model parameters are
006 learned with stochastic gradient descent. Use of automatic differentiation is
007
008 required. Hint: note your limits!
009
010
011 Problem Statement Consider a set of scalars {x1,x2,...,xN} drawn from U(0,1) 012 and a corresponding set {y1,y2,...,yN} where:
013
014
015
016
017
018
019
020
021
022
yi = sin(2πxi)+ ϵi
and ϵi is drawn from N(0,σnoise). Given the following functional form:
yˆi = ∑wjϕj (xi | µj,σj)+ b M
j=1
with:
(1)
(2)
023
024(3)
025
026
027
find estimates ˆb, {µˆj}, {σˆj}, and {wˆj} that minimize the loss function:
028
029(4)
030
031 for all (xi,yi) pairs. Estimates for the parameters must be found using stochastic
032 gradient descent. A framework that supports automatic differentiation must be
033
034 used. Set N = 50,σnoise = 0.1. Select M as appropriate. Produce two plots. First,
035 show the data-points, a noiseless sinewave, and the manifold produced by the
036
regression model. Second, show each of the M basis functions. Plots must be of
037
038 suitable visual quality.
039
056 −4 −2 0 2 4 −4 −2 0 2 4 x x
057
058 Figure 1: Example plots for models with equally spaced sigmoid and gaussian basis functions.
059
