ECE471- Assignment 1 Solved

000          ece, Selected Topics in Machine Learning – Assignment 1

001

002        

003

004          tldr: Perform linear regression of a noisy sinewave using a set of gaussian basis

005            functions with learned location and scale parameters. Model parameters are

006            learned with stochastic gradient descent. Use of automatic differentiation is

007

008             required. Hint: note your limits!

009

010

011             Problem Statement        Consider a set of scalars {x1,x2,...,xN} drawn from U(0,1) 012   and a corresponding set {y1,y2,...,yN} where:

013

014

015

016

017

018

019

020

021

022
yi = sin(2πxi)+ ϵi

and ϵi is drawn from N(0,σnoise). Given the following functional form:

yˆi = ∑wjϕj (xi | µj,σj)+ b M

j=1

with:
(1)

(2)
023

024(3)

025

026

027
find estimates ˆb, {µˆj}, {σˆj}, and {wˆj} that minimize the loss function:
028

029(4)

030

031              for all (xi,yi) pairs. Estimates for the parameters must be found using stochastic

032 gradient descent. A framework that supports automatic differentiation must be

033

034                       used. Set N = 50,σnoise = 0.1. Select M as appropriate. Produce two plots. First,

035            show the data-points, a noiseless sinewave, and the manifold produced by the

036

regression model. Second, show each of the M basis functions. Plots must be of

037

038           suitable visual quality.

039



056                                                                          −4               −2                               0                                 2                                 4                                 −4                               −2                               0                                 2                                 4 x                              x

057

058                 Figure 1: Example plots for models with equally spaced sigmoid and gaussian basis functions.

059