Starting from:
$30
CS464-Homework 2 SVM and Logistic Regression Solved
1 SVM and Logistic Regression
In this question, you are expected to classify aquatic animals using features extracted with two different methods: transfer learning with Inception v3 and histogram of oriented gradients (HOG). Transfer learning is a widely used approach for various tasks such as object recognition and localization. Inception v3 is a deep convolutional neural network architecture developed by Google [1]. In order to extract features from aquatic animal images, an Inception v3 model is pre-trained on ImageNet [2] dataset. Then the model’s last layer is removed to use the output of second to last layer (of size 2048 × 1) as image features. Finally, each image in the dataset is passed from the model for feature extraction. On the other hand, HOG [3] is a feature descriptor used in computer vision for the purpose of object detection, which counts occurrences of gradient orientations in different parts of the image. HOG produces features of size 324 × 1. Image features are extracted and provided for your use.
Aquatic animals dataset is a subset of CIFAR100 dataset [4]. Image features and labels are provided in q1dataset.mat. The dataset contains a train split of 1000 images and a test split of 200 images. You will not use a separate validation split. Each image in the dataset has a subclass label and the corresponding superclass label. Beaver (Subclass 0), dolphin (Subclass 1), otter (Subclass 2), seal (Subclass 3) and whale (Subclass 4) classes are subclasses of aquatic mammal (Superclass 0). In like manner, aquarium fish (Subclass 0), flatfish (Subclass 1), ray (Subclass 2), shark (Subclass 3) and trout (Subclass 4) are subclasses of fish superclass (Superclass 1). q1dataset.mat consists of the following:
• inception features train: A matrix of size 2000 × 2048 whose rows correspond to the features extracted from training split using Inception v3 network.
• inception features test: A matrix of size 400×2048 whose rows correspond to the features extracted from test split using Inception v3 network.
• hog features train: A matrix of size 2000 × 324 whose rows correspond to the features extracted from training split using HOG method.
• hog features test: A matrix of size 400 × 324 whose rows correspond to the features extracted from training split using HOG method.
• superclass labels train: A matrix of size 2000 × 1 which contains the superclass label assignments for train split.
• superclass labels test: A matrix of size 400 × 1 which contains the superclass label assignments for test split.
• subclass labels train: A matrix of size 2000 × 1 which contains the subclass label assignments for train split.
• subclass labels test: A matrix of size 400×1 which contains the subclass label assignments for test split.
You are expected to measure training times of all models.
Superclass Classification Using Logistic Regression
For this part you will use the neural network generated features and HOG features to predict the superclass of an image. You will train each algorithm on HOG features and on neural network generated features separately. You should not test a model trained on HOG features with neural network generated features (vice versa). You must present two sets of results for each part. Assume that fish is the positive class.
Question 1.1 Implement mini-batch gradient ascent algorithm with batch size = 25 and stochastic gradient ascent algorithm to train logistic regression models. Initialize all weights to random numbers drawn from a Gaussian distribution N(0,0.01). Try different learning rates and choose the one which works best for you. Use 1000 iterations to train your model. Report accuracy, precision, recall, negative predictive value (NPV), false positive rate (FPR), false discovery rate (FDR), F1, F2 scores and the confusion matrix using your model on the corresponding test set. Discuss your results.
Question 1.2 Implement full batch gradient ascent algorithm to train your logistic regression model. Initialize all weights to random numbers drawn from a Gaussian distribution N(0,0.01). Use the learning rate you have chosen in Question 1.1 and perform 1000 iterations to train your models. Print model weights in each iteration i ∈ {100,200,....,1000}. Report 10 most important features(in terms of indices) and discuss the relation between weights and individual importance of features. Report accuracy, precision, recall, negative predictive value (NPV), false positive rate (FPR), false discovery rate (FDR), F1, F2 scores and the confusion matrix using your model on the given test set. Discuss your results.
Superclass Classification Using SVM
For this part you will use the neural network generated features and HOG features to predict the superclass of an image. You will train each algorithm on HOG features and on neural network generated features separately. You should not test a model trained on HOG features with neural network generated features (vice versa). You must present two sets of results for each part. Assume fish superclass is the positive class.
Question 1.3 Implement stratifiedkfold(features, labels, k) function which shuffles and splits a dataset into k folds with approximately equal label ratios.
Question 1.4 Train a soft margin SVM model with linear kernel. Tune C hyper-parameter of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for C ∈ {10−2,10−1,1,101,102}. You should select the optimal value of C, denoted by C∗, based on mean of the selected performance metric. Train a soft margin SVM model with linear kernel where C = C∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Question 1.5 Train a hard margin SVM with radial basis function (rbf) kernel. Radial basis function is defined as:
K(x,x0) = exp(−γ||x − x0||2) (1.1)
where γ is an hyper-parameter of the model which determines the radius of influence of support vectors on decision boundary of the model. Tune γ of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for γ ∈ {2−4,2−3,2−2,2−1,20,21,26}. You should select the optimal value of γ, denoted by γ∗, based on mean of the selected performance metric.
Train a hard margin SVM with rbf kernel where γ = γ∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Question 1.6 Train a soft margin SVM with radial basis function (rbf) as kernel. Tune the hyper-parameters of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for (C,γ) ∈ {10−2,1,102} × {2−2,21,26}. You should select the optimal values of C and γ, denoted by C∗ and γ∗, based on mean of the selected performance metric.
Train a soft margin SVM with rbf kernel where C = C∗ and γ = γ∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Subclass Classification Using SVM
For this part you will use the neural network generated features and HOG features to predict the subclass of an image. You will train each algorithm on HOG features and on neural network generated features separately
In this question, you are expected to classify aquatic animals using features extracted with two different methods: transfer learning with Inception v3 and histogram of oriented gradients (HOG). Transfer learning is a widely used approach for various tasks such as object recognition and localization. Inception v3 is a deep convolutional neural network architecture developed by Google [1]. In order to extract features from aquatic animal images, an Inception v3 model is pre-trained on ImageNet [2] dataset. Then the model’s last layer is removed to use the output of second to last layer (of size 2048 × 1) as image features. Finally, each image in the dataset is passed from the model for feature extraction. On the other hand, HOG [3] is a feature descriptor used in computer vision for the purpose of object detection, which counts occurrences of gradient orientations in different parts of the image. HOG produces features of size 324 × 1. Image features are extracted and provided for your use.
Aquatic animals dataset is a subset of CIFAR100 dataset [4]. Image features and labels are provided in q1dataset.mat. The dataset contains a train split of 1000 images and a test split of 200 images. You will not use a separate validation split. Each image in the dataset has a subclass label and the corresponding superclass label. Beaver (Subclass 0), dolphin (Subclass 1), otter (Subclass 2), seal (Subclass 3) and whale (Subclass 4) classes are subclasses of aquatic mammal (Superclass 0). In like manner, aquarium fish (Subclass 0), flatfish (Subclass 1), ray (Subclass 2), shark (Subclass 3) and trout (Subclass 4) are subclasses of fish superclass (Superclass 1). q1dataset.mat consists of the following:
• inception features train: A matrix of size 2000 × 2048 whose rows correspond to the features extracted from training split using Inception v3 network.
• inception features test: A matrix of size 400×2048 whose rows correspond to the features extracted from test split using Inception v3 network.
• hog features train: A matrix of size 2000 × 324 whose rows correspond to the features extracted from training split using HOG method.
• hog features test: A matrix of size 400 × 324 whose rows correspond to the features extracted from training split using HOG method.
• superclass labels train: A matrix of size 2000 × 1 which contains the superclass label assignments for train split.
• superclass labels test: A matrix of size 400 × 1 which contains the superclass label assignments for test split.
• subclass labels train: A matrix of size 2000 × 1 which contains the subclass label assignments for train split.
• subclass labels test: A matrix of size 400×1 which contains the subclass label assignments for test split.
You are expected to measure training times of all models.
Superclass Classification Using Logistic Regression
For this part you will use the neural network generated features and HOG features to predict the superclass of an image. You will train each algorithm on HOG features and on neural network generated features separately. You should not test a model trained on HOG features with neural network generated features (vice versa). You must present two sets of results for each part. Assume that fish is the positive class.
Question 1.1 Implement mini-batch gradient ascent algorithm with batch size = 25 and stochastic gradient ascent algorithm to train logistic regression models. Initialize all weights to random numbers drawn from a Gaussian distribution N(0,0.01). Try different learning rates and choose the one which works best for you. Use 1000 iterations to train your model. Report accuracy, precision, recall, negative predictive value (NPV), false positive rate (FPR), false discovery rate (FDR), F1, F2 scores and the confusion matrix using your model on the corresponding test set. Discuss your results.
Question 1.2 Implement full batch gradient ascent algorithm to train your logistic regression model. Initialize all weights to random numbers drawn from a Gaussian distribution N(0,0.01). Use the learning rate you have chosen in Question 1.1 and perform 1000 iterations to train your models. Print model weights in each iteration i ∈ {100,200,....,1000}. Report 10 most important features(in terms of indices) and discuss the relation between weights and individual importance of features. Report accuracy, precision, recall, negative predictive value (NPV), false positive rate (FPR), false discovery rate (FDR), F1, F2 scores and the confusion matrix using your model on the given test set. Discuss your results.
Superclass Classification Using SVM
For this part you will use the neural network generated features and HOG features to predict the superclass of an image. You will train each algorithm on HOG features and on neural network generated features separately. You should not test a model trained on HOG features with neural network generated features (vice versa). You must present two sets of results for each part. Assume fish superclass is the positive class.
Question 1.3 Implement stratifiedkfold(features, labels, k) function which shuffles and splits a dataset into k folds with approximately equal label ratios.
Question 1.4 Train a soft margin SVM model with linear kernel. Tune C hyper-parameter of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for C ∈ {10−2,10−1,1,101,102}. You should select the optimal value of C, denoted by C∗, based on mean of the selected performance metric. Train a soft margin SVM model with linear kernel where C = C∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Question 1.5 Train a hard margin SVM with radial basis function (rbf) kernel. Radial basis function is defined as:
K(x,x0) = exp(−γ||x − x0||2) (1.1)
where γ is an hyper-parameter of the model which determines the radius of influence of support vectors on decision boundary of the model. Tune γ of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for γ ∈ {2−4,2−3,2−2,2−1,20,21,26}. You should select the optimal value of γ, denoted by γ∗, based on mean of the selected performance metric.
Train a hard margin SVM with rbf kernel where γ = γ∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Question 1.6 Train a soft margin SVM with radial basis function (rbf) as kernel. Tune the hyper-parameters of the model using 5-fold cross validation on the training dataset. Use the function implemented in Question 1.3 to perform cross validation. Calculate a performance metric of your choice for each left-out fold. Report mean of the selected performance metric calculated on the left-out fold for (C,γ) ∈ {10−2,1,102} × {2−2,21,26}. You should select the optimal values of C and γ, denoted by C∗ and γ∗, based on mean of the selected performance metric.
Train a soft margin SVM with rbf kernel where C = C∗ and γ = γ∗ on training dataset and test your model on the corresponding test dataset. Report accuracy, confusion matrix, precision and recall. You should perform all steps for both HOG features and neural network generated features, and compare their results.
Subclass Classification Using SVM
For this part you will use the neural network generated features and HOG features to predict the subclass of an image. You will train each algorithm on HOG features and on neural network generated features separately
1 file (275.2KB)
