$30
Deep Learning in Visual Computing Assignment 6 -Solved
Download Source Data
We will not use DataLoaders for this assignment becasue the dataset is mall and it is faster to train with the dataset loaded in the memory.
Src Train Min: Value- -1.0
Src Train Max: Value- 1.0
Src Test Min: Value- -1.0
Src Test Max: Value- 1.0 src_trX.shape: (73257, 3, 32, 32) src_trY.shape: (73257,) src_tsX.shape: (26032, 3, 32, 32) src_tsY.shape: (26032,) labels
[[1 9 2 3 2 5 9 3 3 1]
[3 3 2 8 7 4 4 1 2 8]
[1 6 2 3 6 3 4 2 5 8]
[1 6 2 3 7 9 5 3 2 2]
[2 6 2 2 5 1 5 4 7 8]
[5 2 1 0 6 1 9 1 1 8]
[3 6 5 1 4 4 1 6 3 4]
[2 0 1 3 2 5 4 1 4 2]
[8 3 8 6 0 1 5 1 1 2]
[9 1 6 9 2 6 1 2 0 6]]
Download Target Data
We will not use DataLoaders for this assignment becasue the dataset is small and it is faster to train with the dataset loaded in th memory.
Tgt Train Min: Value- -1.0
Tgt Train Max: Value- 1.0
Tgt Test Min: Value- -1.0
Tgt Test Max: Value- 1.0 tgt_trX.shape: (60000, 3, 28, 28) tgt_trY.shape: (60000,) tgt_tsX.shape: (10000, 3, 28, 28) tgt_tsY.shape: (10000,) labels
[[5 0 4 1 9 2 1 3 1 4]
[3 5 3 6 1 7 2 8 6 9]
[4 0 9 1 1 2 4 3 2 7]
[3 8 6 9 0 5 6 0 7 6]
[1 8 7 9 3 9 8 5 9 3]
[7 2 1 0 4 1 4 9 5 9]
[0 6 9 0 1 5 9 7 3 4]
[9 6 6 5 4 0 7 4 0 1]
[3 1 3 4 7 2 7 1 2 1]
[1 7 4 2 3 5 1 2 4 4]]
Design the Network Modules
Let's define the network architecures: Convolution Layers are represented as 'Conv(ch)', where 'ch' is the number of output channels. All convolutions are same conolutions with stride=3 and padding=1 Linear layers are represented as 'Linear(fts)', where 'fts' is the number of output features. Every Convolution and Linear Layer is followed by eLU activation except if it is the last layer before the Loss function. MaxPool layers are represented as 'MaxPool(fs)' where 'fs'=2 is the filter size with stride=2 BatchNorm layers are represented as 'BatchNorm(ch)', where 'ch' is the number of channels or inputs feature dimensions. For Convolution layers we use 'BacthNorm2d(ch)' and for linear layers we use 'BatchNorm1d(ch)'
FeatrureExtractor (Input = Batch_Size x 3 x 32 x 32 or Batch_Size x 3 x 28 x 28):
Conv(32) → Conv(32) → MaxPool(2) → BatchNorm(32) → Conv(64) → Conv(64) → MaxPool(2) → BatchNorm(64) → Conv(128) → Conv(128) → AdaptiveAvgPool2d(1) → Linear(128) → BatchNorm(128)
LabelClassifier (Input = Batch_Size x 128):
Linear(64) → BatchNorm → Linear(64) → SoftmaxCrossEntropyLoss
DomainClassifier (Input = Batch_Size x 128):
Linear(64) → BatchNorm → Linear(64) → BatchNorm → Linear(64) → SigmoidBinaryCrossEntropyLoss
In [62]:
# Feel free to modify the architecture within certai constrain to get better results. # Please check the comments for thetestcases to understathe constraints on the network architecture.
# The FeatrureExtractor network module is used to extract features from th source and the target data.
# Every image is extracted into a feature of 128 dimensions class FeatureExtractor(nn.Module):
def __init__(self):
super(FeatureExtractor, self).__init__()
# Define the components of the FeatrureExtractor
# your code here
self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1) self.conv12 = nn.Conv2d(32, 32, kernel_size=3, stride=1, padding=1) self.bn1 = nn.BatchNorm2d(32) self.relu = nn.ReLU()
self.maxpool = nn.MaxPool2d(2, stride=2)
self.conv2 = nn.Conv2d(32, 64, kernel_size=5, stride=1, padding=1) self.conv22 = nn.Conv2d(64, 64, kernel_size=5, stride=1, padding=1)
self.bn2 = nn.BatchNorm2d(64)
self.conv3 = nn.Conv2d(64, 128, kernel_size=5, stride=1, padding=1) self.conv32 = nn.Conv2d(128, 128, kernel_size=5, stride=1, padding=1)
self.bn3 = nn.BatchNorm1d(128) self.bn4 = nn.BatchNorm2d(128) self.avgpool = nn.AdaptiveAvgPool2d(1) self.fc = nn.Linear(128, 128)
# Set up the weight initialization mechanishm for m in self.modules(): if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear): nn.init.kaiming_normal_(m.weight)
def forward(self, x):
# Forward propagate through the FeatrureExtractor network module
# your code here x = self.conv1(x) x = self.relu(x) x = self.conv12(x) x = self.relu(x)
x = self.maxpool(x)
x = self.bn1(x)
x = self.conv2(x) x = self.relu(x) x = self.conv22(x) x = self.relu(x)
x = self.maxpool(x) x = self.bn2(x)
x = self.conv3(x) x = self.relu(x) x = self.conv32(x) x = self.relu(x)
x = self.avgpool(x)
N, C, H, W = x.size()
#reshape x to (N, C*H*W) x = x.view(N, C*H*W)
# The LabelClassifier network module is used to classify the eatures (output of FeatureExtractor) into 10 categories class LabelClassifier(nn.Module): def __init__(self): super(LabelClassifier, self).__init__()
# Define the components of the LabelClassifier DO NOT define the los layer. We will define it later
# your code here
# Set up the weight initialization mechanishm for m in self.modules(): if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):
def forward(self, x):
# Forward propagate through the LabelClassifier network module
# your code here
# Testcases
# The input to the FeatureExtractor is a Tensor (m,3,s,s), where m is number of samples (arbitrary batch_size)
# 3 is the RGB channels, s is the image size. The FeatureExtractor has an AdaptiveAvgPool2d() layer
# feeding a fully connected layer. It is therefore not constarined to a specific image size. It works for both 28 and 32
# The output of the FeatureExtractor is a Tensor (m,n), where m is number of samples (arbitrary batch_size) # and n=128 is feature dimensions.
# The input to the FeatureClassifier is a Tensor (m,n), where m is number of samples (arbitrary batch_size) # and n=128 is feature dimensions
# The output of the FeatureClassifier is a Tensor (m,c), where m is number of samples (arbitrary batch_size) # and c=10 is number of categories
# The input to the DomainClassifier is a Tensor (m,n), where m is number of samples (arbitrary batch_size) # and n=128 is feature dimensions
# The output of the DomainClassifier is a Tensor (m,1), where m is number of samples (arbitrary batch_size) # The values are between [0-1]
# The following testcases test only for the constraints above.
# You are free to select your own architecture to get better results.
f_t = FeatureExtractor() c_t = LabelClassifier() d_t = DomainClassifier()
x_t = torch.Tensor(np.random.randn(5,3,32,32)) x_f_t = f_t(x_t)
npt.assert_array_equal(x_f_t.shape, (5,128)) x_f_t = torch.Tensor(np.random.randn(5,128)) x_c_t = c_t(x_f_t)
npt.assert_array_equal(x_c_t.shape, (5,10)) x_d_t = d_t(x_f_t)
npt.assert_array_equal(x_d_t.shape, (5,1)) assert torch.all(x_d_t>0) and torch.all(x_d_t<= 1.)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:105: UserWarning: Implicit dimension ch oice for softmax has been deprecated. Change the call to include dim=X as an argument.
Intialize Network Module Objects (10 points)
-----------------------------------------FeatrureExtractor--------------------------------------FeatureExtractor(
(conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv12): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU()
(maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2): Conv2d(32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(1, 1))
(conv22): Conv2d(64, 64, kernel_size=(5, 5), stride=(1, 1), padding=(1, 1))
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv3): Conv2d(64, 128, kernel_size=(5, 5), stride=(1, 1), padding=(1, 1))
(conv32): Conv2d(128, 128, kernel_size=(5, 5), stride=(1, 1), padding=(1, 1))
(bn3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(bn4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc): Linear(in_features=128, out_features=128, bias=True) )
------------------------------------------LabelClassifier-----------------------------------------LabelClassifier(
(fc1): Linear(in_features=128, out_features=64, bias=True)
(bn): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU()
(fc2): Linear(in_features=64, out_features=10, bias=True)
(softmax): Softmax(dim=None)
)
------------------------------------------CrossEntropyLoss-----------------------------------------CrossEntropyLoss()
------------------------------------------DomainClassifier-----------------------------------------DomainClassifier(
(fc1): Linear(in_features=128, out_features=64, bias=True)
(bn): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU()
(fc2): Linear(in_features=64, out_features=64, bias=True) (fc3): Linear(in_features=64, out_features=1, bias=True)
Classifier Logits
Define a function for forward pass through the FeatureExtractor and the LabelClassifier
Label Classifier Loss
Define a function that gives gives the classification loss for the LabelClassifier module. The input is a Tensor of images along with corresponding labels. The images are input to the FeatureExtractor followed by the LabelClassifier and the loss is calculated using CrossEntropy Use the 'calc_clf_logits()' function and the 'ce_criterion' nn.CrossEntropyLoss Object to calculate the loss
Evaluate Model
Define a function 'evaluate_model(.)' that returns the accuracy of classification for given input images X and groundtruth labels Y.
Train Classifier Model With Source Data
We will train just FeatureExtractor and LabelClassifier with the source data and evaluate classification accuracies on the target data. The trained model is called 'Source-only'. We will train the FeatureExtractor and the LabelClassifier modules using the source data.
Iterations per epoch: 861
Epoch: 1/10, iter: 0, clf_err: 2.3035, clf_LR: 9.990E-03
Epoch: 1/10, iter: 200, clf_err: 2.0998, clf_LR: 8.178E-03
Epoch: 1/10, iter: 400, clf_err: 1.8370, clf_LR: 6.695E-03
Epoch: 1/10, iter: 600, clf_err: 1.6182, clf_LR: 5.481E-03
Epoch: 1/10, iter: 800, clf_err: 1.5998, clf_LR: 4.487E-03
Epoch: 2/10, iter: 0, clf_err: 1.6004, clf_LR: 4.217E-03
Epoch: 2/10, iter: 200, clf_err: 1.6156, clf_LR: 3.452E-03
Epoch: 2/10, iter: 400, clf_err: 1.6543, clf_LR: 2.826E-03
Epoch: 2/10, iter: 600, clf_err: 1.5752, clf_LR: 2.314E-03
Epoch: 2/10, iter: 800, clf_err: 1.5526, clf_LR: 1.894E-03
Epoch: 3/10, iter: 0, clf_err: 1.5760, clf_LR: 1.780E-03
Epoch: 3/10, iter: 200, clf_err: 1.5804, clf_LR: 1.457E-03
Epoch: 3/10, iter: 400, clf_err: 1.5906, clf_LR: 1.193E-03
Epoch: 3/10, iter: 600, clf_err: 1.5588, clf_LR: 9.767E-04
Epoch: 3/10, iter: 800, clf_err: 1.5196, clf_LR: 7.996E-04
Epoch: 4/10, iter: 0, clf_err: 1.5236, clf_LR: 7.515E-04
Epoch: 4/10, iter: 200, clf_err: 1.5786, clf_LR: 6.152E-04
Epoch: 4/10, iter: 400, clf_err: 1.5509, clf_LR: 5.036E-04
Epoch: 4/10, iter: 600, clf_err: 1.5409, clf_LR: 4.123E-04
Epoch: 4/10, iter: 800, clf_err: 1.5151, clf_LR: 3.375E-04
Epoch: 5/10, iter: 0, clf_err: 1.5174, clf_LR: 3.172E-04
Epoch: 5/10, iter: 200, clf_err: 1.5498, clf_LR: 2.597E-04
Epoch: 5/10, iter: 400, clf_err: 1.5269, clf_LR: 2.126E-04
Epoch: 5/10, iter: 600, clf_err: 1.5380, clf_LR: 1.740E-04
Epoch: 5/10, iter: 800, clf_err: 1.5086, clf_LR: 1.425E-04
Epoch: 6/10, iter: 0, clf_err: 1.5159, clf_LR: 1.339E-04
Epoch: 6/10, iter: 200, clf_err: 1.5298, clf_LR: 1.096E-04
Epoch: 6/10, iter: 400, clf_err: 1.5126, clf_LR: 8.975E-05
Epoch: 6/10, iter: 600, clf_err: 1.5335, clf_LR: 7.347E-05
Epoch: 6/10, iter: 800, clf_err: 1.5094, clf_LR: 6.015E-05
Epoch: 7/10, iter: 0, clf_err: 1.5096, clf_LR: 5.653E-05
Epoch: 7/10, iter: 200, clf_err: 1.5203, clf_LR: 4.628E-05
Epoch: 7/10, iter: 400, clf_err: 1.5076, clf_LR: 3.788E-05
Epoch: 7/10, iter: 600, clf_err: 1.5323, clf_LR: 3.101E-05
Epoch: 7/10, iter: 800, clf_err: 1.5096, clf_LR: 2.539E-05
Epoch: 8/10, iter: 0, clf_err: 1.5064, clf_LR: 2.386E-05
Epoch: 8/10, iter: 200, clf_err: 1.5193, clf_LR: 1.954E-05
Epoch: 8/10, iter: 400, clf_err: 1.5045, clf_LR: 1.599E-05
Epoch: 8/10, iter: 600, clf_err: 1.5314, clf_LR: 1.309E-05
Epoch: 8/10, iter: 800, clf_err: 1.5080, clf_LR: 1.072E-05
Epoch: 9/10, iter: 0, clf_err: 1.5057, clf_LR: 1.007E-05
Epoch: 9/10, iter: 200, clf_err: 1.5193, clf_LR: 8.247E-06
Epoch: 9/10, iter: 400, clf_err: 1.5031, clf_LR: 6.751E-06
Epoch: 9/10, iter: 600, clf_err: 1.5306, clf_LR: 5.527E-06
Epoch: 9/10, iter: 800, clf_err: 1.5073, clf_LR: 4.524E-06
Epoch: 10/10, iter: 0, clf_err: 1.5054, clf_LR: 4.252E-06
Epoch: 10/10, iter: 200, clf_err: 1.5193, clf_LR: 3.481E-06
Epoch: 10/10, iter: 400, clf_err: 1.5025, clf_LR: 2.850E-06
Epoch: 10/10, iter: 600, clf_err: 1.5303, clf_LR: 2.333E-06
Epoch: 10/10, iter: 800, clf_err: 1.5071, clf_LR: 1.910E-06
Evaluate Classification Accuracies With Source-only model (10 points)
Calculate accuracy on source and target datasets. Notice how the Network does not perform well on the target data.
Source train acc: 94.88
Source test acc: 92.04
Target train acc: 50.80
Target test acc: 52.75
Intialize the Domain Alignment network
Reintalize the network with same paramters for DANN training. This time we will use the DomainClassifier to align the domains in the DANN style
Importance of Domain Alignment
The parameter λ or 'lam' in our code, controls the importance of the DomainClassifier in the overall objective of domain alignment. The DomainClassifier gradient can conflict with the training of the LabelClassifier with noisy gradients in the initial stages of the training. We therefore set λ = 0 initially and gradually increase it after the FeatureExtractor has stabilised. This scheduling of 'lam' is implemented in the following 'adjust_lambda' function. Refer to Sec 5.2.2 in the DANN paper (https://arxiv.org/pdf/1505.07818.pdf) for more details.
λ = − 1
itr+epoch×no_itrs_per_epoch where, γ = 10 and p = is the fraction of total iterations completed in the training.
n_epochs×no_itrs_per_epoch
Domain Classification Loss
Similar to the Label Classifier Loss function defined in 'src_clf_loss(.)', we will define a Domain Classifier Loss function 'domain_clf_err(.)' which will return the loss of domain classification. The input is a Tensor of source images 's_img', a Tensor of target images 't_img' and the λ parameter 'lam'. We need to create the labels for the groundtruth. The source images belong to class 1 and target images to class 0. Propagate 's_img' through the FeatureExtractor and 't_img' through the FeatureExtractor. Concatenate the outputs and propagate the concatenated result through the DomainClasssifier to obtain the logits. Apply the 'bce_criterion()' defined earlier to determine the loss based on the logits and the groundtruth. The 'bce_criterion' is similar to 'ce_criterion' for CrossEntropyLoss
It may not be possible to concatenate 's_img' and 't_img' and forward propagate the concatenated output through the FeatureExtractor and DomainClassifier - there may be a dimension mismatch.
In [76]:
def domain_clf_loss(s_img, t_img, lam):
'''
The function returns the BinaryCrosEntropyLoss trying to distingiish between the source images 's_img' from the target images 't_img'
Inputs: s_img: Tensor of source input images of dimensions (m,3,32,32), where m is number of samples,
3 is for the RGB channels, 32 is the image size
t_img: Tensor of target input images of dimensions (m,3,28,28), where m is number of samples,
3 is for the RGB channels, 28 is the image size
lam: lambda parameter controlling the importance of domain alignment
Outputs: dom_loss: scalar loss of domain classification
'''
# Generate the groundtruth labels 1 for source and 0 for target
# Concatenate the groundtruth labels to get 'labels' and move 'labels' to cuda() if is_cuda
# Concatenate the output of 'ftr_extr(s_img) using 's_img' and 'ftr_extr(t_img) using 't_img' to get 'imgs' # and move 'imgs' to cuda if is_cuda
# Forward propagate 'imgs' through 'dom_clsfr(.)' using 'lam' to get the logits for domain classification
# Estimate domain classification loss 'dom_loss' by comparing the logits with the groundtruth using bce_crite rion(.)
# your code here
Y_src = torch.ones(s_img.size(0))
Y_t = torch.zeros(t_img.size(0)) labels = torch.cat((Y_src, Y_t))
imgs = torch.cat((ftr_extr(s_img), ftr_extr(t_img))) if is_cuda: labels = labels.cuda() imgs = imgs.cuda()
logits = dom_clsfr.forward(imgs, lam=lam) logits = torch.reshape(logits, labels.size()) dom_loss = bce_criterion(logits, labels) return dom_loss
Train the DANN Network
DANN training loop.
: 1.000E-04 /usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:105: UserWarning: Implicit dimension ch oice for softmax has been deprecated. Change the call to include dim=X as an argument.
Epoch: 1/10, iter: 200, lambda: 0.12, clf_loss: 1.5022, clf_LR: 2.640E-10, dom_loss: 0.1868, dom_LR : 1.000E-04
Epoch: 1/10, iter: 400, lambda: 0.23, clf_loss: 1.5017, clf_LR: 2.162E-10, dom_loss: 0.1190, dom_LR : 1.000E-04
Epoch: 1/10, iter: 600, lambda: 0.33, clf_loss: 1.5270, clf_LR: 1.770E-10, dom_loss: 0.0938, dom_LR : 1.000E-04
Epoch: 1/10, iter: 800, lambda: 0.43, clf_loss: 1.5059, clf_LR: 1.449E-10, dom_loss: 0.1209, dom_LR : 1.000E-04
Epoch: 2/10, iter: 0, lambda: 0.46, clf_loss: 1.4920, clf_LR: 1.362E-10, dom_loss: 0.1151, dom_LR : 1.000E-04
Epoch: 2/10, iter: 200, lambda: 0.55, clf_loss: 1.5022, clf_LR: 1.115E-10, dom_loss: 0.1743, dom_LR : 1.000E-04
Epoch: 2/10, iter: 400, lambda: 0.62, clf_loss: 1.5017, clf_LR: 9.125E-11, dom_loss: 0.1032, dom_LR : 1.000E-04
Epoch: 2/10, iter: 600, lambda: 0.69, clf_loss: 1.5270, clf_LR: 7.470E-11, dom_loss: 0.0825, dom_LR : 1.000E-04
Epoch: 2/10, iter: 800, lambda: 0.75, clf_loss: 1.5059, clf_LR: 6.115E-11, dom_loss: 0.1090, dom_LR : 1.000E-04
Epoch: 3/10, iter: 0, lambda: 0.76, clf_loss: 1.4920, clf_LR: 5.748E-11, dom_loss: 0.1138, dom_LR : 1.000E-04
Epoch: 3/10, iter: 200, lambda: 0.81, clf_loss: 1.5022, clf_LR: 4.705E-11, dom_loss: 0.1657, dom_LR : 1.000E-04
Epoch: 3/10, iter: 400, lambda: 0.84, clf_loss: 1.5017, clf_LR: 3.852E-11, dom_loss: 0.0880, dom_LR : 1.000E-04
Epoch: 3/10, iter: 600, lambda: 0.87, clf_loss: 1.5270, clf_LR: 3.153E-11, dom_loss: 0.0680, dom_LR : 1.000E-04
Epoch: 3/10, iter: 800, lambda: 0.90, clf_loss: 1.5059, clf_LR: 2.582E-11, dom_loss: 0.0950, dom_LR : 1.000E-04
Epoch: 4/10, iter: 0, lambda: 0.91, clf_loss: 1.4920, clf_LR: 2.426E-11, dom_loss: 0.1008, dom_LR : 1.000E-04
Epoch: 4/10, iter: 200, lambda: 0.92, clf_loss: 1.5022, clf_LR: 1.986E-11, dom_loss: 0.1590, dom_LR : 1.000E-04
Epoch: 4/10, iter: 400, lambda: 0.94, clf_loss: 1.5017, clf_LR: 1.626E-11, dom_loss: 0.0729, dom_LR : 1.000E-04
Epoch: 4/10, iter: 600, lambda: 0.95, clf_loss: 1.5270, clf_LR: 1.331E-11, dom_loss: 0.0525, dom_LR : 1.000E-04
Epoch: 4/10, iter: 800, lambda: 0.96, clf_loss: 1.5059, clf_LR: 1.090E-11, dom_loss: 0.0832, dom_LR : 1.000E-04
Epoch: 5/10, iter: 0, lambda: 0.96, clf_loss: 1.4920, clf_LR: 1.024E-11, dom_loss: 0.0960, dom_LR : 1.000E-04
Epoch: 5/10, iter: 200, lambda: 0.97, clf_loss: 1.5022, clf_LR: 8.385E-12, dom_loss: 0.1461, dom_LR : 1.000E-04
Epoch: 5/10, iter: 400, lambda: 0.98, clf_loss: 1.5017, clf_LR: 6.864E-12, dom_loss: 0.0627, dom_LR : 1.000E-04
Epoch: 5/10, iter: 600, lambda: 0.98, clf_loss: 1.5270, clf_LR: 5.619E-12, dom_loss: 0.0451, dom_LR : 1.000E-04
Epoch: 5/10, iter: 800, lambda: 0.99, clf_loss: 1.5059, clf_LR: 4.600E-12, dom_loss: 0.0683, dom_LR : 1.000E-04
Epoch: 6/10, iter: 0, lambda: 0.99, clf_loss: 1.4920, clf_LR: 4.324E-12, dom_loss: 0.0958, dom_LR : 1.000E-04
Epoch: 6/10, iter: 200, lambda: 0.99, clf_loss: 1.5022, clf_LR: 3.539E-12, dom_loss: 0.1343, dom_LR : 1.000E-04
Epoch: 6/10, iter: 400, lambda: 0.99, clf_loss: 1.5017, clf_LR: 2.898E-12, dom_loss: 0.0536, dom_LR : 1.000E-04
Epoch: 6/10, iter: 600, lambda: 0.99, clf_loss: 1.5270, clf_LR: 2.372E-12, dom_loss: 0.0400, dom_LR : 1.000E-04
Epoch: 6/10, iter: 800, lambda: 0.99, clf_loss: 1.5059, clf_LR: 1.942E-12, dom_loss: 0.0584, dom_LR : 1.000E-04
Epoch: 7/10, iter: 0, lambda: 1.00, clf_loss: 1.4920, clf_LR: 1.825E-12, dom_loss: 0.0893, dom_LR : 1.000E-04
Epoch: 7/10, iter: 200, lambda: 1.00, clf_loss: 1.5022, clf_LR: 1.494E-12, dom_loss: 0.1261, dom_LR : 1.000E-04
Epoch: 7/10, iter: 400, lambda: 1.00, clf_loss: 1.5017, clf_LR: 1.223E-12, dom_loss: 0.0506, dom_LR : 1.000E-04
Epoch: 7/10, iter: 600, lambda: 1.00, clf_loss: 1.5270, clf_LR: 1.001E-12, dom_loss: 0.0352, dom_LR : 1.000E-04
Epoch: 7/10, iter: 800, lambda: 1.00, clf_loss: 1.5059, clf_LR: 8.197E-13, dom_loss: 0.0564, dom_LR : 1.000E-04
Epoch: 8/10, iter: 0, lambda: 1.00, clf_loss: 1.4920, clf_LR: 7.704E-13, dom_loss: 0.0904, dom_LR : 1.000E-04
Epoch: 8/10, iter: 200, lambda: 1.00, clf_loss: 1.5022, clf_LR: 6.307E-13, dom_loss: 0.1149, dom_LR : 1.000E-04
Epoch: 8/10, iter: 400, lambda: 1.00, clf_loss: 1.5017, clf_LR: 5.163E-13, dom_loss: 0.0472, dom_LR : 1.000E-04
Epoch: 8/10, iter: 600, lambda: 1.00, clf_loss: 1.5270, clf_LR: 4.227E-13, dom_loss: 0.0323, dom_LR : 1.000E-04
Epoch: 8/10, iter: 800, lambda: 1.00, clf_loss: 1.5059, clf_LR: 3.460E-13, dom_loss: 0.0499, dom_LR : 1.000E-04
Epoch: 9/10, iter: 0, lambda: 1.00, clf_loss: 1.4920, clf_LR: 3.252E-13, dom_loss: 0.0881, dom_LR : 1.000E-04
Epoch: 9/10, iter: 200, lambda: 1.00, clf_loss: 1.5022, clf_LR: 2.662E-13, dom_loss: 0.1024, dom_LR
: 1.000E-04
Epoch: 9/10, iter: 400, lambda: 1.00, clf_loss: 1.5017, clf_LR: 2.180E-13, dom_loss: 0.0469, dom_LR
: 1.000E-04
Epoch: 9/10, iter: 600, lambda: 1.00, clf_loss: 1.5270, clf_LR: 1.784E-13, dom_loss: 0.0320, dom_LR
: 1.000E-04
Epoch: 9/10, iter: 800, lambda: 1.00, clf_loss: 1.5059, clf_LR: 1.461E-13, dom_loss: 0.0472, dom_LR
: 1.000E-04
Epoch: 10/10, iter: 0, lambda: 1.00, clf_loss: 1.4920, clf_LR: 1.373E-13, dom_loss: 0.0784, dom_L
R: 1.000E-04
Epoch: 10/10, iter: 200, lambda: 1.00, clf_loss: 1.5022, clf_LR: 1.124E-13, dom_loss: 0.0935, dom_L
R: 1.000E-04
Epoch: 10/10, iter: 400, lambda: 1.00, clf_loss: 1.5017, clf_LR: 9.201E-14, dom_loss: 0.0438, dom_L
R: 1.000E-04
Epoch: 10/10, iter: 600, lambda: 1.00, clf_loss: 1.5270, clf_LR: 7.532E-14, dom_loss: 0.0280, dom_L
R: 1.000E-04
Epoch: 10/10, iter: 800, lambda: 1.00, clf_loss: 1.5059, clf_LR: 6.166E-14, dom_loss: 0.0416, dom_L
R: 1.000E-04
Evaluate the DANN Network
Calculate accuracy on source and target datasets using DANN. Observe the improvement in the target classification accuracies with the alignment of the source and target datasets. You should observe the accuracy of the target increase by at least 10 points compared to Source-only training. The settings mentioned here yielded a target test accuracy of 71%. Modify the network and the training procedure to the extent allowed to improve the target classification accuracies.
In [81]:
src_train_acc2 = evaluate_model(src_trX, src_trY) src_test_acc2 = evaluate_model(src_tsX, src_tsY) tgt_train_acc2 = evaluate_model(tgt_trX, tgt_trY) tgt_test_acc2 = evaluate_model(tgt_tsX, tgt_tsY)
print("With Domain Adversarial Training:\nSource train acc: %.2f\nSource test acc: %.2f\nTarget train acc: %.2f\n Target test acc: %.2f" \
%(src_train_acc2, src_test_acc2, tgt_train_acc2, tgt_test_acc2))
# Hidden testcases follow
# Test improvement of target accuracy by at least
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:105: UserWarning: Implicit dimension ch oice for softmax has been deprecated. Change the call to include dim=X as an argument.
With Domain Adversarial Training:
Source train acc: 91.89
Source test acc: 90.44
Target train acc: 71.06
Target test acc: 72.57
