1 Sentiment Classifier
I developed the classifier described in the exercise, and experimented with all the possible hyper-parameters. All the comments and the observations are included in the python notebook. My best model is a bi-directional stacked Reccurent Neural Network, consisted of the following hyperparameters:
• Cell type: LSTM
• Dropout Rate: 20
• Learing rate: 0.001
• Hidden layer neurons: 200
• Gradient clipping rate: (-100, 100)
All the other objectives (ROC plot, classification report, predicting the test dataset), are included in the notebook.
2 Attention Layer
I added an attention layer, as well as the pooling one, following the link that was provided in the MNIST LSTM notebook, which was integrated to my model class.
1
Then, I tested my best model by applying attention, and the results were slightly better than the original one, while avoiding overfitting, however the training was significantly slower.
2
