General overview: In this homework you will learn how to implement and test simple neural network models for solving supervised problems. It is divided in two tasks. The regression task will consist in a simple function approximation problem, similar to the one discussed during the Lab practices. The classification task will consist in a simple image recognition problem, where the goal is to correctly classify images of Zalando's article images (FashionMNIST).
In both cases, you should explore the use of advanced optimizers and regularization methods (e.g., initialization scheme, momentum, ADAM, early stopping, L2, L1 / sparsity, dropout…) to improve convergence of stochastic gradient descent and promote generalization. Learning hyperparameters should be tuned using appropriate search procedures, and final accuracy should be evaluated using a crossvalidation setup. For the image classification task, you can also implement more advanced convolutional architectures and explore advanced feature visualization techniques (e.g., visualization of feature maps and/or input optimization by maximizing the activation gradient) to better understand how the deep network is encoding information at different processing layers.
Additional information and details related to the datasets are provided in a Python notebook uploaded in the course Moodle.
Technical notes: The homework should be implemented in Python using the PyTorch framework. The student can explore additional libraries and tools to implement the models; however, please make sure you understand the code you are writing because during the exam you might receive specific questions related to your implementation. The entire source code required to run the homework must be uploaded as a compressed archive in a Moodle section dedicated to the homework. If your code will be entirely included in a single Python notebook, just upload the notebook file.
