COMP90049 -  Introduction to Machine Learning - Project 3 - Solved

 Romance or Thriller? Movie Genre Prediction from Audio, Visual, and Text Features!

Task:                       Build a movie genre classifier



Overview
The goal of this project is to build and critically analyse some supervised Machine Learning algorithms, to automatically identify the genre(s) of a movie on the basis of its audio, visual and textual (metadata) features. That is, given a list of movies, your job is to come up with one or more implemented Machine Learning model(s), which are trained using the training dataset, and evaluated using the validation and test dataset.

This project aims to reinforce the largely theoretical machine learning concepts around models, data, and evaluation covered in the lectures, by applying them to an open-ended problem. You will also have an opportunity to practice your general problem-solving skills, written communication skills, and creativity. This project has two stages.

The focus of this assignment will be the report, where you will demonstrate the knowledge that you have gained, in a manner that is accessible to a reasonably informed reader.



Dataset
Each movie (instance) is represented through a large set of features (described in detail in the README), and listed in the features.tsv files. Each movie is labelled with a single genre tag, which is provided int the labels.tsv files.

The data files are available via the LMS. You will be provided with a set of training documents, validation documents and a set of test documents. The files are provided in tsv format, which stands for tab-separated values.

•    train features.tsv: Contains features of 5240 training instances.

•    train labels.tsv: Contains a single genre label for each training instance

•    valid features.tsv: Contains features of 299 validation instances.

•    valid labels.tsv: Contains a single genre label for each validation instance.

•    test features.tsv: Contains features of 298 test instances.

Each movie in the data set is indexed with a unique movieID. We provide three different types of features. Details are provided in the README file, as well as the references listed under Terms of Use:

•    Metadata features: For each movie, we provide its title, its year of release, and a list of tags (like ‘predictable’, ‘boring’, or ‘based on a book’) provided by human annotators.

•    Visual features: We provide 107 pre-computed visual features, pre-extracted from the movie trailer. Each feature has a continuous floating point value. Note that these features are not interpretable.

•    Audio features: We provide 20 pre-computed audio features capturing acoustic signals from the movie trailer. Again, each feature takes a continuous value, and the values are not interpretable.

In addition to those features, we also provide the following information which you may find useful in your error analysis (but are not required to use):

•    A youtube link pointing to the movie trailer. We don’t guarantee that all links are functional. Each movie is labelled with its genre, i.e., with a single label from one of 18 possible genre labels:

’Action’, ’Adventure’, ’Animation’, ’Children’, ’Comedy’, ’Crime’, ’Documentary’, ’Drama’, ’Fantasy’, ’Film noir’, ’Horror’, ’Musical’, ’Mystery’, ’Romance’, ’Sci fi’, ’Thriller’, ’War’, ’Western’

Task
You will develop machine learning models which predict the movie genre based on a diverse set of features, capturing acoustic and visual features of the movie trailers, as well as movie metadata such as its title and human-provided tags. You will implement and analyze different machine learning models in their performance; and explore the utility of the different types of features for movie genre prediction.

We will use a hold-out strategy to evaluate the trained model using a validation, and a test set:

1.    The training phase: This will involve training your classifier(s) and parameter tuning where required.

You will use the train  features and train labels files.

2.    The validation phase: This is where you observe the performance of the classifier(s). The validation data is labelled: you should run the classifier that you built in the training phase on this data to calculate one or more evaluation metrics to discuss in your report. This phase will help you to find the best model that can be used for the testing phase.

3.    The testing phase: The test data is unlabeled; you should use your preferred model to produce a prediction for each test instance, and submit your predictions to the Kaggle website; we will use this output to confirm the observations of your approach.

N.B: Various machine learning techniques have been (or will be) discussed in this subject (Naive Bayes, Decision Trees, 0-R, etc.); many more exist. You are strongly encouraged to make use of machine learning software and/or existing libraries (such as sklearn) in your attempts at this project.



Report
The report should be 1,500-2,000 words in length and should follow the structure of a short research paper. It should provide a basic descriptions of:

1.    The task, and a short summary of some related literature

2.    What you have done, including any learners that you have used, or features that you have engineered[2]

3.    Evaluation of your classifier(s) over the validation dataset

You should also aim to have a more detailed discussion, which:

4.    Contextualises the behaviour of the method(s), in terms of the theoretical properties we have identified in the lectures

5.    Attempt some error analysis of the method(s)

And don’t forget:

6.    A bibliography, which includes Deldjoo et al. (2018), Harper et al. (2015), and other related work

Note that we are more interested in seeing evidence of you having thought about the task and determined reasons for the relative performance of different methods, rather than the raw scores of the different methods you select. This is not to say that you should ignore the relative performance of different runs over the data, but rather that you should think beyond simple numbers to the reasons that underlie them.

We will provide LATEXand RTF style files that we would prefer that you use in writing the report. Reports are to be submitted in the form of a single PDF file. If a report is submitted in any format other than PDF, we reserve the right to return the report with a mark of 0.

Your name and student ID should not appear anywhere in the report, including any metadata (filename, etc.). If we find any such information, we reserve the right to return the report with a mark of 0.



Reflective writing
A comprehensive written reflection piece summarizing your critical reflection on this project within 400-800 words. This report is not anonymous.



 
[1] https://www.kaggle.com/
[2] This should be at a conceptual level; a detailed description of the code is not appropriate for the report.