csc2516 Final Project Solution

1 Logistics
For the course project, you will implement a research idea related to the course material. The purpose of the final project is to give you some experience working on a piece of original research and writing up your results in a paper style format. You are expected to describe your research idea/application clearly in the project proposal, relate to existing work. You will document the project progress in the final report.
You must form a group of two or three to complete the project. Your report must clearly list the contributions of each team member. Once your group is formed, please sign up your group through Quercus. Instructions of sign-up can be found here: https://qstudents. utoronto.ca/group-tool-the-student-side-of-things/
3 Large Language Model Policy
4 Writing format
• 1/4 page introduction
• 1/2 page related works
• 1/2 page method / algorithm
• 1/4 page abstract and reference
The point of the proposal is mainly for us to give you feedback and formulate a plan for the final report. The proposal will NOT be graded. We will set up project consultation appointments after we have collected all the project proposals. You will submit your proposal report through Qeurcus.
Final report: You will expand out your project proposal to include experiments and comprehensive method sections. You are expected to discuss the experimental results in details and highlight any interesting findings. We recommend the final report to be FOUR pages plus the references. Appendix is allowed with no page limit, but note that the teaching staffs reserve the right to judge the final project solely on the basis of the 4 pages of the main report; looking at any extra material is up to the discretion of the reviewers and is not required. You will submit your final report through Quercus. You must also submit the code necessary to reproduce your experiments.
5 How to choose a project
The course projects should build top of the course materials. You are encouraged to use neural networks as the function approximators for your method or application. There are two categories of projects to choose from.
Understanding and analysis: For the students who would like to have a more in-depth understanding of the course material, it is often a good idea to re-implement an existing method and re-evaluate the implementation against some standard benchmarks.
• Reproduce the experimental results from some existing papers. Perform sensitivity analysis on hyperparameters.
• Apply / extend existing algorithms to a new application / task / dataset.
If you choose to work on this of this category, you will need to implement and analyze the performance of at least two different deep learning algorithms / methods in a task domain, e.g., image recognition or natural language processing. You are asked to discuss the strength and weakness of each of the approaches backed by your experimental findings.
Doing a proper analysis for the existing methods is non-trivial. Here are two great examples of this type of study: Visualizing and Understanding Convolutional Networks https://arxiv.org/pdf/1311.2901.pdf and https://arxiv.org/pdf/1506.02078.pdf
• Improve / fix an existing algorithm. Evaluate the improvement on benchmark environments.
• Develop novel model architectures / algorithms to a new application / area / environment.
If you decide to work on a research idea, you will need to implement and compare the performance of your method against at least one existing approach in your problem.
Here is some advice on picking a good research problem from Bill Freeman: https://billf.mit.edu/sites/default/files/documents/cvprPapers.pdf and from David Patterson’s slides part III and IV: https://people.eecs.berkeley.edu/ pattrsn/talks/BadCareer.pdf.
You are welcome to do a project related to your research. In this case, your project proposal and final report must each clearly explain the relationship to your research, what work was already done prior to the course, and what work (if any) was done by people not on the project team.Our expectations will be higher in this case.
6 Grading scheme
• Quality [35%] Is the report technically sound? Are claims well-supported by theoretical analysis or experimental results? Is this a complete piece of work, or merely a position report? Are the authors careful (and honest) about evaluating both the strengths and weaknesses of the work? To get full mark in this category, you will need to include at least one of:
– An algorithm box.
– Equations describing your model.
– A theorem or formally stated conjecture.
• Clarity [25%] Is the report clearly written? Is it well-organized? (If not, feel free to make suggestions to improve the manuscript.) Does it adequately inform the reader? Are the figures/tables properly labeled? (A superbly written report provides enough information for the expert reader to reproduce its results.)
• Originality [20%] Are the problems or approaches new? Is this a novel combination of familiar techniques? Is it clear how this work differs from previous contributions? Is related work adequately referenced? We recommend that you check the proceedings of recent NIPS conferences to make sure that each report is significantly different from papers in previous proceedings. Abstracts and links to many of the previous NeurIPS papers are available from https://papers.nips.cc/.
• Significance [5%] Are the results important? Are other people (practitioners or researchers) likely to use these ideas or build on them? Does the report address a difficult problem in a better way than previous research? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions on existing data, or a unique theoretical or pragmatic approach?
7 Lateness
There is NO late acceptance for the project proposal, as it is not graded. See Sec 4 for more info about the proposal.