CSC Final Project Solution

    • Logistics

For the course project, you will implement a research idea related to the course material. The purpose of the nal project is to give you some experience working on a piece of original research and writing up your results in a paper style format. You are expected to describe your research idea/application clearly in the project proposal, relate to existing work. You will document the project progress in the nal report.

You must form a group of two or three to complete the project. Your report must clearly list the contributions of each team member. Once your group is formed, please sign up your group through Quercus. Instructions of sign-up can be found here: https://qstudents. utoronto.ca/group-tool-the-student-side-of-things/

There are two important dates: the initial project proposal is due 11:59 pm March 4th, and the nal report is due 11:59 pm April 15th. The write-ups are to be submitted through Quercus1. The policy regarding late submission can be found in Section 4.


    • Writing format

All submissions must be in PDF format. You may include algorithm blocks, tables, and gures. The write-ups should be prepared in the NeurIPS paper format: https://nips.cc/Conferences/2021/PaperInformation/StyleFiles.

You may    nd online editors such as Overleaf helpful for writing the reports:

https://www.overleaf.com/latex/templates/neurips-2021/bfjnthbqvhgs

Proposal: The project proposal is limited to two pages. It should roughly have the following sections:

    • 1/4 page introduction

    • 1/2 page related works

    • 1/2 page method / algorithm

    • 1/4 page abstract and reference

The point of the proposal is mainly for us to give you feedback and formulate a plan for the nal report. The proposal will NOT be graded. We will set up project consultation appointments after we have collected all the project proposals. You will submit your proposal report through Qeurcus.

Final report: You will expand out your project proposal to include experiments and com-prehensive method sections. You are expected to discuss the experimental results in details and highlight any interesting ndings. We recommend the nal report to be FOUR pages plus the references. Appendix is allowed with no page limit, but note that the teaching sta s reserve the right to judge the nal project solely on the basis of the 4 pages of the main report; looking at any extra material is up to the discretion of the reviewers and is not required. You will submit your nal report through Quercus. You must also submit the code necessary to reproduce your experiments.


    • https://q.utoronto.ca/courses/250829

1
CSC413 / CSC2516      Final Project



    • How to choose a project

The course projects should build top of the course materials. You are encouraged to use neural networks as the function approximators for your method or application. There are two categories of projects to choose from.

Understanding and analysis: For the students who would like to have a more in-depth understanding of the course material, it is often a good idea to re-implement an existing method and re-evaluate the implementation against some standard benchmarks.

    • Reproduce the experimental results from some existing papers. Perform sensitivity analysis on hyperparameters.

    • Apply / extend existing algorithms to a new application / task / dataset.

If you choose to work on this of this category, you will need to implement and analyze the perfor-mance of at least two di erent deep learning algorithms / methods in a task domain, e.g., image recognition or natural language processing. You are asked to discuss the strength and weakness of each of the approaches backed by your experimental ndings.

Doing a proper analysis for the existing methods is non-trivial. Here are two great examples of

this type of study: Visualizing and Understanding Convolutional Networks https://arxiv.org/pdf/1311.2901.pdf and https://arxiv.org/pdf/1506.02078.pdf

Exploratory research: You may also choose to work on a novel research idea that may lead to a potential publication. The examples of such projects are:

    • Improve /  x an existing algorithm. Evaluate the improvement on benchmark environments.

    • Develop novel model architectures / algorithms to a new application / area / environment.

If you decide to work on a research idea, you will need to implement and compare the performance of your method against at least one existing approach in your problem.

Here is some advice on picking a good research problem from Bill Freeman: https://billf.mit.edu/sites/default/ les/documents/cvprPapers.pdf and from David Patterson’s slides part III and IV: https://people.eecs.berkeley.edu/ pattrsn/talks/BadCareer.pdf.

You are welcome to do a project related to your research. In this case, your project proposal and nal report must each clearly explain the relationship to your research, what work was already done prior to the course, and what work (if any) was done by people not on the project team.Our expectations will be higher in this case.

    • Grading scheme


You may receive full marks for the course project by choosing either of the two categories. There will be no advantage for choosing an exploratory project over an analysis one regarding achieving a higher grade. The goal of the project is for you and your group to conduct original research. The proposal and the nal report will be graded according to the criteria of top machine learning conference submissions. We will use the NeurIPS review criteria for this purpose:




2
CSC413 / CSC2516      Final Project



    • Quality [35%] Is the report technically sound? Are claims well-supported by theoretical analysis or experimental results? Is this a complete piece of work, or merely a position report? Are the authors careful (and honest) about evaluating both the strengths and weaknesses of the work? To get full mark in this category, you will need to include at least one of:

{ An algorithm box.

{ Equations describing your model.

{ A theorem or formally stated conjecture.

    • Clarity [25%] Is the report clearly written? Is it well-organized? (If not, feel free to make suggestions to improve the manuscript.) Does it adequately inform the reader? Are the gures/tables properly labeled? (A superbly written report provides enough information for the expert reader to reproduce its results.)

    • Originality [20%] Are the problems or approaches new? Is this a novel combination of familiar techniques? Is it clear how this work di ers from previous contributions? Is related work adequately referenced? We recommend that you check the proceedings of recent NIPS conferences to make sure that each report is signi cantly di erent from papers in previous proceedings. Abstracts and links to many of the previous NeurIPS papers are available from https://papers.nips.cc/.

    • Signi cance [5%] Are the results important? Are other people (practitioners or researchers) likely to use these ideas or build on them? Does the report address a di cult problem in a better way than previous research? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions on existing data, or a unique theoretical or pragmatic approach?

    • Participation [15%] We will adopt a peer-review system through Quercus in which students will participate in reviewing the other classmates’ reports. Reports will be made anonymously. We expect each student to review at least 2 reports. The participation score will be given based on the quality of the reviews by each student. You may nd the following link helpful re-garding how to write a good review: https://iclr.cc/Conferences/2021/ReviewerGuide.


Late submission on Quercus, except in the case of an o cial Student Medical Certi cate, will be accepted with 10% penalty every 24 hours from the deadline up-to three full days. So, you will get 0% on the assignment if it is submitted 4 days late.


















3