CSC413 Final Project Solution

• Logistics

For the course project, you will implement a research idea related to the course material. The purpose of the nal project is to give you some experience working on a piece of original research and writing up your results in a paper style format. You are expected to describe your research idea/application clearly in the project proposal, relate to existing work. You will document the project progress in the nal report.

You must form a group of two or three to complete the project. Your report must clearly list the contributions of each team member. Once your group is formed, please sign up your group through Quercus. Instructions of sign-up can be found here: https://qstudents. utoronto.ca/group-tool-the-student-side-of-things/

There are two important dates: the initial project proposal is due 11:59 pm, and the nal report is due 11:59 pm. The write-ups are to be submitted through Quercus1. The policy regarding late submission can be found in Section 6.

• Academic Integrity

Any work or ideas that are not your own must be appropriately acknowledged with a citation in the nal report. This include any code used for the experiments and text in the report itself. It is important to note that plagiarism and other forms of academic misconduct are not tolerated in this course. The University of Toronto’s Code of Behaviour on Academic Matters outlines the behaviours that constitute academic misconduct, the processes for addressing academic o enses, and the penalties that may be imposed. You are expected to be familiar with the contents of this document.

• Large Language Model Policy

We will adopt a similar guideline to the International Conference on Machine Learning 2023 policy about the usage of Large Language Models (LLMs), such as ChatGPT, for the nal report. The LLM policy prohibits text produced entirely by LLMs (i.e., \generated"). This does not prohibit authors from using LLMs for editing or polishing author-written text.

The LLM policy is largely predicated on the principle of being conservative with respect to guarding against potential issues of using LLMs, including plagiarism.

• Writing format

All submissions must be in PDF format. You may include algorithm blocks, tables, and gures.

The write-ups should be prepared in the NeurIPS paper format:

https://nips.cc/Conferences/2021/PaperInformation/StyleFiles.

You may nd online editors such as Overleaf helpful for writing the reports:

https://www.overleaf.com/latex/templates/neurips-2022/kxymzbjpwsqx

• https://q.utoronto.ca/courses/293810

1

CSC413 / CSC2516 Final Project

Proposal: The project proposal is limited to two pages. It should roughly have the following sections:

• 1/4 page introduction

• 1/2 page related works

• 1/2 page method / algorithm

• 1/4 page abstract and reference

The point of the proposal is mainly for us to give you feedback and formulate a plan for the nal report. The proposal will NOT be graded. We will set up project consultation appointments after we have collected all the project proposals. You will submit your proposal report through Qeurcus.

Final report: You will expand out your project proposal to include experiments and com-prehensive method sections. You are expected to discuss the experimental results in details and highlight any interesting ndings. We recommend the nal report to be FOUR pages plus the references. Appendix is allowed with no page limit, but note that the teaching sta s reserve the right to judge the nal project solely on the basis of the 4 pages of the main report; looking at any extra material is up to the discretion of the reviewers and is not required. You will submit your nal report through Quercus. You must also submit the code necessary to reproduce your experiments.

• How to choose a project

The course projects should build top of the course materials. You are encouraged to use neural networks as the function approximators for your method or application. There are two categories of projects to choose from.

Understanding and analysis: For the students who would like to have a more in-depth understanding of the course material, it is often a good idea to re-implement an existing method and re-evaluate the implementation against some standard benchmarks.

• Reproduce the experimental results from some existing papers. Perform sensitivity analysis on hyperparameters.

• Apply / extend existing algorithms to a new application / task / dataset.

If you choose to work on this of this category, you will need to implement and analyze the perfor-mance of at least two di erent deep learning algorithms / methods in a task domain, e.g., image recognition or natural language processing. You are asked to discuss the strength and weakness of each of the approaches backed by your experimental ndings.

Doing a proper analysis for the existing methods is non-trivial. Here are two great examples of

this type of study: Visualizing and Understanding Convolutional Networks https://arxiv.org/pdf/1311.2901.pdf and https://arxiv.org/pdf/1506.02078.pdf

Exploratory research: You may also choose to work on a novel research idea that may lead to a potential publication. The examples of such projects are:

• Improve / x an existing algorithm. Evaluate the improvement on benchmark environments.

2

CSC413 / CSC2516 Final Project

• Develop novel model architectures / algorithms to a new application / area / environment.

If you decide to work on a research idea, you will need to implement and compare the performance of your method against at least one existing approach in your problem.

Here is some advice on picking a good research problem from Bill Freeman: https://billf.mit.edu/sites/default/ les/documents/cvprPapers.pdf and from David Patterson’s slides part III and IV: https://people.eecs.berkeley.edu/ pattrsn/talks/BadCareer.pdf.

You are welcome to do a project related to your research. In this case, your project proposal and nal report must each clearly explain the relationship to your research, what work was already done prior to the course, and what work (if any) was done by people not on the project team.Our expectations will be higher in this case.

• Grading scheme

You may receive full marks for the course project by choosing either of the two categories. There will be no advantage for choosing an exploratory project over an analysis one regarding achieving a higher grade. The goal of the project is for you and your group to conduct original research. The proposal and the nal report will be graded according to the criteria of top machine learning conference submissions. We will use the NeurIPS review criteria for this purpose:

• Quality [35%] Is the report technically sound? Are claims well-supported by theoretical analysis or experimental results? Is this a complete piece of work, or merely a position report? Are the authors careful (and honest) about evaluating both the strengths and weaknesses of the work? To get full mark in this category, you will need to include at least one of:

{ An algorithm box.

{ Equations describing your model.

{ A theorem or formally stated conjecture.

• Clarity [25%] Is the report clearly written? Is it well-organized? (If not, feel free to make suggestions to improve the manuscript.) Does it adequately inform the reader? Are the gures/tables properly labeled? (A superbly written report provides enough information for the expert reader to reproduce its results.)

• Originality [20%] Are the problems or approaches new? Is this a novel combination of familiar techniques? Is it clear how this work di ers from previous contributions? Is related work adequately referenced? We recommend that you check the proceedings of recent NIPS conferences to make sure that each report is signi cantly di erent from papers in previous proceedings. Abstracts and links to many of the previous NeurIPS papers are available from https://papers.nips.cc/.

• Signi cance [5%] Are the results important? Are other people (practitioners or researchers) likely to use these ideas or build on them? Does the report address a di cult problem in a better way than previous research? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions on existing data, or a unique theoretical or pragmatic approach?

3

CSC413 / CSC2516 Final Project

• Participation [15%] We will adopt a peer-review system through Quercus in which students will participate in reviewing the other classmates’ reports. Reports will be made anonymously. We expect each student to review at least 2 reports. The participation score will be given based on the quality of the reviews by each student. You may nd the following link helpful regarding how to write a good review with examples: https://iclr.cc/Conferences/2022/ ReviewerGuide.

• Lateness

There is NO late acceptance for the project proposal, as it is not graded. See Sec 4 for more info about the proposal.

Final project late submission on Quercus will be accepted with 10% penalty every 24 hours from the deadline up-to three full days. Similar to the assignments. So, you will get 0% on the assignment if it is submitted 4 days late. Every student has a total of 7 grace days for both the assignments and the nal project in this course. If you would like to apply grace days to extend the nal project deadline without penalty, the entire group must use one late day per person to extend the deadline by a day. This can be from a single member’s bank of grace days, or the grace days can be evenly distributed among the team members. For example, a team of two members must have a total budget of four grace days remaining in order to extend the deadline for two days.