Final Project Solution

Overview

The Course Project is an opportunity for you to practice what you have learned from this course and apply to some problems that you are interested in. There are two types of projects, Type-A and Type-B.

Type-A Project: This type of project is about small-size neural network on small-size dataset. To finish this type of project, you do not need a powerful GPU as computing platform. The aim of this project is to help you have a deeper understanding the basic principle of deep neural network and how it works.
Type-B Project: This type of project is about some advanced topics of neural network and its applications. This type of work might be challenging since it may require train and test large-size neural network on large-size dataset. You may need GPU for your experiment.

Topics of Type‐A and Type‐B Projects

The following is some example topics you can use for your Type-A and Type-B. Available Type-A Project Topics
    1) Evaluate the performance of different types of optimizer on a LeNet-5 network using MNIST data. At least you need to evaluate SGD, AdaGrad, RMSprop.
    2) Evaluate the performance of model ensemble on a LeNet-5 network using MNIST dataset. You need to evaluate and compare the performance when ensembling 3, 5, 7 network models.

    3) Evaluate the performance of dropout on fully-connected layers and batch normalization on convolutional layers. The model is LeNet-5 on MNIST dataset. Among four options 1) FC with dropout, CONV with BN; 2) FC with dropout, CONV without BN; 3) FC without dropout, CONV with BN; 4) FC without dropout, CONV without dropout, identify which one has the best performance?

Available Type-B Project Topics

    4) Manually implement a fully-connected NN using Numpy (no automatic gradient). Train and test on MNIST dataset. The 90% accuracy should be achieved.
    5) Manually implement a convolutional NN using Numpy (no automatic gradient). Train and test on MNIST dataset. The 93% accuracy should be achieved.
    6) Manually implement a Recurrent NN using Numpy (no automatic gradient) on MNIST datsaset. 90% accuracy should be achieved.

    7) Prune a LeNet-5 convolutional NN on MNIST dataset with high compression ratio and negligible performance loss.
    8) Launch one type of adversarial attack for LeNet-5 on MNIST dataset.

    9) Apply deep neural network to a domain problem: e.g. plate number recognition, image caption, blood cell recognition, digital communication, transportation etc.

Name Your Own Type‐B Project

You can propose a topic for your Type-B project. The complexity of your proposed Type-B project should be higher than Type-A project.
A one-page project proposal is needed if you would like to name your own Type-B project. The project proposal should include: the motivation of this topic, the planned used data, the state-of-the-art approaches and your planned selection, evaluation metrics.

The project proposal should be submitted and approved by instructor.

Collaboration Policy

Type-A project should be done individually.

Type-B project can be done in a team of 2 persons.

Honor Code

You may refer to public and online resource when preparing your project as long as sufficient and clear citation and notation is presented.
Incorporating other students’ codes into your project is forbidden.

Submission Guideline

The due date of final submission is May 14.

The final submission includes final report and source codes.

The final submission should be submitted in the ZIP/RAR format.

Your final report should include abstract, introduction, related work, data description, method description, model description, experimental procedure and results, conclusion and reference.
At least one Type-A or Type-B project should be done and submitted. If multiple projects are submitted, the highest score among those submissions will be counted to your course score.