In this assignment you will practice writing backpropagation code, and training Neural Networks and Convolutional Neural Networks. The goals of this assignment are as follows:
understand Neural Networks and how they are arranged in layered architectures
understand and be able to implement (vectorized) backpropagation
implement various update rules used to optimize Neural Networks
implement batch normalization for training deep networks
implement dropout to regularize networks
effectively cross-validate and find the best hyperparameters for Neural Network architecture
understand the architecture of Convolutional Neural Networks and train gain experience with training these models on data
Q1: Fully-connected Neural Network (Completed)
The IPython notebook FullyConnectedNets.ipynb will introduce you to our modular layer design, and then use those layers to implement fully-connected networks of arbitrary depth. To optimize these models you will implement several popular update rules.
Q2: Batch Normalization (Completed)
In the IPython notebook BatchNormalization.ipynb you will implement batch normalization, and use it to train deep fully-connected networks.
Q3: Dropout (Completed)
The IPython notebook Dropout.ipynb will help you implement Dropout and explore its effects on model generalization.
Q4: ConvNet on CIFAR-10 (A Little Bit Left)
In the IPython Notebook ConvolutionalNetworks.ipynb you will implement several new layers that are commonly used in convolutional networks. You will train a (shallow) convolutional network on CIFAR-10, and it will then be up to you to train the best network that you can.
