You may complete this homework assignment either individually or in teams up to 2 people.
Principal Component Analysis (PCA) [30 points]: In this problem you will implement Principal Component Analysis (PCA). Note that you are required to implement PCA from scratch; you may not use any o -the-shelf software (e.g., sklearn). Apply the PCA algorithm you implemented to the 10,000 examples in the MNIST test set. (There’s no deep conceptual reason for this choice; it’s just because the test set is smaller and will reduce the computation time.) In particular, for each image x in the MNIST test set, project it onto the rst and second principal components, i.e., the two eigenvectors
~ ~
of XX with the highest and second-highest associated eigenvalues. Call these two projections p1 and p2; then plot (p1; p2) of each MNIST image on a 2-D plane, where the color of each point is determined by the class label of that image x. You should obtain an image similar to the following. Note, however, that the colors in your image could di er (since they are arbitrary); also, the orientation of the image coudl also be di erent (since the eigenvectors of a given matrix are not unique).
You should attach the gure that you rendered as either a PNG or a PDF.
Submit your Python code in a le called homework5 WPIUSERNAME1.py
(or homework5 WPIUSERNAME1 WPIUSERNAME2.py for teams). Submit your gure (either a PDF or PNG) using an analogous naming convention.
1
