Homework 4: Clustering, Compression and Image Processing Solution

For this assignment, we will focus on two main aspects: (1) image processing and convolutional computations, and (2) unsupervised machine learning with K-Means and processing. Along the way we will use several “standard” machine learning datasets and touch on artificial neurons. The Advanced portion implements text comparisons such as the Smith-Waterman algorithm.




NOTE: This is the last of four assignments that consists of a Basic component, to be done by everyone, and an Advanced component, to be done by students who wish to do 3 homeworks and a project. Please see the separate steps below for the Advanced component. If you are on the “Advanced” track and want to do a project, note that only your top three Homeworks will be counted.

Homework 4: KMeans and Images
As usual, you should first open Jupyter and a Terminal, then clone the Homework 4 Bitbucket repository:




git clone https://upenn-cis@bitbucket.org/pennbigdataanalytics/hw4.git




Then go into the hw4 directory in Jupyter.




The Homework 4 has 2 notebooks: Homework-4-Images.ipynb and Homework-4-KMeans.ipynb. Please update and submit both notebooks. The notebooks can be worked on independently.




In general, we’ve tried to be clear with the column names and headers necessary - whenever we have a variable created in the form of:

lena_3D_arr = ()

lena_2D_arr = ()




Feel free to replace these with your own code. Note that you should either overwrite these, or put your code below - we set the variables to just unit items () so it’s clear what the variables are.




S3 Links: You’ll notice that we have try-catch blocks in the code, some of which are editable. These are used for grading only, and our S3 bucket with the dataset copies is not accessible until we grade.

Submitting Homework 4
Retrieve from your Docker container the following notebook files and zip them into hw4.zip, much as you did for previous homework assignments. The notebooks should be:




Homework-4-KMeans.ipynb
Homework-4-Images.ipynb



Next, go to the submission site, and if necessary click on the Google icon and log in using your Google@SEAS or GMail account. At this point the system should know you are in the appropriate course. Select hw4-2019 and upload hw4.zip from your Jupyter folder, typically found under /Users/{myid}.




If you check on the submission site after a few minutes, you should see whether your submission passed validation. Validation only checks if you have the correct names. You may resubmit as necessary.

Homework 4 Advanced
You should make sure the basic assignment is complete before moving on to this part. To start, go to Jupyter Notebook in your web browser (http://localhost:8888/tree with the big token as before). Click on your work directory, then New|Terminal. Run:




git clone https://bitbucket.org/pennbigdataanalytics/hw4-adv.git




To get the new notebook.

What to work on
In Jupyter’s tree browser, go into hw4-adv and work on Homework-4-Advanced.ipynb.

Submitting Homework 4-Advanced
Go to the submission site, and if necessary click on the Google icon and log in using your Google@SEAS or GMail account. At this point the system should know you are in the appropriate course. Select hw4adv-2019 and upload hw4adv.zip from your Jupyter/hw4-adv folder, typically found under /Users/{myid}.