$18.99
Note: The assignment will be autograded. It is important that you do not use additional libraries, or change the provided functions’ input and output. Also, you should include your best-trained ‘trained weights.np’ under codefromscratch when you make submission to SVN.
Part 1: Setup
• Remote connect to an EWS machine.
ssh (netid)@remlnx.ews.illinois.edu
• Load python module, this will also load pip and virtualenv
module load python/3.4.3
• Reuse the virtual environment from mp0.
source ~/cs446sp_2018/bin/activate
• Copy mp3 into your svn directory, and change directory to mp3.
cd ~/(netid)
svn cp https://subversion.ews.illinois.edu/svn/sp18-cs446/_shared/mp3 . cd mp3
• Install the requirements through pip.
pip install -r requirements.txt
• Create data directory and Download the data into the data directory.
mkdir data
wget --user (netid) --ask-password \
http://courses.engr.illinois.edu/cs446/secure/assignment3_data.zip \
-O data/assignment3_data.zip
• Unzip assignment3 data.zip
unzip data/assignment3_data.zip -d data/
1
• Prevent svn from checking in the data directory.
svn propset svn:ignore data .
Part 2: Exercise
In this exercise, we will build a logistic model based binary classifier that can separate two-
class audio datasets. To be specific, the datasets consist of features extracted from audio waveforms of letters of the alphabet. The two classes represent the ee-set (b,c,d,e,g,p,t,v,z) and the eh-set (f,l,m,n,s,x), respectively. The binary classifier will be based on the logistic regression model from our lecture as well as the sigmoid function from the written homework.
The overall flow of the binary classifier is illustrated in Fig. 1 and we will implement each of the blocks in both “code-from-scratch” and “TensorFlow” parts. In main.py , the overall program structure is provided for you.
Training
Forward Operation (Compute Score/Loss)
Backward Operation (Gradient Update)
Data
(x,y) Preprocessing Model Score Loss
Output
Testing
Forward Operation (Classification)
Figure 1: High-level Flow
Part 2.1 Code-From-Scratch Implementation
• Reading and Preprocessing Data.
In io tools.py , we will fill in one function for reading in the dataset as well as pre- processing it. The downloaded dataset has only “trainset” because the real “testset” belongs to autograder, which is used to evaluate the actual accuracy of your classifier. No matter “trainset” or “testset”, they share the same format. Specifically, the “train- set” has an “indexing.txt” and a folder “samples”, where the former file lists all samples line-by-line in the form of “class label sample file.txt” and the latter folder contains all sample files in ASCII text format. For more details, check the “README.txt”. After
the dataset is read, it should be preprocessed to match the specified form in return arguments of the function. More details are provided in the function docstring.
• Logistic Model
In logistic model.py , we will implement a class for logistic models and use it for binary classification. The class will support the following operations:
– Forward operation. Forward operation is the function which takes input fea- tures and generate a probability score. In this case, for logistic models, it is
1+e−w x
score = g(w| x) = 1 |
, where the sigmoid function is used from the written
homework. Note that, we redefine x = [1, x] and w = [b, w].
– Backward operation. Backward operation is for computing the gradient of the loss function with respect to the model parameters. This is computed after the forward operation to update the model. Note that, in this part, loss function
should use the Pi log 1 + exp(−y(i)wT φ(x(i))) as in the lecture slides.
– Classify operation. Classify operation performs the binary classification on input dataset. Each of input samples is computed based on current update-to- date model and then is predicted as one of the two classes {+1/ − 1}. Since all samples are taken in as input, the final output will be a vector of predicted class labels.
– Fit operation. It is the training function which utilizes the above methods as necessary to fit the logistic model to the input dataset through the gradient descent approach. As in the lecture, gradient descent is an optimization algorithm, where the model adjusts its parameters in direction of the negative gradient of the loss value. The gradient descent should be repeated iteratively until convergence.
– Save/Load operation. After training with hundreds or thousands of itera- tions, the final model should be saved into a binary file using the methods – np.tofile(‘trained weights.np’) and np.fromfile(‘trained weights.np’), so that, in future, the well-trained model can be restored immediately and per- form classification tasks. Note that, you should also include your best- trained ‘trained weights.np’ under codefromscratch when submitting your code to SVN.
• Running Experiments. The main.py shows the overall flow containing all previous methods. We will not grade the main.py file, feel free to modify it.
Relevant Files: everything in the folder codefromscratch
Part 2.2 TensorFlow Implementation
In this part, we will implement the model in the previous section using TensorFlow (TF). The
2
TF part focuses more on training method using the quadratic loss function Pi yi − g(w xi )
where y ∈ {0, 1}, instead of the previous loss function Pi log 1 + exp(−y(i) wT φ(x(i) ))
where y ∈ {−1, 1}. Also, for simplicity, less operations are required to implement in the TF
part.
• Reading and Preprocessing Data. In general, everything is the same except the class labels.
• Logistic Model The class in TF part only needs to support these operations:
– Build Graph operation. This operation builds the entire training and test- ing TF graph for our logistic classifier, which includes model creation, forward score/loss computation, optimizer definition, as well as training accuracy compu- tation. It is better to put the entire TF graph in a single method, according to our experience.
– Fit operation. The training operation in TF utilizes the quadratic loss as men- tioned earlier, which also includes iteratively gradient descent update on weights as well as computation of forward score and training accuracy during each iter- ation. After the model is trained, this operation also returns the score vector of last iteration.
• Running Experiments. The main.py shows the overall flow containing all previous methods. We will not grade the main.py file, feel free to modify it.
Relevant Files: everything in the folder codefromtf
Part 3: Writing Tests
In test scratch.py and test tf.py , we have provided basic test-cases. Feel free to write
more. To test the code, run
python test_*.py -v or
nose2
Part 4: Submit
Submitting the code is equivalent to committing the code. This can be done with the follow
command:
svn commit -m "Some meaningful comment here."
Lastly, double check on your browser that you can see your code at
https://subversion.ews.illinois.edu/svn/sp18-cs446/(netid)/mp3/