In this homework, you will implement a discrimination by regression algorithm for multiclass classification using Python. Here are the steps you need to follow:
1. Your discrimination by regression algorithm will be developed using the following modifications to the linear discrimination algorithm with the softmax function we discussed in the lectures.
a. Instead of the softmax function, you are going to use sigmoid functions to generate #!" values. Please note that, in such a case, the summation of #!" values is not guaranteed to be 1. However, you are going to pick the largest value to predict the class label.
b. Instead of the negative log-likelihood (i.e., − ∑&!$% ∑#"$% !" log #!" ), you are going to use the sum squared errors as the error function to minimize (i.e., 0.5 ∑&!$% ∑#"$%( !" − #!")'). Please note that you need to find the correct update euations for this modified model.
2. Generate random data points from three bivariate Gaussian densities with the following parameters:
%
+0.0
%
+3.2
+0.0
%
= 120
'
= 1+2.54 ,
'
= 1+0.0
+1.24 ,
−2.5
+1.2
+0.8
'
= 80
(
= 1−2.04 ,
(
= 1+0.8
+1.24 ,
+2.5
+1.2
−0.8
(
= 100
= 1−2.04 ,
= 1−0.8
+1.24 ,
Your data points should be like the following figure.
3. Learn a discrimination by regression algorithm for this multiclass classification problem using the following learning parameters.
eta = 0.01
epsilon = 0.001
Your parameter estimations should be like the following figures.
print(W)
print(w0)
[[
0.02528161
-2.23729285
2.44057305]
[
4.60790215
-2.46097419
-2.26881565]]
[[-1.13651539
-4.28418412
-3.6081088 ]]
4. Draw the objective function values throughout the iterations. Your figure should be like the following figure.
5. Calculate the confusion matrix for the data points in your training set using the discrimination by regression algorithm you will develop using the estimated parameters from Step 3. Your confusion matrix should be like the following matrix.
y_truth
1
2
3
y_pred
1
117
1
3
2
2
78
0
3
1
1
97
6. Draw your decision boundaries that you will calculate using the discrimination by regression algorithm you will develop using the estimated parameters from Step 3 together with data points and clearly mark misclassified data points. Your figure should be like the following figure.
What to submit: You need to submit your source code in a single file (.py file) and a short report explaining your approach (.doc, .docx, or .pdf file). You will put these two files in a single zip file named as STUDENTID.zip, where STUDENTID should be replaced with your 7-digit student number.
How to submit: Submit the zip file you created to Blackboard. Please follow the exact style mentioned and do not send a zip file named as STUDENTID.zip. Submissions that do not follow these guidelines will not be graded.
Late submission policy: Late submissions will not be graded.
Cheating policy: Very similar submissions will not be graded.
