CSCE 421: Homework 4

Problem 1: Implement Cross Entropy Cost




Use 2d classification data entropy.csv file provided inHome work 4 on Canvas.







For this question, you will be implementing Cross Entropy Loss and model training based using the implemented loss function.




[Code] Fill in the function read classification data, which takes in the filename as string, and returns the data as 2 numpy arrays. Each with shape (number of rows in dataframe, 1) in the following order first row, second row.



[Code] Implement the sigmoid function and return the sigmoid of the given array s as a numpy number.



[Code] Fill in the function cost function that returns the loss value(type: float) given the weight, bias, input, and target.



[Code] Fill in the function cross entropy optimizer to calculate the cross-entropy loss in each iteration and update weight and bias terms. Then the updated weight and updated bias terms as well as the costs saved in a list in the following order: updated weight, updated bias, costs.















Problem 2: Multi-class perceptron




[Written] Complete question 7.4 from the textbook (Machine Learning Refined). (See Figure 1.)

   



























































Figure 1: multi-class and two-class Perceptron cost




Problem 3: Complete ML Pipeline




Consider the binary classification that consists of distinguishing class 6 from the rest of the data points. For this question, you will be using SVMs combined with polynomial kernels to solve this classification problem. Use setimage train and test .csv files provided in Homework 4 on Canvas.




Problem 3-a: Data preprocessing




In this section, you will be preparing the dataset for the SVM classifier.




[Code] Fill in the function read sat image data, which takes in the filename as a string, and returns the data as a pandas dataframe.



[Code] Fill in the function remove nan, Remove nan values from the dataframe and return it.



[Code] Fill in the function normalize data to normalize the dataframe and return the normalized dataframe as the output. (You can use StandardScaler from sklearn for Z score normalization)



[Code] Fill in the function labels to binary to convert the labels into binary format. Change labels 1, 2, 3, 4, 5 to 0 and label 6 to 1. Return the updated dataframe.






Problem 3-b: Hyperparameter Tuning




In this section, you will be tuning the hyperparameters of the SVM classifier. Use SVMs combined with polynomial kernels to solve this classification problem. For each value of the polynomial degree, d = 1, 2, 3, 4, plot the average 5-fold cross-validation error plus or minus one standard deviation as a function of C (let the other parameters of the polynomial kernels be equal to their default values) ON THE TRAINING DATA. Report the best value of the trade-off constant C measured on the training internal cross-validation.




[Code] Fill in the function cross validate c vals, which takes features dataframe (X), label dataframe



as well as the number of folds and SVM hyperparameters (c vals and d vals). The function returns two numpy arrays for average and standard deviation of error values (order: (ERRAVGdc, ERRSTDdc))



 [Written + Code] Fill in the function plot cross val err vs c, to generate the graphs as described in the question. Please include the plots to the report document.






Problem 3-c: (Model Training and Testing)




Let (C*; d*) be the best pair found previously in the 5-fold internal cross-validation. Build a model for each pair on the full training data using the best c. Then test the model on the TEST DATA and plot the test errors for each model, as a function of d.




[Code] Fill in the function evaluate c d pairs, which takes train features dataframe (X train), train label dataframe (y train), test features dataframe (X test), test label dataframe (y test) as well as the number of folds and SVM hyperparameters (c vals and d vals). The function returns four numpy arrays for Average MSE values, Average Support Vectors array, Average Number of Support Vectors that Violate the Margin array and Average Value of Hyperplane Margins array SuppVect (order: (ERRAVGdcTEST, SuppVect, vmd, MarginT))



[Written + Code] Fill in the function plot test errors to plot the error for each model as a function of d.






Problem 3-d: (Results Evaluation)




[Written + Code] Fill in the function plot avg support vec to plot the average number of support vectors as a function of d. Add the plot to the report document.



[Written + Code] Fill in the function plot avg violating support vec to plot the average number of violating support vectors as a function of d. How many of the support vectors lie on the margin hyperplanes? Please provide your reasoning in the report document.






Problem 3-e: (Conceptual)




[Written + Code] Fill in the function plot avg hyperplane margins Explain how the parameter d influences the model fit (margin size and number of support vectors).



[Written] Assume you were using an RBF kernel instead of a polynomial kernel, what would the parameter influence in terms of the model fit (margin size and number of support vectors. Explain your reasoning in the report document.






Note: function definitions and comments for each function provide a description of the problems the functions are supposed to address.

























Problem 1: Implement Cross Entropy Cost




Use 2d classification data entropy.csv file provided inHome work 4 on Canvas.







For this question, you will be implementing Cross Entropy Loss and model training based using the implemented loss function.




[Code] Fill in the function read classification data, which takes in the filename as string, and returns the data as 2 numpy arrays. Each with shape (number of rows in dataframe, 1) in the following order first row, second row.



[Code] Implement the sigmoid function and return the sigmoid of the given array s as a numpy number.



[Code] Fill in the function cost function that returns the loss value(type: float) given the weight, bias, input, and target.



[Code] Fill in the function cross entropy optimizer to calculate the cross-entropy loss in each iteration and update weight and bias terms. Then the updated weight and updated bias terms as well as the costs saved in a list in the following order: updated weight, updated bias, costs.















Problem 2: Multi-class perceptron




[Written] Complete question 7.4 from the textbook (Machine Learning Refined). (See Figure 1.)

   



























































Figure 1: multi-class and two-class Perceptron cost




Problem 3: Complete ML Pipeline




Consider the binary classification that consists of distinguishing class 6 from the rest of the data points. For this question, you will be using SVMs combined with polynomial kernels to solve this classification problem. Use setimage train and test .csv files provided in Homework 4 on Canvas.




Problem 3-a: Data preprocessing




In this section, you will be preparing the dataset for the SVM classifier.




[Code] Fill in the function read sat image data, which takes in the filename as a string, and returns the data as a pandas dataframe.



[Code] Fill in the function remove nan, Remove nan values from the dataframe and return it.



[Code] Fill in the function normalize data to normalize the dataframe and return the normalized dataframe as the output. (You can use StandardScaler from sklearn for Z score normalization)



[Code] Fill in the function labels to binary to convert the labels into binary format. Change labels 1, 2, 3, 4, 5 to 0 and label 6 to 1. Return the updated dataframe.






Problem 3-b: Hyperparameter Tuning




In this section, you will be tuning the hyperparameters of the SVM classifier. Use SVMs combined with polynomial kernels to solve this classification problem. For each value of the polynomial degree, d = 1, 2, 3, 4, plot the average 5-fold cross-validation error plus or minus one standard deviation as a function of C (let the other parameters of the polynomial kernels be equal to their default values) ON THE TRAINING DATA. Report the best value of the trade-off constant C measured on the training internal cross-validation.




[Code] Fill in the function cross validate c vals, which takes features dataframe (X), label dataframe



as well as the number of folds and SVM hyperparameters (c vals and d vals). The function returns two numpy arrays for average and standard deviation of error values (order: (ERRAVGdc, ERRSTDdc))



 [Written + Code] Fill in the function plot cross val err vs c, to generate the graphs as described in the question. Please include the plots to the report document.






Problem 3-c: (Model Training and Testing)




Let (C*; d*) be the best pair found previously in the 5-fold internal cross-validation. Build a model for each pair on the full training data using the best c. Then test the model on the TEST DATA and plot the test errors for each model, as a function of d.




[Code] Fill in the function evaluate c d pairs, which takes train features dataframe (X train), train label dataframe (y train), test features dataframe (X test), test label dataframe (y test) as well as the number of folds and SVM hyperparameters (c vals and d vals). The function returns four numpy arrays for Average MSE values, Average Support Vectors array, Average Number of Support Vectors that Violate the Margin array and Average Value of Hyperplane Margins array SuppVect (order: (ERRAVGdcTEST, SuppVect, vmd, MarginT))



[Written + Code] Fill in the function plot test errors to plot the error for each model as a function of d.






Problem 3-d: (Results Evaluation)




[Written + Code] Fill in the function plot avg support vec to plot the average number of support vectors as a function of d. Add the plot to the report document.



[Written + Code] Fill in the function plot avg violating support vec to plot the average number of violating support vectors as a function of d. How many of the support vectors lie on the margin hyperplanes? Please provide your reasoning in the report document.






Problem 3-e: (Conceptual)




[Written + Code] Fill in the function plot avg hyperplane margins Explain how the parameter d influences the model fit (margin size and number of support vectors).



[Written] Assume you were using an RBF kernel instead of a polynomial kernel, what would the parameter influence in terms of the model fit (margin size and number of support vectors. Explain your reasoning in the report document.






Note: function definitions and comments for each function provide a description of the problems the functions are supposed to address.




















































3
























3