Solve, algebraically, for the Bayes minimum error classifier; i.e., give the resulting decision rule algebraically, in simplest form.
Plot (by hand or by computer) the decision boundary and label the decision regions in 2D nonaugmented feature space.
(b) Repeat part (a) with the same numbers except ( ") = 0.1, ( #) = 0.9.
2. This problem can be done by hand. If prefer you may use a computer to help with the plots, as long as you know how to plot them by hand.
In this problem, use Kernel Density Estimation (KDE) with binary kernel (window) functions given (in unnormalized form) by:
Φ )
u
u
< 1:F
h
- = C8−1 ≤ h
in which G[ ]K denotes indicator function of X.
You are given the following prototypes for a 2-class problem in 1D feature space:
": 0, 0.4, 0.9, 1.0, 6.0, 8.0
#: 2.0, 4.0, 4.5, 5.0, 5.8, 6.7, 7.0
For parts (a)-(c) below, choose ℎ = 0.5 so that the windows have width of 1.
(a) Graph the KDE estimates of the density functions p ( x S1 ) and p ( x S2 ) . Be sure to label pertinent values on both axes.
(b) Estimate the prior probabilities based on frequency of occurrence of the prototypes in each class.
(c) Use the estimates you have developed in (a)-(b) above to find the decision boundaries and regions for a Bayes minimum-error classifier based on KDE. Only the part of feature space where at least one density is nonzero need be classified.
(d) Repeat (a)-(c) for h = 1.
(e) Repeat (a)-(c) for h = 2.
p. 1 of 2
3. This problem can be solved by hand (using a calculator). However, if you prefer, you may use a computer for all or part of your solution, as long as you know how to solve it by hand.
For the following data in 1D feature space (listed as (x!, !), = 1, ):
(-.9, .81), (-.7, .49), (-.5, .25), (-.3, .09), (-.1, .01), (.1, .01), (.3, .09), (.5, .25), (.7,
.49), (.9, .81):
(a) Use an (unweighted) kNN regressor with = 4 to estimate T( ) at the following points:
= 0, = .4
(b) You are now given that the true target function is in fact ( ) = #. Calculate the MSE on the “test set” predictions of part (a). (Use ( ) to compute its correct outputs.)
(c) Repeat parts (a)-(b) except using kernel regression with = 4. For the kernel function, use:
k"x − x$& = 1 − d"dx, x$&
%&'
in which ()* = distance to the 5th nearest neighbor, and " , +& denotes Euclidean distance.
p. 2 of 2
