• Dominican School of Philosophy and Theology

• STAT 240

 IEOR 142: Introduction to Machine Learning and Data Analytics, Spring 2021 Homework Assignment #2 Solution March 2, 2021 Problem 1: (20 points) Consider a binary classification problem where there is a single feature X ∈ R and the dependent variable Y ∈ {0, 1}. Let PX,Y denote the joint distribution over pairs (X, Y ), and let h : R → {0, 1} denote a generic classifier. We define the error rate of h as R(h) := Pr(Y 6= h(X)), where the probability is computed from the joint distribution PX,Y . Suppose that we collect data (x1, y1), . . . ,(xn, yn), which are assumed to be independent and identically distributed from the distribution PX,Y , and that we train a classifier hˆ using this data. Below we consider two precise specifications of the joint distribution PX,Y . In both cases, derive the largest numerical value ε ∗ that you can for which it holds that R(hˆ) ≥ ε ∗ . Carefully explain how you arrived at your specific value of ε ∗ in both cases. a) (10 points) X is restricted to the set {0, 1} (i.e., is categorical) and PX,Y is given by the distribution in Table 1. b) (10 points) The marginal distribution of Y is given by Pr(Y = 0) = 0.4 and Pr(Y = 1) = 0.6. Given that Y = 0, the distribution of X is normal with mean 3 and variance 1. Given that Y = 1, the distribution of X is normal with mean −1 and variance 1. Table 1 Outcome Probability (X = 0, Y = 0) 0.15 (X = 0, Y = 1) 0.25 (X = 1, Y = 0) 0.4 (X = 1, Y = 1) 0.2