Data mining assignment 3
- 1. Data mining Assignment week 3 BARRY KOLLEE 10349863
- 2. Regression | CPU performance Exercise 1: Decision Trees and Logical Forms Imagine a scenario where you want to decide whether to provide a loan. Given the following logical formula in disjunctive normal form, draw the corresponding decision tree. ( age > 25 = no ^ lives_by_himself = no ) ( age > 25 = yes ^ employed = yes ) ( age > 25 = yes ^ employed = no ^ in_education = yes ) 2
- 3. Regression | CPU performance Exercise 2: Information Gain and Attribute Selection Given the following training data: Which attribute (i.e., a1 or a2) has the higher information gain when chosen as the first branching in a decision tree? Explain this first intuitively (you don't need a calculator for this) and then explain it by giving the respective information gains (you can use a calculator for this). Observation: I think that a1 has a higher information gain. That’s because I see an equal distribution at a2. I conclude this by the following: • 50 % of all the instances has a ‘+’ class and the other 50 % has a ‘-‘ class. • 50 % of all the true values have a ‘+’ class and 50 % of all true values have a ‘-‘ class. • 50 % of all false values have a ‘+’ class and 50 % of all false values have a ‘-‘ class. The information gain for class a1 becomes: H(a1, true) = -(1/3)log2(1/3) – (2/3)log2(2/3) = 0.9183 H(a1, false) = -(1/3)log2(1/3) – (2/3)log2(2/3) = 0.9183 H(a1) = 0.5 * 0.9183 + 0.5 * 0.9183 = 0.9183 So eventually the gain of ‘a1’ will be: Gain(a1) = 1 – 0.9183 = 0.0817 The information gain for class a2 becomes: H(a2, true) = -(1/2)log2(1/2) – (1/2)log2(1/2) = 1 H(a2, false = -(1/2)log2(1/2) – (1/2)log2(1/2) = 1 H(a2) = 0.5 * 1 + 0.5 * 1 = 1 So eventually the gain of ‘a2’ will be: Gain(a2) = 1 – 1 = 0 3
- 4. Regression | CPU performance Exercise 3: Overfitting Give a simple example of a decision tree and a data set where the decision tree overfits the data. Show explicitly (see the definition of overfitting) that the decision tree in your example overfits. My example for showing overfitted data is in the column and decision tree below. Each and every animal has a unique number (animalnumber). If this training dataset contains a lot of animals we can easily search for the value ‘is_in_cage’ for every animal and eventually know if this animal is within a cage. However if the zoo retrieves new animals we can’t state/predict if this animal should be in a cage. We shouldn’t assume this. Example of a (training) dataset of the zoo: animalnumber number_of_visits is_in_cage animal_1 200 No animal_2 400 Yes animal_3 100 No animal_n .. .. Example it’s decision tree: ( number_of_visits > 140 = yes is_in_cage = yes ) 4