Identify Rules that Predict Patient’s Heart Disease - An Application of Decision Tree Modelling in Orange
- 1. Decision Tree Modelling With Orange Identify Rules that Predict Patient’s Heart Disease Author: Anthony Mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com
- 2. Characteristics of Orange Visual programming makes data mining accessible to a broader audience Provides comprehensive data preprocessing tools A vast collection of machine learning algorithms is available Excels in interactive data visualisation Scalable, and integrates with external software packages An open-source project with a vibrant community
- 3. Project’s Context, Objective & Strategies Make Insight-informed Decisions Clinic collected data on heart disease diagnosis and other patient information, and wants to use the data to make insight- informed decisions Predict Patient’s Well-being To identify the rules that will predict whether a patient will have heart disease in the future, based on the data collected on him/her Deploy Decision Tree Model Create a Decision Tree Model, with rules, to predict whether a patient will have a heart disease in the future based on collected data To train and evaluate the model Boost the model’s performance Conduct predictions
- 4. Exploratory Data Analysis (EDA) Findings Target = Heart Disease This is a categorical variable, which has a limited number of possible values; making it easier to predict than a continuous variable, like blood pressure or cholesterol level Feature Columns = 9 Row Instances = 918 Blanks & Outliers = None
- 5. Decision Tree Workflow in Orange
- 6. Loading File, Selecting Columns & Splitting Data Loading File Medical.csv file was loaded into workflow with ‘Gender’, ‘FastingBS’ & ‘Exercise’ classified as ‘categorical’ data & given ‘feature’ role, and ‘HeartDisease’ classified as categorical data &given the ‘target’ role in the ‘File’ Widget Selecting Columns In the ‘Select Column’ Widget, all feature columns were posted into the ‘Features’ box. The ‘HeartDisease’, which is the ‘target’ was clicked into the ‘Target’ box in this widget Splitting Data Dataset divided into 70% for training the model while keeping the remaining 30% for testing the model
- 7. Initial Evaluation of Decision Tree Model Evaluation of Model (30%) Classification Accuracy for this model, trained on 30% of the dataset, is 76.4% Tree Depth Limited to 10 For initial assessment of the performance of the Decision Tree Model, in the Tree Widget, the maximal tree depth was limited to 10 Evaluation of Model (70%) Classification Accuracy for this model, trained on 70% of the dataset, is 97.1% Findings At the Tree Depth of 10, the model displayed a difference of 15% when fed with training & testing dataset Conclusion Suggests that the Decision Tree Model Has Been Overfitted to the training data Follow-up To tune the hyperparameters of the model to enable it to generalise better to perform well with the testing data
- 8. Tuning the Model to Improve Generalisation Evaluation of Model (30%) Classification Accuracy for this model, trained on 30% of the dataset, is 80.7% Tree Depth Now Limited to 3 To tune the model, the maximal tree depth was adjusted several times. The depth of 3 was chosen as Classification Accuracy scores on training and testing data are high (about 80%) while the difference between scores is negligible (at 1.6%) Evaluation of Model (70%) Classification Accuracy for this model, trained on 70% of the dataset, is 82.3%
- 9. Confusion Table: False Positives/Negatives Tree Depth at 10 Tree Depth at 3 False Negative = 17.8% False Positive = 27.4% Patients may become untreatable when their conditions go untreated (for False Negatives) or may have to pay for unwanted treatments and bare the consequences of unneedful side-effects from the treatment (for False Positives). So, reducing the number of False Negatives and False Positives in the model is beneficial False Negative = 19.1% False Positive = 19.4% While False Negatives have increased by 1.3%, False Positives have dropped by 8% with the overall model’s Classification Accuracy improved by 4.3%
- 10. Rules Predicting Patient’s Heart Disease* Sequence of splitting the criteria suggests that Exercise as the top priority rule with Cholesterol and MaxHR as the two other influencers to likelihood of Heart Disease in patients * More details are found in the project report, which are not released at the request of the Clinic
- 11. Decision Tree Modelling With Orange Identify Rules that Predict Patient’s Heart Disease Author: Anthony Mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com