Towards A Differential Privacy Preserving Utility Machine Learning Classifier
Download as PPTX, PDF0 likes740 views
This paper presents research on achieving differential privacy while preserving utility in machine learning classifiers. The researchers applied differential privacy to a dataset of political donations, then used an AdaBoost ensemble classifier on the original and private datasets. They found that differential privacy maintained statistical properties but introduced classification errors. While increasing weak learners reduced errors on the original data, it did not help with the private data. The study shows the challenges of balancing privacy and utility.
Towards A Differential Privacy Preserving Utility Machine Learning Classifier
- 1. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 Towards A Differential Privacy and Utility Preserving Machine Learning Classifier Kato Mivule, Claude Turner, and Soo-Yeon Ji Computer Science Department Bowie State University Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 1
- 2. Complex Adaptive Systems 2012 – Washington DC USA, Outline November 14-16 Introduction Related work Essential Terms Methodology Results Conclusion 2
- 3. Introduction Entities transact in ‘big data’ containing personal identifiable information (PII). Organizations are bound by federal and state law to ensure data privacy. In the process to achieve privacy, the utility of privatized datasets diminishes. Achieving balance between privacy and utility is an ongoing problem. Therefore, we investigate a differential privacy preserving machine learning classification approach that seeks an acceptable level of utility. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 3
- 4. Related Work There is a growing interest in investigating privacy preserving data mining solutions that provide a balance between data privacy and utility. Kifer and Gehrke (2006) did a broad study of enhanced data utility in privacy preserving data publishing by using statistical approaches. Wong (2007) described how achieving global optimal privacy while maintaining utility is an NP-hard problem. Krause and Horvitz (2010) noted that endeavours of finding trade-offs between privacy and utility is still an NP-hard problem. Muralidhar and Sarathy (2011) showed that differential privacy provides strong privacy guarantees but utility is still a problem due to noise levels. Finding the optimal balance between privacy and utility remains a challenge—even with differential privacy. 4 Complex Adaptive Systems 2012 – Washington DC USA, November 14-16
- 5. Data Utility verses Privacy Data utility is the extent of how useful a published dataset is to the consumer of that publicized dataset. In the course of a data privacy process, original data will lose statistical value despite privacy guarantees. Image Source: Kenneth Corbin/Internet News. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 5
- 6. Objective Achieving an optimal balance between data privacy and utility remains an ongoing challenge. Such optimality is highly desired and remains our investigation goal. Image Source: Wikipedia, on Confidentiality. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 6
- 7. Ensemble classification Is a machine learning process, in which a collection of several independently trained classifiers are merged to achieve better prediction. Examples include single trained decision trees joined to make accurate predictions. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 7
- 8. AdaBoost Ensemble – Adaptive Boosting Proposed by Freund and Schapire (1995), uses several iterations by adding weak learners to create a powerful learner, adjusting weights to center on misclassified data in earlier iterations. Classification Error in AdaBoost Ensemble is computed as below: Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 8
- 9. AdaBoost Ensemble (Cont’d ) AdaBoost Ensemble computes as follows: Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 9
- 10. Differential Privacy Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 10
- 11. Differential Privacy (Cont’d) Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 11
- 12. Methodology (Cont’d) We utilized a public available Barack Obama 2008 campaign donations dataset. The data set contained 17,695 records of original unperturbed data. Two attributes, the donation amount and income status, are utilized to classify data into three classes. The three classes are low income, middle income, and high income, for donations $1 to $49, $50 to $80, $81 and above respectively. Validating our approach, the dataset comprised 50 percent on training and the remainder on testing, on both Original and Privatized datasets. Oracle database is queried via MATLAB ODBC connector. MATLAB is used for differential privacy and machine learning classification. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 12
- 13. Results Essential statistical traits of the original and differential privacy datasets, a necessary requirement to publish privatized datasets, are kept. As depicted, the mean, standard deviation, and variance of the original and differential privacy datasets remained the same. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 13
- 14. Results (Cont’d) There is a strong positive covariance of 1060.8 between the two datasets, which means that they grow simultaneously, as illustrated below: Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 14
- 15. Results (Cont’d) There is almost no correlation (the correlation was 0.0054) between the original and differentially privatized datasets. Indicates some privacy assurances, and difficulty for an attacker, dealing only with the privatized dataset, to correctly infer any alterations. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 15
- 16. Results (Cont’d) After applying differential privacy, AdaBoost ensemble classifier is performed. The outcome of the donors’ dataset was Low, Middle, and High income, for donations 0 to 50, 51 to 80, and 81 to 100, respectively. This same classification outcome is used for the perturbed dataset to investigate whether the classifier would categorize the perturbed dataset correctly. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 16
- 17. Results (Cont’d) The training dataset from the original data showed that the classification error dropped from 0.25 to 0 with increased weak decision tree learners. The results changed with the training dataset on the differentially private data when the classification error dropped from 0.588 to 0.58. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 17
- 18. Results (Cont’d) When the same procedure is applied to the test dataset of the original data the classification error dropped from 0.03 to 0. However, when this procedure perform on the differentially private data, the error rate did not change even with increased number of weak decision tree. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 18
- 19. Conclusion In this study, we found that while differential privacy might guarantee strong confidentiality, providing data utility still remains a challenge. However, this study is instructive in a variety of ways: The level of Laplace noise does affect the classification error. Increasing the number of weak learners is not too significant. Adjusting the Laplace noise parameter, ε, is essential for further study. However, accurate classification means loss of privacy. Tradeoffs must be made between privacy and utility. We plan on investigating optimization approaches for such tradeoffs. Complex Adaptive Systems 2012 – Washington DC USA, November 14-16 19
- 20. Complex Adaptive Systems 2012 – Washington DC USA, Questions? November 14-16 Contact: Kato Mivule: kmivule@gmail.com Thank You. 20