BMAW 2014 - Using Bayesian Networks to Identify and Prevent Split Purchases in Brazil
3 likes3,610 views
The document discusses the use of Bayesian networks for detecting and preventing procurement fraud, specifically focusing on split purchases within Brazilian government contracts. It outlines the methodology for analyzing historical procurement data, highlights the importance of early fraud detection, and presents the project's objectives to enhance transparency and reduce corruption. The authors demonstrate the effectiveness of their model by achieving high accuracy in identifying suspicious transactions.
BMAW 2014 - Using Bayesian Networks to Identify and Prevent Split Purchases in Brazil
- 1. USING BAYESIAN NETWORKS TO IDENTIFY AND PREVENT SPLIT PURCHASES IN BRAZIL
- 2. USING BAYESIAN NETWORKS TO IDENTIFY AND PREVENT SPLIT PURCHASES IN BRAZIL Rommel N. Carvalho, Leonardo J. Sales, Henrique A. da Rocha, and Gilson L. Mendes ! Data Science Team Leader / Professor http://about.me/rommelnc ! Department of Research and Strategic Information (DIE) / Department of Computer Science (CIC) Brazil’s Office of the Comptroller General (CGU) / University of Brasília (UnB) ! BMAW workshop @ UAI 2014 Quebec City, Quebec, Canada - 07/27/2014 2
- 3. Introduction Methodology Data Understanding and Preparation Modeling and Evaluation Deployment Conclusion 3 AGENDA
- 4. INTRODUCTION 4 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 5. CGU is responsible for inspecting and auditing Brazilian Government projects and programs providing transparency and preventing corruption All contracts with the private sector must follow the national procurement law Select the most advantageous proposal for a contract in its interest Susceptible to many forms of corruption 5 INTRODUCTION Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 6. Prevent and detect government corruption For that it must gather information from a variety of sources and combine it to evaluate whether further action, such as an investigation, is required Information explosion Growing Acceleration Program (PAC) alone has a budget greater than 250 billion dollars More than one thousand projects only on the state of Sao Paulo 6 CGU’S RESPONSIBILITY Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 7. The Public Agent may favor a specific supplier that she happens to know The Public Agent may receive, from the bidder, a financial compensation for awarding a contract to that firm Bidders may collude as to set the results of the procurement And many others from within and outside the public administration 7 CORRUPTION Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 8. Law has many articles that established penalties for firms or/and public legislators caught in corruption activities Two types of penalties Administrative actions Penal actions Enforcing the law is difficult Avoid corruption in the first place 8 PREVENTION VS PUNISHMENT Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 9. Apply Data Mining methods to create models that will aid the experts in identifying procurement frauds If possible, identify suspicious transactions as soon as possible in order to prevent the fraud from happening 9 PROJECT OBJECTIVE Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 10. METHODOLOGY 10 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 11. 11 CRoss Industry Standard Process for Data Mining Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion CRISP-DM
- 12. DATA UNDERSTANDING AND PREPARATION 12 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 13. Related to procurements and contracts of IT services on the Brazilian Federal Government from 2005 to 2010 Merged data from different databases 42 attributes 70,365 transactions 13 THE DATA Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 14. CGU has been working closely with experts on the domain and want to identify certain fraud topologies, such as: Identify if owners from different companies are actually partners Identify split purchases, i.e., if big contracts are being broken down in small ones (adding up to more than 8,000) More than 20 more types of frauds to identify Focus on the second one 14 WHAT FOR?* Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion *This is a bit more on business understanding.
- 15. Loaded in Excel Removed some characters , “ ‘ Replaced missing values NA, -9, -8, and 0 by ? 15 PREPARING THE DATA I Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 16. Removed some attributes Mostly IDs E.g., we kept the name of the state instead of its ID Because it is easier to understand what it means From 42 to 26 attributes Saved the data as CSV file and loaded in Weka Changed the year to nominal Removed rows with missing values in final price 16 PREPARING THE DATA II Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 17. Typical cases of data that should not be trusted or considered incorrect Values from cents to hundreds of trillions of dollars Proposed unit price was 4 thousand whereas the final actual price was a dollar Number of cases with a really high value is small Analyzed by experts Remove noise (done for obvious cases) Identify outliers (still being analyzed by experts) 17 ANALYZING THE DATA Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 18. Interested in transactions that sum to 8,000 in a given period of time Created class attribute using R Irregular* - transactions that involved the same institutions on the same month and year that added up to more than 8,000 Regular - all others This classification is easy but... 18 CHANGING THE DATA Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion *This is a simplification, since we have restricted our work to IT purchases.
- 19. MODELING AND EVALUATION 19 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 20. The idea is to try to classify, without looking at the sums, which transactions should be considered suspicious Avoid waiting for the problem to occur Prevention over punishment Identify the problem as soon as possible Find suspicious transactions in days (proactive) Not after a month (reactive) 20 ...THIS IS HARD!!! Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 21. 21 BACK TO PREPARATION Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 22. 22 NAÏVE BAYES VS BAYES NET (K2) - WITH AND WITHOUT RESAMPLING Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 23. 23 CHANGING ALGORITHMS AND NUMBER OF PARENTS Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 24. 24 CAN IT GET BETTER? Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 25. Analyzing the structure of the BN we were able to identify some interesting relations about The government office responsible for the procurement And also the winner of the procurements Due to confidentiality nature of the problem we are not allowed to discuss these relations further I can say that more analysis is needed Still to be done due to time constrain 25 UNDERSTANDING WHY I Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 26. Tried to find if there was something obvious about the data that made it easy to predict Used Apriori with min confidence .1 and min lift 1.1 generating 50 rules Nothing interesting at the first run (confidence 1 and lift 1.82) Removed uninteresting attributes Still nothing interesting (now with highest confidence of .62 and lift of 1.11) 26 UNDERSTANDING WHY II Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 27. DEPLOYMENT 27 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 28. Identifying suspicious transactions as soon as possible to avoid the occurrence of fraud What kind of fraud? The type of fraud we are concerned about in this project is split purchases. I.e., when a procurement that should be done just once with a value higher than 8,000 is broken down in smaller ones to avoid the normal procurement procedure, allowing the office to hire an enterprise directly 28 BACK TO THE GOAL Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 29. Detect one or more of these procurements, but that still do not sum up to more than 8,000, as soon as they become suspicious (using the classifier) and act proactively Faster response and prevent irregularities in the first place Warn/educate/teach the people involved that they should be careful, since breaking down big procurements in small ones is not allowed 29 THE BIG IDEA Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 30. Make those that do it on purpose to think twice Educate those that just don’t know better Expect a decrease in the number of irregular transactions associated to this type of fraud Important to keep track of statistics to verify this expectation As people learn and change their behavior, it is also expected that our classifier will also need some changes to cope with these modifications 30 THE CONSEQUENCES Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 31. CONCLUSION 31 Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 32. Data Mining can be used to identify and prevent procurement frauds in Brazil Prioritize depth instead of breadth Results paid off Correctly classified all split purchases Really high ROC area (.999) High accuracy (99.197%) 32 CONCLUSIONS Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 33. Deploy this classifier in CGU’s intelligence system and analyze its impact Better understand why the model is capable of getting such good results By carefully analyzing the structure of the BNs generated and the variable’s CPTs Be careful with overfitting Still a lot of different types of frauds to be analyzed 33 FUTURE WORK Introduction - Methodology - Data Understanding and Preparation - Modeling and Evaluation - Deployment - Conclusion
- 34. 34 OBRIGADO!