Assurance Scoring: using machine learning and analytics to reduce risk in the public sector

Assurance Scoring:
Using Machine Learning
and Analytics to Reduce
Risk in the Public Sector
Matt Thomson
17/11/2016

2Copyright © Capgemini 2012. All Rights Reserved
Presentation Title | Date
Outline
Introduction
Traditional Fraud Detection
Assurance Scoring
Machine Learning
Business Rules
Anomaly Detection
Graph Links

Who am I?
Matt Thomson
Senior Data Scientist at Capgemini
PhD in Astrophysics (http://arxiv.org/abs/1010.3315)
Several years experience in fraud detection
Capgemini
Big Data Analytics team
~100 Data Scientists, Big Data Engineers and Data Analysts
Focus on Open Source and Big Data technologies to solve client
problems
Sponsor the meetup today!

Introduction to the Problem
Public sector constantly working in an environment of reduced resources
Want to provide a better service but with greater efficiency
Therefore very important that limited resources are focussed correctly
Assurance Scoring
 Use ML and other analytical methods to identify the least risky people or applications so
that investigators resources can be targeted on the most risky

Hypothetical Example – 2016 Olympics tickets
Imagine running the application process for selling tickets to the 2016
Olympics
Avoid selling tickets to touts/resellers
 Vast majority of people applying for tickets are genuine
 Fraud detection with big class imbalance problem (<0.1%)
 Avoid approach of investigating each person applying
Lets say we know from 2012 Olympics which people ended up reselling
their tickets – training data
Use ML to identify the 30% (say) least likely to be touts – fast tracked
Investigators focus on the high risk

Traditional Fraud Detection
Identify
Historical
Training Data
Feature
Engineering
Model
Training and
Evaluation
Model
Execution
Feedback

Assurance Scoring
Focus on low-risk
Allows resources to be better focussed
Not limited to Machine Learning
Built using Python!
 Pandas, Scikit-learn etc
 Scala version using Spark MLlib

Assurance Scoring

POLE ‘Analytical’ Data Layer
Disparate data sources - Atomic Layer
Atomic data is
Transformed and
Loaded into POLE
POLE Layer
EventLocationObjectPerson

POLE ‘Analytical’ Data Layer
POLE contains ALL entities from the Atomic Layer, plus their inter-linkages

Assurance Scoring

Machine learning
Transform Selection Model
Training
Validation
Test
Feature extraction and selection Model Building
Variety of output files: logs, graphics, pickle models, etc
Testing: Unit tests, monitoring tests and integration tests
Vector Build
Input Data
Manipulate, Explore
Data
Framework: Structure, flexibility, consistency

Machine learning : Feature Engineering
SQL, Python
Transform
Explore
Select
Ask
questions,
validate
Refine
features
• Feature Extraction
• Data exploration
• Feature selection
Historical Data

Machine Learning: Model Building
Training
Validation
Test
Split Datasets
Build
Models
Hyper-
parameter
tuning
Selected
features Models
Training
results
Validation
results
Tests
results
Compare
Models

Low risk? High risk? Depends on classifier’s
threshold
• True-positives : applications the
model correctly classifies as high
risk
• True negatives: applications model
correctly classifies as low risk
• False-positives: applications the
model scores as high risk but are
not
• False-negatives: applications the
model scores as low risk but were in
fact high risk

Assurance Scoring

Business Rules
Identifying Fraud often been done using deterministic rules
Look for transactions near a threshold or at the end of the day
Primarily data queries on your feature vector
Olympics example – Anyone applying for more than £10,000 tickets

Assurance Scoring

Anomaly Detection
Use the training data to create a baseline of applications by postcode
(say)
If a particular postcode has a larger than expected number of applications
then those cases pushed into high-risk bucket

Assurance Scoring

Graph Links - Matching
Key part of assurance scoring – bringing data together from disparate
sources
Probability of Match: 80%
Attribute Data Source 1 Data Source 2
Name Matt Thomson Matthew Thosmon
Phone Number 07123 456 789 07123 456 798
Favourite Sport Football Cricket

Assurance Scoring

Further Details
Come and find me!
matt.thomson@capgemini.com / @MattGThomson
Assurance Scoring brochure: http://ow.ly/4nbEUI
Blogs:
• Introduction: https://www.capgemini.com/node/1380596
• Integrating multiple techniques: http://bit.ly/24BmszV
• Machine Learning: http://bit.ly/1QTMGnq
• Many more on other topics

We’re Hiring!
Data Science
https://www.uk.capgemini.com/careers/jobs/data-scientist-0
Big Data Engineer
https://www.uk.capgemini.com/careers/jobs/big-data-engineer
matt.thomson@capgemini.com

The information contained in this presentation is proprietary.
© 2012 Capgemini. All rights reserved.
www.capgemini.com
About Capgemini
With more than 120,000 people in 40 countries, Capgemini is one
of the world's foremost providers of consulting, technology and
outsourcing services. The Group reported 2011 global revenues
of EUR 9.7 billion.
Together with its clients, Capgemini creates and delivers
business and technology solutions that fit their needs and drive
the results they want. A deeply multicultural organization,
Capgemini has developed its own way of working, the
Collaborative Business ExperienceTM, and draws on Rightshore ®,
its worldwide delivery model.
Rightshore® is a trademark belonging to Capgemini

Assurance Scoring: using machine learning and analytics to reduce risk in the public sector

More Related Content

Similar to Assurance Scoring: using machine learning and analytics to reduce risk in the public sector (20)

More from South West Data Meetup (11)

Recently uploaded (20)

Assurance Scoring: using machine learning and analytics to reduce risk in the public sector