Interaction Mining: the new frontier of Call Center Analytics
3 likes1,083 views
This document discusses using interaction mining to analyze call center data. It outlines how interaction mining can identify customer-oriented behaviors, root causes of problems, problematic customers, and best practices for dealing with customers. The document describes applying automatic argumentative analysis to call center conversations to label statements with discourse relations and argumentative labels. It evaluates this approach on meeting conversations, achieving 81% precision and 98% recall. Experiments applying these techniques to a corpus of 213 manually transcribed call center conversations are also discussed.
![Iden(fy
Root
Cause
of
Problems
• Coopera(veness
score
Argumenta=ve
Categories
Coopera=veness
Accept
explana(on
5
– a
measure
obtained
by
averaging
the
score
Suggest
4
obtained
by
mapping
Propose
3
argumenta(ve
labels
of
Provide
opinion
2
each
turn
in
the
Provide
explana(on/jus(fica(on
1
conversa(on
into
a
[-‐5
Request
explana(on/jus(fica(on
0
+5]
scale.
Ques(on
-‐1
Raise
issue
-‐2
• Sen(ment
Analysis
Provide
nega(ve
opinion
-‐3
module.
Disagree
-‐4
Reject
explana(on
or
jus(fica(on
-‐5
©
2011
interAnaly(cs
17](https://image.slidesharecdn.com/dart11-120419131252-phpapp02/85/Interaction-Mining-the-new-frontier-of-Call-Center-Analytics-17-320.jpg)
Interaction Mining: the new frontier of Call Center Analytics
- 1. Interac(on Mining: the new fron(er of Call Center Analy(cs Vincenzo Pallo:a Rodolfo Delmonte Lammert Vrieling David Walker © 2011 interAnaly(cs 1
- 2. Outline • Call Center Analy(cs • Automa(c Argumenta(ve Analysis for Interac(on Mining • Experiments with Call Center Data • Conclusions © 2011 interAnaly(cs 2
- 3. CALL CENTER ANALYTICS © 2011 interAnaly(cs 3
- 4. Call Center Analy(cs • Call centers data represent a valuable asset for companies, but it is oOen underexploited for business purposes because: – it is highly dependent on quality of speech recogni(on technology – it is mostly based on text-‐based content analysis. • Interac(on Mining as a viable alterna(ve: – more robust – tailored for the conversa(onal domain – slanted towards pragma&c and discourse analysis © 2011 interAnaly(cs 4
- 5. Mainstream Call Center Analy(cs re al v eil er ot un stom oe s n ut cu D abo n hts sfac=o sig a= in s © 2011 interAnaly(cs 5
- 6. Call Center Analy(cs: metrics and KPIs • Agent Performance Sta(s(cs: – Average Speed of Answer, Average Hold Time, Call Abandonment Rate, A<ained Service Level, and Average Talk Time. – Quan(ta(ve measurements that can be obtained directly through ACD (Automa(c Call Distribu(on), Switch Output and Network Usage Data. • Peripheral Performance Data: – Cost Per Call, First-‐Call Resolu&on Rate, Customer Sa,sfac,on, Account Reten&on, Staff Turnover, Actual vs. Budgeted Costs, and Employee Loyalty. – Quan(ta(ve, with the excep(on of Customer Sa&sfac&on that is usually obtained through Customer Surveys. • Performance Observa(on: – Call Quality, Accuracy and Efficiency, Adherence to Script, Communica,on E,que;e, and Corporate Image Exemplifica,on. – Qualita=ve metrics based on analysis of recorded calls and session monitoring by a supervisor. © 2011 interAnaly(cs 6
- 7. Four objec(ves 1. Iden(fy Customer Oriented Behaviors, – which are highly correlated to posi(ve customer ra(ngs (Rafaeli et al. 2007); 2. Iden(fy Root Cause of Problems – by looking at controversial topics and how agents are able to deal with them; 3. Iden(fy customers who need par(cular a:en(on – based on history of problema(c interac(ons; 4. Learn best prac(ces in dealing with customers – by iden(fying agents able to carry coopera(ve conversa(ons. © 2011 interAnaly(cs 7
- 8. ARGUMENTATIVE ANALYSIS FOR INTERACTION MINING © 2011 interAnaly(cs 8
- 9. Argumenta(ve Structure of Conversa(ons DISCUSS(issue) <- PROPOSE(alternative) 1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a REJECT(alternative) 1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a PROVIDE(justification) 1714.09 B: because there's no point in doing that if it's John: because there's no point in doing that if it's going to to be better . {s} {s} 61b+ not not goingbe anyany better . 61b+ ACCEPT(justification) 1712.69 David: okay . {s^bk} 62b PROPOSE(alternative) 1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} Why was David’s proposal on microphones rejected? 1727.34 John: yeah . {b} © 2011 interAnaly(cs 9
- 10. Automa(c Argumenta(ve Analysis • Based on the GETARUNS system1. • Clauses in Turns are labelled with Primi(ve Discourse Rela(ons: – statement, narra,on, adverse, result, cause, mo,va,on, explana,on, ques,on, hypothesis, elabora,on, permission, incep,on, circumstance, obliga,on, evalua,on, agreement, contrast, evidence, hypoth, seCng, prohibi,on. • And then Turns are labelled with Argumenta(ve labels: – ACCEPT, REJECT/DISAGREE, PROPOSE/SUGGEST, EXPLAIN/JUSTIFY, REQUEST EXPLANATION/ JUSTIFICATION. 1 Delmonte R., Bistrot A., Pallo:a V.,Deep Linguis(c Processing with GETARUNS for spoken dialogue Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora). © 2011 interAnaly(cs 10
- 11. Evalua(on ICSI corpus of mee(ngs (Janin et al., 2003) Precision: 81.26% Recall: 97.53% Total Correct Incorrect Precision Found Accept 662 16 678 98% Reject 64 18 82 78% Propose 321 74 395 81% Request 180 1 181 99% Explain 580 312 892 65% Total 1826 421 2247 81.26% Delmonte R., Bistrot A., Pallo:a V.,Deep Linguis(c Processing with GETARUNS for spoken dialogue Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora). © 2011 interAnaly(cs 11
- 12. EXPERIMENTS WITH CALL CENTER DATA © 2011 interAnaly(cs 12
- 13. Ra(onale: implement the four objec(ves 1. Iden(fy Customer Oriented Behaviors, 2. Iden(fy Root Cause of Problems 3. Iden(fy customers who need par(cular a:en(on 4. Learn best prac(ces in dealing with customers © 2011 interAnaly(cs 13
- 14. The Data • Corpus of 213 manually transcribed conversa(ons of a help desk call center in the banking domain. • Average of 66 turns per conversa(on. • Average of 1.6 calls per agent. • Collected for a study aimed at iden(fying customer oriented behaviors that could favor sa(sfactory interac(on with customers (Rafaeli et al. 2007). © 2011 interAnaly(cs 14
- 15. Iden(fy Customer Oriented Behaviors • Based on the work of Rafaeli et al. 2006. • Customer Oriented Behaviors – an(cipa(ng customers requests 22,45% – educa(ng the customer 16,91% – offering emo(onal support 21,57% – offering explana(ons / jus(fica(ons 28,57% – personaliza(on of informa(on 10,50% © 2011 interAnaly(cs 15
- 16. Significant correla(on with argumenta(ve labels © 2011 interAnaly(cs 16
- 17. Iden(fy Root Cause of Problems • Coopera(veness score Argumenta=ve Categories Coopera=veness Accept explana(on 5 – a measure obtained by averaging the score Suggest 4 obtained by mapping Propose 3 argumenta(ve labels of Provide opinion 2 each turn in the Provide explana(on/jus(fica(on 1 conversa(on into a [-‐5 Request explana(on/jus(fica(on 0 +5] scale. Ques(on -‐1 Raise issue -‐2 • Sen(ment Analysis Provide nega(ve opinion -‐3 module. Disagree -‐4 Reject explana(on or jus(fica(on -‐5 © 2011 interAnaly(cs 17
- 18. Top 20 Controversial Topics with average coopera(veness scores and sen(ment © 2011 interAnaly(cs 18
- 19. Coopera(veness of speakers on top discussed topics © 2011 interAnaly(cs 19
- 20. Iden(fy problema(c customers © 2011 interAnaly(cs 20
- 21. Select a specific customer © 2011 interAnaly(cs 21
- 22. Visualize a selected call © 2011 interAnaly(cs 22
- 23. CONCLUSIONS © 2011 interAnaly(cs 23
- 24. Conclusions • New Genera(on Call Center Analy(cs requires Interac(on Mining – Call Center Qualita(ve metrics and KPIs can be only implemented with a full understanding of the customer interac(on dynamics • Argumenta(on is pervasive in conversa(ons. – In order to recognize argumenta(ve acts, advanced Natural Language Understanding is necessary. • Future work: – Scalability: need to process millions of call per day! – Mul(-‐language: call centers all over the world. © 2011 interAnaly(cs 24
- 25. The Team Dr. Lammert Vrieling (1968) - Chief Executive Officer ‣ 15 years in both profit and not-for-profit organizations as consultant, trainer/coach and as executive. ‣ Experience in the steel and aluminium industry, multimedia publishing and newspaper, financial services and in the not-for-profit sector. David E. Walker (1964) - Chief Operating Officer ‣ 25+ years in IT as Software engineer, developer, project manager, and architect. ‣ Senior Software Solutions Architect with extensive experience in designing, developing and delivering enterprise solutions for payment processing, human resource, healthcare, marketing, manufacturing and scientific research environments. Prof. Dr. Rodolfo Delmonte (1946) - Chief Science Officer ‣ Since 1993 head of Computational Linguistics Laboratory of the University of Venice, Italy ‣ 30+ years experience in computational linguistics ‣ From 1986 to 1992 he worked with the Department of Engineering of the University of Parma. From 1978 to 1986 worked with the Department of Dr. Vincenzo Pallotta (1966) - Chief Technology Officer ‣ 30 years in ICT ‣ 10 years in R&D ‣ Human-Language Technology, Digital Libraries, Artificial Intelligence, Ubiquitous Computing, Human-Computer Interaction, Usability Engineering, Information Retrieval, Web Search Engines, Semantic Web, Computational Logics, Training and Education, e-learning. 23 © 2011 interAnaly(cs 25
- 26. …find us at www.interanaly=cs.ch