Promise and Perils of Predictive Analytics
1 like558 views
The document discusses challenges in talent acquisition, such as ambiguity, complexity, and friction in recruitment processes, while highlighting the promise of predictive analytics for improving hiring outcomes. It contrasts descriptive and predictive analytics, emphasizing the need for careful consideration of data biases and algorithm transparency. Lastly, it stresses the importance of human judgment alongside analytics and calls for responsible practices in using predictive tools in recruitment.
![© 2016, Future of Talent Institute
What are the criteria?
What is the context?
When and to who was the comment made?
Sentiment Analysis
19
“Even in the best of circumstances,
[sentiment analysis] is only 65% to 70%
accurate.”
-Susan Etlinger, analyst Altimeter Group](https://image.slidesharecdn.com/promiseperilsofpredictiveanalytics-161130180045/85/Promise-and-Perils-of-Predictive-Analytics-19-320.jpg)
Promise and Perils of Predictive Analytics
- 1. © 2016, Future of Talent Institute Kevin Wheeler Talent Acquisition Tech Conference Austin | November 15-16, 2016 The Promise & Perils of Predictive Analytics 1
- 2. © 2016, Future of Talent Institute Challenge 1: Ambiguity Recruiters are asked to find people with skills we’re never heard of, in unrealistic timeframes, in remote places.
- 3. © 2016, Future of Talent Institute Challenge 2: Complexity We need to know how to find the signal in the noise. Desperatly these days!
- 4. © 2016, Future of Talent Institute Challenge 3: Friction Our processes do not flow. Required information is often not easily accessible. There are choke points and ambiguities everywhere.
- 5. © 2016, Future of Talent Institute Challenge 4: Interaction Communication is unclear. We need to know what to say. What is effective?
- 6. © 2016, Future of Talent Institute Challenge 5: Innovation 6 Old thinking and bureaucratic processes stifle innovation and change.
- 7. © 2016, Future of Talent Institute The Promise 7 MORE EFFECTIVE COMMUNICATION FASTER PROCESSES GREATER CANDIDATE UNDERSTANDING HIGHER QUALITY HIRES INCREASED INTERACTION ENAGING CANDIDATE SERVICE PEOPLE ANALYTICS
- 8. © 2016, Future of Talent Institute Predictive Analytics Promises Answers 8 • Who are our top performers? • When & how should we connect with them? • What attracts them to our firm? • Which assessment is more accurate? • Which hires will be the most productive? • What would increase our quality of hire? • Which interview questions are most effective? • What will our turnover rate be in the next quarter?
- 9. © 2016, Future of Talent Institute Descriptive & Predictive Analytics Compared 9 Descriptive Analytics Predictive Analytics Purpose Understand the Past Observe Trends Discuss Gain Insights Make Decisions Take Action Timeframe Past and Current Future Metrics Type Lagging Leading Data Used Raw/Tabulated Information Data Type Structured Structured and Unstructured Benefits Understanding Efficiency Information & Insights Effectiveness
- 10. © 2016, Future of Talent Institute Predictive analytics uses algorithms, machine learning, statistical analysis, sentiment analysis, semantic analysis, and other complex methods to provide insight. But there are challenges and many things to lead you astray. . . 10
- 11. © 2016, Future of Talent Institute Disruptive Technologies 11 Internet Social Mobile Cloud Big Data - Analytics Technology Foundation Trends & Innovations Internet of Things Robotics Disruptive Scenarios Passive candidate assessment Algorithms Automate Recruiting Intelligent Personal Agent Ultrasonic Tracking Predictive Analytics DNA Analysis/Assessment Virtual/Augmented Reality Chatbots Biometric Assessment Life Span Blockchain Supply Chains Contingent Workers Climate Change Decline of Nation State Urbanization Emerging Economies
- 12. © 2016, Future of Talent Institute Data Does Not Tell a Story 12 Data by itself takes no position and holds no bias. Biases & other issues only occur when we interpret it, look for predictions, use it to make decisions,.
- 13. © 2016, Future of Talent Institute The Many Perils, Traps and Biases 13 Assumptions Predictions based on proxies Not questioning The future = the past Privacy Black boxes Using data without candidates knowledge Lack of guidelines Basics No clear problem statement Predicting what has no impact Structural Lack of clean data Sample size too small Too simplistic/too complex models Biases Unintentional/Inherent Favoring one set of data over another
- 14. © 2016, Future of Talent Institute What do you want to predict? What is the problem you want to solve? Do you have the right data? Do we have enough data? Do you have enough relevant data? How do we prevent diversity issues? What is a quality hire? How do we define effective? What’s in the algorithm? Weapons of Conformity & Discrimination? 14 Only 14% of organizations have data to prove the positive business impact of their assessment strategy. -Aberdeen 2014
- 15. © 2016, Future of Talent Institute Assumptions What are we assuming as we search? How valid are our search criteria? How do you know? Scoring algorithms What is in the algorithm? How do we know it is actually measuring what we think it is? Is it discriminating? Is it fostering clones? How do we introduce diverse thinking? Sourcing 15
- 16. © 2016, Future of Talent Institute Is a Facebook/LinkedIn profile accurate? Can they predict ability, skill, or job performance? Using Facebook or LinkedIn 16 “. . .researchers hired HR types to rate hundreds of college students’ Facebook pages according to how employable they seemed. . . .the over 500 guinea pigs, just 56 of the employers responded. So the sample is small, but the researchers found a strong correlation between those employers’ reviews and the employability predictions they had made based on folks’ profile pages.”
- 17. © 2016, Future of Talent Institute Check out a candidate’s social media profiles – even informally Use any of the social media info to influence your opinion of a candidate Do not let a candidate know if you looked Do not let them know what you looked for Social Media & Privacy 17 Potential legal problems if you. . .
- 18. © 2016, Future of Talent Institute What are the criteria? Does personality testing correlate with performance? Do we want everyone the same? Assessment 18 The personality test is a black box, and it’s not clear what it is actually assessing, and whether using it constitutes discriminatory hiring practices.
- 19. © 2016, Future of Talent Institute What are the criteria? What is the context? When and to who was the comment made? Sentiment Analysis 19 “Even in the best of circumstances, [sentiment analysis] is only 65% to 70% accurate.” -Susan Etlinger, analyst Altimeter Group
- 20. © 2016, Future of Talent Institute Matching algorithms Accuracy? Relevance? Privacy? Discrimination? Passive Assessment Privacy? Relevance? Accuracy? Chatbots Assumptions? Discrimination? Online Assessments/games Privacy? Relevance? Correlation=Causation? A Few Emerging Tools 20 How transparent are the vendors about their algorithms and assumptions?
- 21. © 2016, Future of Talent Institute Could We Automate Selection & Assessment? Should We? 21 People are complex, contradictory, and so varied that even complex tools and algorithms are rarely accurate. We will continue to need human judgement and tolerance.
- 22. © 2016, Future of Talent Institute Most of the tools we are using today create as many questions as answers. No single test can predict anything with high certainty. Many tools use proxies, which may not represent reality. We tend to validate a tool when it confirms our suspicions. Sample sizes are often way too tiny to be valid. There is inherent bias in almost all screening. People are highly complex and there are no simple ways to say one person is better than another. Unfortunately. . . 22
- 23. © 2016, Future of Talent Institute PA can provide insight and validate or disprove assumptions. Can augment human judgement. Valuable when used responsibly following openness guidelines. Can provide early warning that employees are unhappy or are thinking about leaving. Can identify competencies and skills and predict their value to a particular role. Predictive Analytics 23
- 24. © 2016, Future of Talent Institute Have a privacy disclosure policy that is shared with candidates and is on your career site. Let candidates know why you have rejected them, especially if based on social media information. Have clear definitions of what you are looking for and how you will know when you find it. Train recruiters in what is acceptable, reliable, and accurate. Always use human judgement along with any AI or other assessment or prediction. Good Predictive Analytics Practices 24
- 25. © 2016, Future of Talent Institute . . .use more than one test or predictive tool. . . .when choosing a tool, know what is in the algorithm. Know what is being looked for, tested, analyzed, and how each factor is weighted. . . .make sure proxies are valid and really predictive. . . .not adopt tools hastily and without careful thought and knowledge. . . .always question our own assumptions and beliefs. We should. . . 25
- 26. © 2016, Future of Talent Institute 26 THANKS YOUR THOUGHTS & QUESTIONS Follow me on Twitter: @kwheeler Email: kwheeler@futureoftalent.org