SlideShare a Scribd company logo
People
Analytics
Introduction
Edith Soghomonyan
Lifelong Learning Consultant
Outline
 What is People Analytics?
 Benefits of People Analytics
 Signals & Trends
 People Analytics examples
 How can HR start building People Analytics
capabilities?
 The future of People Analytics
What is People Analytics?
The use of data and analytic tools to inform
decisions about how to manage people
What is People Analytics?
Analytics, applied to “people issues” – hiring,
performance management, compensation, retention,
etc.
People Analytics_Introduction
People Analytics known as…
Touches all people-related issues in organizations to
make predictions
– Performance evaluation
– Hiring / Assessment
– Retention
– Learning and development
– Team composition
– Etc.
HR ANALYTICS TALENT MANAGEMENT
WORKFORCE ANALYTICS
Why is People Analytics emerging
now?
› Technical progress – HR data availability,
processing power, analytical tools, etc.
› Analytical capabilities – data analysis skills,
plenty of online courses, increasing use of popular
analytical tools in day to day work (e.g. Microsoft
Excel, Power BI, Tableau, etc.)
› Growing recognition of behavioral biases.
› Increase in ROI ($10,66 ROI per 1$ -> Nucleus
Research, 2011)
What does People Analytics require?
› Metrics/predictors for HR - such as attrition
rates, KPIs/performance stats, retention data, etc.
› Software for meaningful data collection that will
help to diagnose the efficiency of current HR
processes and their impact on wellbeing, happiness,
and bottom-line performance.
› Statistical modeling and machine learning methods –
to establish the probability of various scenarios and
make informed decisions that facilitate planning and
growth.
What about the size of data?
The size and scope of data for People Analytics
can depend on several factors:
• These can be large and complex datasets or
simpler measures such as employee surveys.
• The amount of people data used may depend on
the size of the organization and its needs.
People Analytics comes to minimize
human biases and replace intuitive
decision making
Use Employee Data Responsibly
Why does HR need People
Analytics?
Deloitte's 2023 Global Human Capital Trends report
https://www2.deloitte.com/us/en/insights/focus/human-
capital-trends.html#leading-in-a-boundaryless-world
56 % of respondents said
organizations have made moderate
to significant progress in
People Analytics over the past
10 years.
Benefits of People
Analytics
Leading Companies in People Analytics
display seven key characteristics.
People Analytics_Introduction
Signals: This trend applies to you if …
 Your employees feel as if their every move is being
monitored, (increasing stress, job dissatisfaction, and
turnover, and leading to a lack of trust).
 Employees are sharing information (data) more readily
outside of the organization (e.g., LinkedIn) but are
reticent to provide it through organizational channels due
to a perceived lack of benefit to them.
 Your organization faces increased challenges and pressure
from regulators (e.g. GDPR) related to data reporting,
privacy, and protection. Source: Deloitte 2023 Global Human Capital Trends survey
Use Employee Data Responsibly
Barriers When asked to identify top
barriers to realizing value
from people data, 27% of
respondents cited culture,
making it the most common
barrier.
Source: Deloitte 2023 Global Human Capital Trends survey
Benefits When asked to identify the top
benefits from their
organization’s approach to using
people data, the top response
was increased worker engagement
and well-being of the workforce.
Source: Deloitte 2023 Global Human Capital Trends survey
Which performance metrics do Armenian companies collect for
PA?
How do companies make sure they don’t harbor surveillance in the
workplace and there is responsible use of worker data?
Are there legal requirements for processing data?
People Analytics
Examples
• Tracks the performance of
all teachers, comparing it
to evaluations when they
were hired
• Refined the most productive
steps in the hiring process,
where to allocate more
resources, etc.
• Systematically tracked
interview predictions about
new hires to figure out how
good they were at it
• Answer: Not very
• So dramatically reduced the #
of interviewers
Predicting Hire Success
People Analytics
Examples
• Believes a 1% increase in retention can save $75- $
100m/year
• 3-year study: Changing jobs increases employee
“stickiness”
• Increased internal postings of open jobs from <50%
Many, Many Others
• Firms in technology, financial services,
telecommunications, automotive, consumer packaged
goods, energy, not-for-profit…
…are finding:
• Better levers for retaining key employees
• More diagnostic methods for hiring
• Who their most valuable employees are
• How to compose the most productive teams
• Etc.
People
Analytics
Performance
Evaluations
People Analytics In Use
Purpose of performance evaluation
• Feedback
• Rewards / punishment
• Performance evaluation, not talent management.
Tough to compare employees if not in identical
situations.
• Helpful starting place, for this seminar (and often for
life):
Begin by assuming all employees are equal
ability
Benefit
Pitfal
ls
NOISE
› The fundamental challenge in performance evaluation is
that performance measures are noisy (i.e., outcomes
are imperfectly related to employee effort)
› For any given level of effort, a range of outcomes can
occur due to factors outside the employee’s control:
• Competitors, team members, bosses, the economy, etc.
The challenge: Separating skill from luck
A Simple Model
› There are two components to performance:
• In informal terms:Real Tendency + Luck
• In more formal terms: y = x + e,
• x = true ability, and
• e = error, randomly distributed around 0.
 What happens when we sample on extreme performance?
What underlies extreme success and failure?
• Extreme success = f(superior ability, positive
error)
• Extreme failure = f(inferior ability, negative
error)
• Consequences?
Regression to the Mean
A study was recently conducted examining the performance
of the 283 stock mutual funds that existed during the
1990s. The study divided the 1990s into an early period
(1990-1994) and a late period (1995-1999). Below are
the 10 funds that had the highest rate of return in the
early period (with their names disguised), ranked from
1 to 10. Predict their rank for the late 1990s.
Examples
• Officer in the Israeli Air Force— “Punishment is more
effective than praise. Whenever I punish a pilot after
a really poor flight, I see better performance the next
time. Whenever I praise a pilot after an excellent
flight, I see worse performance the next time.”
• Peters and Waterman’s book, In Search of Excellence.
They selected 43 high performing companies in the
early 1980s, and looked to see what practices they
used (some that they discovered were the
organizational equivalent of “brushing teeth”)
People Analytics_Introduction
Regression to the Mean
• Anytime you sample based on extreme values of one attribute, any other
attribute that is not perfectly related will tend to be closer to the mean value
• “Attributes” can be:
• Performance at different points in time
• E.g., last year’s stock returns and this year’s
• Different qualities within the same entity
• E.g., a person’s running speed and language ability
What Gets in the Way of Seeing This?
Among other things:
• Outcome bias
• Hindsight bias
• Narrative seeking
In short, we make sense of the past
• We find a story that connects all the dots
• Chance plays too small a role in these stories
People Analytics_Introduction
Extrapolating From Small Samples
• Principle: Sample means converge to the population mean as the sample
size increases. (This is known as the Central Limit Theorem.) Thus, you
will see more extreme values in small samples.
• When are you more likely to see a .400 season batting average in
baseball – May 1 or Sept. 1?
• In which hospital are you more likely to see a dramatically higher % of
boys than girls (or vice versa) born on any given day – a small
community hospital (e.g., 5 births/day) or a large city hospital (e.g., 100
births/day)?
Extrapolating From Small Samples
Your firm has two plants, one large and one small, which mass produce a
standard computer chip. Other than the amount they produce, the two plants
are identical in all essential regards. Both use the same technology to
produce the same product. When properly functioning, this particular
technology produces one percent (1%) defective items. Whenever the
number of defective items from one day’s production exceeds two percent
(2%), a special note is made in the quality control log to “flag” the problem.
At the end of the quarter, which plant would you expect to have more
“flagged” days in its quality control log? Please mark one.
22% A) The small plant
30% B) The large plant
48% C) The same number on average
“Law of Small Numbers”
• People believe small samples closely share the properties of the
underlying population
• This means they too readily infer the population’s properties (e.g., average)
from the sample’s
• That is: They neglect the role variability (aka chance) inevitably plays in
small samples
The Wisdom of Crowds
• The average of a large number of forecasts
reliably outperforms the average individual
forecast
• Idiosyncratic errors offset each other
• E.g., Galton’s (1906) county fair contest
• Many other examples
The Wisdom of Crowds
• But the value of the crowd critically depends on the independence of
their opinions
• Independent means uncorrelated
• If correlated, the value of additional opinions quickly diminishes
Impact of Correlation
Actual Number of Experts (n)
Equivalent Number of
Independent Experts (n*) n*=
n
1+(n- 1)r
n*®1/ r
as n ® ¥
Clemen and Winkler (1985)
Signal Independence
• People are bad at accounting for this effect
• Even when you tell them exactly what the correlation is, people do
not properly adjust (Enke & Zimmerman, 2015)
Signal Independence
• Sources of correlation between two opinions?
• They’ve discussed it already!
• They talk to the same people
• They have the same background – from the same place, trained the
same way, same historical experiences, etc.
• Need to find ways to keep opinions independent, and add independent
perspectives to experienced groups
Consider Broader Set of Objectives
• Organizations generally care about how a person goes about his/her job
• Most important: impact on others
• People consider too few objectives (Bond, Carlson & Keeney, 2008)
• Systematically omit nearly ½ of the objectives they later identify as
personally relevant
• Leads many firms to rely on too narrow a set of performance measures
Process vs. Outcome
• Famously hard-charging Dell Computers changed their performance
evaluations in the early 2000s
• Before change: 100% results
• After change:
• 50% what an employee accomplished
• 50% how he/she accomplished it, as judged those affected
Process vs. Outcome
• The more uncertainty in the environment, i.e., the less control an
employee has over exact outcomes, the more a firm should emphasize
process in their evaluations.
Process vs. Outcome
• Use analytics to better understand, and focus on, the processes that
tend to produce desired outcomes
• Key issue: Identify the fundamental drivers of value
Performance Evaluation Summary
1. Understand your environment
• Know you’re biased
• Account for chance
2. Ask the critical questions
Understand Your Environment
How much lottery and how much math problem?
vs.
Know You’re Biased
1) Non-regressive predictions
2) Outcome bias
3) Hindsight bias
4) Narrative bias
Account for Chance
• The key issue: Persistence
• The more fundamental (skill-related) a performance measure is, the
more it will persist over time
• The more chance-related a performance measure is, the more it will
regress to the mean over time
People
Analytics
STAFFING
The Staffing Cycle
Hirin
g
Internal
Mobility &
Career
Development
Attritio
n
• Basic facts about staffing
processes
• The value of analysis
• Possible analytic
approaches
Question
Which of the following methods
of evaluating job candidates
is most effective at
predicting subsequent
performance?
Which is least effective?
Options
•Job knowledge
tests
• Cognitive ability
tests
• Personality tests
• Reference checks
•Structured
interviews
•Unstructured
interviews
•Work samples
•Integrity tests
Correlation with subsequent performance (0-1)
0.54
0.5
1
0.5
1
0.48
0.4
1
0.3
1
0.3
1
0.26
Work Samples
Cognitive Ability
Tests
Structured Interviews
Job Knowledge Tests
Integrity Tests
Unstructured
interviews
Personality test
(conscientiousness)
Reference Checks
Evaluating Staffing Options
• Which routes lead to better
performance?
• What is the effect on cost?
How Does Data Analysis Compare to
Human Judgment
The Bad News
• Combination of various tests and selection methods leaves much of
performance unexplained
The Worse News
• Implementation of algorithms reduced turnover in call centers
• Turnover was lower the less often managers over-ruled the algorithm
https://hbr.org/2019/02/how-companies-can-use-employee-
data-responsibly
https://www.hr-brew.com/stories/2022/05/06/burnout-tech-
seeks-to-identify-signs-of-workers-mental-di...

More Related Content

PDF
People analytics the great data dilemma
PDF
People Analytics: Creating The Ultimate Workforce
PPTX
HRAnalyticsstarting2october262021.pptx
PPTX
HRAnalytics starting capturing methodss2october2021.pptx
PPTX
What People Analytics Can’t Capture" by Daniel Goleman
PPTX
The Future of HR: From Metrics to Analytics [Webcast]
PDF
xv-whitepaper-workforce
PDF
Dr. Priyameet Kaur Keer: Emergence of HR Analytics
People analytics the great data dilemma
People Analytics: Creating The Ultimate Workforce
HRAnalyticsstarting2october262021.pptx
HRAnalytics starting capturing methodss2october2021.pptx
What People Analytics Can’t Capture" by Daniel Goleman
The Future of HR: From Metrics to Analytics [Webcast]
xv-whitepaper-workforce
Dr. Priyameet Kaur Keer: Emergence of HR Analytics

Similar to People Analytics_Introduction (20)

PDF
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
PDF
Hr analytics – demystified!
PPTX
What people analytics can’t capture!
PPTX
Human Capital Analytics 2.2016
PPTX
module 3 HR Analytics for VTU MBA syllabus.pptx
PPTX
human resource analytics in automobiles vtu mba.pptx
PPTX
Lecture 1 Human Resource Analytics
PPTX
Hr analytics
PPTX
Hr analytics 2
PDF
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
PDF
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
PDF
CIPD Getting started with people analytics
PPTX
EMPOWERING MANAGERS THROUGH POSITIVE POLITICAL SKILLS MENTORSHIP INTRODU...
PPTX
HR Analytics First module corporate expericence.pptx
PDF
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
PPTX
Future Belongs to People Analytics
PPTX
module 3 HR Analytics for 4th MBA VTU .pptx
PDF
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
PPTX
Hr analytics versie hr techdag
PDF
A Study on Employees Perception with Special Reference to Private Bank, Tirun...
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
Hr analytics – demystified!
What people analytics can’t capture!
Human Capital Analytics 2.2016
module 3 HR Analytics for VTU MBA syllabus.pptx
human resource analytics in automobiles vtu mba.pptx
Lecture 1 Human Resource Analytics
Hr analytics
Hr analytics 2
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
CIPD Getting started with people analytics
EMPOWERING MANAGERS THROUGH POSITIVE POLITICAL SKILLS MENTORSHIP INTRODU...
HR Analytics First module corporate expericence.pptx
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
Future Belongs to People Analytics
module 3 HR Analytics for 4th MBA VTU .pptx
(eBook PDF) Predictive HR Analytics: Mastering the HR Metric 2nd Edition
Hr analytics versie hr techdag
A Study on Employees Perception with Special Reference to Private Bank, Tirun...
Ad

Recently uploaded (20)

PPT
lectureusjsjdhdsjjshdshshddhdhddhhd1.ppt
PPTX
Managing Community Partner Relationships
PPTX
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
PPTX
retention in jsjsksksksnbsndjddjdnFPD.pptx
PDF
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
PPTX
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
PPTX
Market Analysis -202507- Wind-Solar+Hybrid+Street+Lights+for+the+North+Amer...
PPTX
Pilar Kemerdekaan dan Identi Bangsa.pptx
PPTX
importance of Data-Visualization-in-Data-Science. for mba studnts
PPTX
STERILIZATION AND DISINFECTION-1.ppthhhbx
PDF
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
PPTX
Introduction to Inferential Statistics.pptx
PPTX
Database Infoormation System (DBIS).pptx
PPT
Predictive modeling basics in data cleaning process
PDF
Introduction to Data Science and Data Analysis
PDF
Global Data and Analytics Market Outlook Report
PPTX
IBA_Chapter_11_Slides_Final_Accessible.pptx
PDF
annual-report-2024-2025 original latest.
PPT
ISS -ESG Data flows What is ESG and HowHow
PDF
Microsoft Core Cloud Services powerpoint
lectureusjsjdhdsjjshdshshddhdhddhhd1.ppt
Managing Community Partner Relationships
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
retention in jsjsksksksnbsndjddjdnFPD.pptx
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
Market Analysis -202507- Wind-Solar+Hybrid+Street+Lights+for+the+North+Amer...
Pilar Kemerdekaan dan Identi Bangsa.pptx
importance of Data-Visualization-in-Data-Science. for mba studnts
STERILIZATION AND DISINFECTION-1.ppthhhbx
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
Introduction to Inferential Statistics.pptx
Database Infoormation System (DBIS).pptx
Predictive modeling basics in data cleaning process
Introduction to Data Science and Data Analysis
Global Data and Analytics Market Outlook Report
IBA_Chapter_11_Slides_Final_Accessible.pptx
annual-report-2024-2025 original latest.
ISS -ESG Data flows What is ESG and HowHow
Microsoft Core Cloud Services powerpoint
Ad

People Analytics_Introduction

  • 2. Outline  What is People Analytics?  Benefits of People Analytics  Signals & Trends  People Analytics examples  How can HR start building People Analytics capabilities?  The future of People Analytics
  • 3. What is People Analytics? The use of data and analytic tools to inform decisions about how to manage people
  • 4. What is People Analytics? Analytics, applied to “people issues” – hiring, performance management, compensation, retention, etc.
  • 6. People Analytics known as… Touches all people-related issues in organizations to make predictions – Performance evaluation – Hiring / Assessment – Retention – Learning and development – Team composition – Etc. HR ANALYTICS TALENT MANAGEMENT WORKFORCE ANALYTICS
  • 7. Why is People Analytics emerging now? › Technical progress – HR data availability, processing power, analytical tools, etc. › Analytical capabilities – data analysis skills, plenty of online courses, increasing use of popular analytical tools in day to day work (e.g. Microsoft Excel, Power BI, Tableau, etc.) › Growing recognition of behavioral biases. › Increase in ROI ($10,66 ROI per 1$ -> Nucleus Research, 2011)
  • 8. What does People Analytics require? › Metrics/predictors for HR - such as attrition rates, KPIs/performance stats, retention data, etc. › Software for meaningful data collection that will help to diagnose the efficiency of current HR processes and their impact on wellbeing, happiness, and bottom-line performance. › Statistical modeling and machine learning methods – to establish the probability of various scenarios and make informed decisions that facilitate planning and growth.
  • 9. What about the size of data? The size and scope of data for People Analytics can depend on several factors: • These can be large and complex datasets or simpler measures such as employee surveys. • The amount of people data used may depend on the size of the organization and its needs.
  • 10. People Analytics comes to minimize human biases and replace intuitive decision making Use Employee Data Responsibly
  • 11. Why does HR need People Analytics?
  • 12. Deloitte's 2023 Global Human Capital Trends report https://www2.deloitte.com/us/en/insights/focus/human- capital-trends.html#leading-in-a-boundaryless-world 56 % of respondents said organizations have made moderate to significant progress in People Analytics over the past 10 years.
  • 14. Leading Companies in People Analytics display seven key characteristics.
  • 16. Signals: This trend applies to you if …  Your employees feel as if their every move is being monitored, (increasing stress, job dissatisfaction, and turnover, and leading to a lack of trust).  Employees are sharing information (data) more readily outside of the organization (e.g., LinkedIn) but are reticent to provide it through organizational channels due to a perceived lack of benefit to them.  Your organization faces increased challenges and pressure from regulators (e.g. GDPR) related to data reporting, privacy, and protection. Source: Deloitte 2023 Global Human Capital Trends survey Use Employee Data Responsibly
  • 17. Barriers When asked to identify top barriers to realizing value from people data, 27% of respondents cited culture, making it the most common barrier. Source: Deloitte 2023 Global Human Capital Trends survey
  • 18. Benefits When asked to identify the top benefits from their organization’s approach to using people data, the top response was increased worker engagement and well-being of the workforce. Source: Deloitte 2023 Global Human Capital Trends survey
  • 19. Which performance metrics do Armenian companies collect for PA? How do companies make sure they don’t harbor surveillance in the workplace and there is responsible use of worker data? Are there legal requirements for processing data?
  • 20. People Analytics Examples • Tracks the performance of all teachers, comparing it to evaluations when they were hired • Refined the most productive steps in the hiring process, where to allocate more resources, etc.
  • 21. • Systematically tracked interview predictions about new hires to figure out how good they were at it • Answer: Not very • So dramatically reduced the # of interviewers Predicting Hire Success
  • 22. People Analytics Examples • Believes a 1% increase in retention can save $75- $ 100m/year • 3-year study: Changing jobs increases employee “stickiness” • Increased internal postings of open jobs from <50%
  • 23. Many, Many Others • Firms in technology, financial services, telecommunications, automotive, consumer packaged goods, energy, not-for-profit… …are finding: • Better levers for retaining key employees • More diagnostic methods for hiring • Who their most valuable employees are • How to compose the most productive teams • Etc.
  • 25. People Analytics In Use Purpose of performance evaluation • Feedback • Rewards / punishment • Performance evaluation, not talent management. Tough to compare employees if not in identical situations. • Helpful starting place, for this seminar (and often for life): Begin by assuming all employees are equal ability
  • 28. NOISE › The fundamental challenge in performance evaluation is that performance measures are noisy (i.e., outcomes are imperfectly related to employee effort) › For any given level of effort, a range of outcomes can occur due to factors outside the employee’s control: • Competitors, team members, bosses, the economy, etc. The challenge: Separating skill from luck
  • 29. A Simple Model › There are two components to performance: • In informal terms:Real Tendency + Luck • In more formal terms: y = x + e, • x = true ability, and • e = error, randomly distributed around 0.  What happens when we sample on extreme performance? What underlies extreme success and failure? • Extreme success = f(superior ability, positive error) • Extreme failure = f(inferior ability, negative error) • Consequences?
  • 30. Regression to the Mean A study was recently conducted examining the performance of the 283 stock mutual funds that existed during the 1990s. The study divided the 1990s into an early period (1990-1994) and a late period (1995-1999). Below are the 10 funds that had the highest rate of return in the early period (with their names disguised), ranked from 1 to 10. Predict their rank for the late 1990s.
  • 31. Examples • Officer in the Israeli Air Force— “Punishment is more effective than praise. Whenever I punish a pilot after a really poor flight, I see better performance the next time. Whenever I praise a pilot after an excellent flight, I see worse performance the next time.” • Peters and Waterman’s book, In Search of Excellence. They selected 43 high performing companies in the early 1980s, and looked to see what practices they used (some that they discovered were the organizational equivalent of “brushing teeth”)
  • 33. Regression to the Mean • Anytime you sample based on extreme values of one attribute, any other attribute that is not perfectly related will tend to be closer to the mean value • “Attributes” can be: • Performance at different points in time • E.g., last year’s stock returns and this year’s • Different qualities within the same entity • E.g., a person’s running speed and language ability
  • 34. What Gets in the Way of Seeing This? Among other things: • Outcome bias • Hindsight bias • Narrative seeking In short, we make sense of the past • We find a story that connects all the dots • Chance plays too small a role in these stories
  • 36. Extrapolating From Small Samples • Principle: Sample means converge to the population mean as the sample size increases. (This is known as the Central Limit Theorem.) Thus, you will see more extreme values in small samples. • When are you more likely to see a .400 season batting average in baseball – May 1 or Sept. 1? • In which hospital are you more likely to see a dramatically higher % of boys than girls (or vice versa) born on any given day – a small community hospital (e.g., 5 births/day) or a large city hospital (e.g., 100 births/day)?
  • 37. Extrapolating From Small Samples Your firm has two plants, one large and one small, which mass produce a standard computer chip. Other than the amount they produce, the two plants are identical in all essential regards. Both use the same technology to produce the same product. When properly functioning, this particular technology produces one percent (1%) defective items. Whenever the number of defective items from one day’s production exceeds two percent (2%), a special note is made in the quality control log to “flag” the problem. At the end of the quarter, which plant would you expect to have more “flagged” days in its quality control log? Please mark one. 22% A) The small plant 30% B) The large plant 48% C) The same number on average
  • 38. “Law of Small Numbers” • People believe small samples closely share the properties of the underlying population • This means they too readily infer the population’s properties (e.g., average) from the sample’s • That is: They neglect the role variability (aka chance) inevitably plays in small samples
  • 39. The Wisdom of Crowds • The average of a large number of forecasts reliably outperforms the average individual forecast • Idiosyncratic errors offset each other • E.g., Galton’s (1906) county fair contest • Many other examples
  • 40. The Wisdom of Crowds • But the value of the crowd critically depends on the independence of their opinions • Independent means uncorrelated • If correlated, the value of additional opinions quickly diminishes
  • 41. Impact of Correlation Actual Number of Experts (n) Equivalent Number of Independent Experts (n*) n*= n 1+(n- 1)r n*®1/ r as n ® ¥ Clemen and Winkler (1985)
  • 42. Signal Independence • People are bad at accounting for this effect • Even when you tell them exactly what the correlation is, people do not properly adjust (Enke & Zimmerman, 2015)
  • 43. Signal Independence • Sources of correlation between two opinions? • They’ve discussed it already! • They talk to the same people • They have the same background – from the same place, trained the same way, same historical experiences, etc. • Need to find ways to keep opinions independent, and add independent perspectives to experienced groups
  • 44. Consider Broader Set of Objectives • Organizations generally care about how a person goes about his/her job • Most important: impact on others • People consider too few objectives (Bond, Carlson & Keeney, 2008) • Systematically omit nearly ½ of the objectives they later identify as personally relevant • Leads many firms to rely on too narrow a set of performance measures
  • 45. Process vs. Outcome • Famously hard-charging Dell Computers changed their performance evaluations in the early 2000s • Before change: 100% results • After change: • 50% what an employee accomplished • 50% how he/she accomplished it, as judged those affected
  • 46. Process vs. Outcome • The more uncertainty in the environment, i.e., the less control an employee has over exact outcomes, the more a firm should emphasize process in their evaluations.
  • 47. Process vs. Outcome • Use analytics to better understand, and focus on, the processes that tend to produce desired outcomes • Key issue: Identify the fundamental drivers of value
  • 48. Performance Evaluation Summary 1. Understand your environment • Know you’re biased • Account for chance 2. Ask the critical questions
  • 49. Understand Your Environment How much lottery and how much math problem? vs.
  • 50. Know You’re Biased 1) Non-regressive predictions 2) Outcome bias 3) Hindsight bias 4) Narrative bias
  • 51. Account for Chance • The key issue: Persistence • The more fundamental (skill-related) a performance measure is, the more it will persist over time • The more chance-related a performance measure is, the more it will regress to the mean over time
  • 53. The Staffing Cycle Hirin g Internal Mobility & Career Development Attritio n • Basic facts about staffing processes • The value of analysis • Possible analytic approaches
  • 54. Question Which of the following methods of evaluating job candidates is most effective at predicting subsequent performance? Which is least effective?
  • 55. Options •Job knowledge tests • Cognitive ability tests • Personality tests • Reference checks •Structured interviews •Unstructured interviews •Work samples •Integrity tests
  • 56. Correlation with subsequent performance (0-1) 0.54 0.5 1 0.5 1 0.48 0.4 1 0.3 1 0.3 1 0.26 Work Samples Cognitive Ability Tests Structured Interviews Job Knowledge Tests Integrity Tests Unstructured interviews Personality test (conscientiousness) Reference Checks
  • 57. Evaluating Staffing Options • Which routes lead to better performance? • What is the effect on cost?
  • 58. How Does Data Analysis Compare to Human Judgment The Bad News • Combination of various tests and selection methods leaves much of performance unexplained The Worse News • Implementation of algorithms reduced turnover in call centers • Turnover was lower the less often managers over-ruled the algorithm

Editor's Notes

  • #18: However, “culture” may be a broad proxy for misaligned values or disagreements over if, how, or when worker data should be used.
  • #19: However, “culture” may be a broad proxy for misaligned values or disagreements over if, how, or when worker data should be used.