SlideShare a Scribd company logo
Establishing an Ethics
Framework for Predictive
Analytics in Higher Education
Cyber Summit 2016, Banff
Stephen Childs, Institutional Analyst
October 27, 2016
Disclamer
 The content of this presentation represents my views only.
and not that of my employer, the University of Calgary.
 I am not qualified to accurately describe University of Calgary
policy in the areas discussed in this talk.
 Please contact the University if you have policy questions.
Data Abundance in Higher Education
3
Big Data, Big Problems
 Advancing technology
—Better data collection
—Handle more data
—Apply algorithms to data
 We know more about our students
 Can make predictions about their behavior
 Very few guidelines about this practice
Solutions
 Develop an ethics framework around student data.
 Build on existing guidelines.
 Build on the norms of service to students
 Do this now while these practices are new.
Outline
 Introduction
 Students and Student Data
 Predictive Analytics
 Existing Frameworks
 Next Steps
About Me
About My Office
 Office of Institutional Analysis
 https://oia.ucalgary.ca/
What OIA Does
About the University
Students
Student Data
 Application
 Student Information System
 LMS
 Unicard
 Surveys
 Residence
 Facilities
 Awarding Degrees
 Grades
 USRI
 IT usage
 Others…
Student Data
 Students can opt out of some data collection, but not all
 Student give us their data because they trust us
 We need to deserve that trust!
—Respect student privacy
—Transparency about how data is used
—Accountability
—Consultation
—Consider the Consequences
Privacy
Access to Data
Transparency and Accountability
 Internalize norms is not enough!
 How Universities use data should be known
—We aren’t corporations with competitive secrets
—We need to set up ways to report and share
 We need to be able describe what happened!
 Log events
 Version control your software
 Develop reporting methods
Consultation
Consider the Consequences
 Moving from institutional decision
making to acting on individual data
 Lathe of Heaven – a mad social
scientist
Predictive Analytics
Best Practices using Predictive Analytics
 Have to carefully present information to students
—Present a positive outlook
—Don’t personalize it – talk about a group of similar
students.
 The factors in the model may be less deterministic than
unobserved factors.
 Difference between causality and correlation.
 Beware the self-fulfilling prophecy
Cathy O’Neil
 @mathbabe, mathbabe.org
 Mathematician, former hedge-fund
quant
Weapons of Math Destruction
 Three factors make a model a WMD:
—Is the participant aware of the model? Is the model
opaque or invisible?
—Does the model work against the participant’s interest? Is
it unfair? Does it create feedback loops?
—Can the model scale?
Student Data Principles
 http://studentdataprinciples.org/
 Purpose and use of student data
 Timely access to data
 Data should not replace professional judgement.
 Data governance, security, breach notification
Student Data Pledge
 http://www.edtechmagazine.com/k12/article/2015/03/prote
ct-personal-student-information-pair-organizations-
recommends-commitment
 Don’t sell student data, use data to target ads, or profile
students for non-educational purposes
 Don’t collect more information or retain information longer
than necessary.
 Do disclose how, what and why
uCalgary Data Rules
 Freedom of Information and Privacy Act (1999)
—Students must be able to correct own info
—University must provide own info upon confirmation of ID
 Categories of Data Confidentiality
 Research Ethics Boards
—Data collection for University operations does not
generally fall under REB jurisdiction.
Financial Modeler’s Manifesto
 https://www.wilmott.com/financial-modelers-manifesto/
 Emanuel Derman and Paul Wilmott – January 7, 2009
 The Modelers’ Hippocratic Oath
— I will remember that I didn’t make the world, and it doesn’t satisfy my
equations.
— Though I will use models boldly to estimate value, I will not be overly
impressed by mathematics.
— I will never sacrifice reality for elegance without explaining why I have
done so.
— Nor will I give the people who use my model false comfort about its
accuracy. Instead, I will make explicit its assumptions and oversights.
— I understand that my work may have enormous effects on society and
the economy, many of them beyond my comprehension.
Responsible Use of Student Data in Higher Education
 http://gsd.su.domains/
 Opportunity to understand student learning and enhance
educational attainment.
 New questions about the ethical collection, use, and sharing
of information.
 Commitments to honor the integrity, discretion, and humanity
of students.
 Improve practice in light of accumulating information and
knowledge.
Maciej Cegłowski
 https://pinboard.in and @pinboard
 http://idlewords.com/talks/
 Two talks on Data in particular:
—http://idlewords.com/talks/deep_fried_data.htm
—http://idlewords.com/talks/haunted_by_data.htm
Basic Framework
 Safeguard Student Privacy
—Vendors; Monetizing Data
 Strong internal norms around data
 Consider and Measure Outcomes
 Work with Data Owners and Stewards
 Responsibility to Educate
 Consult with Students and Stakeholders
 Data should have a clear purpose
Next Steps
 Write down your norms/expectations for working with
Student data
 Set up a discussion with your co-workers about it.
 Seek out others who perform a similar role and discuss it.
 Discuss with the Student Data Steward at your institution.
 Send me your comments!
Continue the Conversation
 Follow me on twitter: @sechilds
 Stephen.Childs@ucalgary.ca or sechilds@gmail.com
 https://oia.ucalgary.ca/Contact
 https://www.meetup.com/PyData-Calgary/

More Related Content

PPTX
Managing confidential data
PDF
Privacy tool osha comments
PPTX
Assessing Digital Output in New Ways
PDF
Big Data for Library Services (2017)
PPTX
Confidential data management_key_concepts
PPTX
Introduction to Big Data Analytics
PPTX
BROWN BAG TALK WITH MICAH ALTMAN, SOURCES OF BIG DATA FOR SOCIAL SCIENCES
PDF
Data and Knowledge as Commodities
Managing confidential data
Privacy tool osha comments
Assessing Digital Output in New Ways
Big Data for Library Services (2017)
Confidential data management_key_concepts
Introduction to Big Data Analytics
BROWN BAG TALK WITH MICAH ALTMAN, SOURCES OF BIG DATA FOR SOCIAL SCIENCES
Data and Knowledge as Commodities

What's hot (20)

PDF
Data science and ethics in fundraising
PPTX
Data science.chapter-1,2,3
PPTX
Text analytics
PPTX
Open Data Bay Area: Interesting Problems in Academic Data
PDF
Making data sharing count
PDF
Slides | Research data literacy and the library
PPTX
Data Discovery and Visualization
PPT
The Semantic Web: It's for Real
PPTX
Behind the scenes of data science
PPTX
Managing Your Research Data
PPT
Linking Data to Publications through Citation and Virtual Archives
PPTX
Research Ethics and Use of Restricted Access Data
PPTX
Fsci 2018 wednesday1_august_am6
PPTX
Methodologies for Addressing Privacy and Social Issues in Health Data: A Case...
PPTX
Noshir Contractor's view on the future of Linked Data
PPT
Introduction to Data Management and Sharing
PPSX
USING BIGDATA WITH ACADEMIC LIBRARY SERVICES: A VIEW
PPTX
Data Mining and Data Warehouse
PDF
Knowledge Representation on the Web
PPTX
Advancing Foundation and Practice of Software Analytics
Data science and ethics in fundraising
Data science.chapter-1,2,3
Text analytics
Open Data Bay Area: Interesting Problems in Academic Data
Making data sharing count
Slides | Research data literacy and the library
Data Discovery and Visualization
The Semantic Web: It's for Real
Behind the scenes of data science
Managing Your Research Data
Linking Data to Publications through Citation and Virtual Archives
Research Ethics and Use of Restricted Access Data
Fsci 2018 wednesday1_august_am6
Methodologies for Addressing Privacy and Social Issues in Health Data: A Case...
Noshir Contractor's view on the future of Linked Data
Introduction to Data Management and Sharing
USING BIGDATA WITH ACADEMIC LIBRARY SERVICES: A VIEW
Data Mining and Data Warehouse
Knowledge Representation on the Web
Advancing Foundation and Practice of Software Analytics
Ad

Similar to Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education (20)

PPTX
Learning and Educational Analytics
PPTX
Learning Analytics in Education: Using Student’s Big Data to Improve Teaching
PDF
Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
PDF
Prioritizing Ethical Use of Learning Data.pdf
PPTX
Ethical challenges for learning analytics
PDF
How AI will change the way you help students succeed - SchooLinks
PPTX
LERU Presentation - March 2017
PPT
BIG-DATAPPTFINAL.ppt
PPT
Big Data ( Charactertics of 6vs of Big Data)
PPTX
How to Use Learning Analytics in Moodle
PPTX
Precon presentation 2015
PDF
Statistical Ways of Seeing Things in Maths
PPTX
Real-time applications of Data Science.pptx
PPT
Assessment
PPTX
On the horizon for learning analytics
PPTX
Ways of seeing learning - 2017v1.0 - NUI Galway University of Limerick postgr...
PDF
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...
PDF
How data informs decision making 2
PDF
insight-centre-galway-learning-analytics
PPTX
Aligning Learning Analytics with Classroom Practices & Needs
Learning and Educational Analytics
Learning Analytics in Education: Using Student’s Big Data to Improve Teaching
Sdal air education workforce analytics workshop jan. 7 , 2014.pptx
Prioritizing Ethical Use of Learning Data.pdf
Ethical challenges for learning analytics
How AI will change the way you help students succeed - SchooLinks
LERU Presentation - March 2017
BIG-DATAPPTFINAL.ppt
Big Data ( Charactertics of 6vs of Big Data)
How to Use Learning Analytics in Moodle
Precon presentation 2015
Statistical Ways of Seeing Things in Maths
Real-time applications of Data Science.pptx
Assessment
On the horizon for learning analytics
Ways of seeing learning - 2017v1.0 - NUI Galway University of Limerick postgr...
Recommendation of Data Mining Technique in Higher Education Prof. Priya Thaka...
How data informs decision making 2
insight-centre-galway-learning-analytics
Aligning Learning Analytics with Classroom Practices & Needs
Ad

More from Cybera Inc. (20)

PDF
Cyber Summit 2016: Technology, Education, and Democracy
PPTX
Cyber Summit 2016: Understanding Users' (In)Secure Behaviour
PPT
Cyber Summit 2016: Insider Threat Indicators: Human Behaviour
PPTX
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
PPTX
Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data
PDF
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
PDF
Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...
PDF
Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data
PPT
Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...
PPT
Privacy, Security & Access to Data
PDF
Do Universities Dream of Big Data
PDF
Predicting the Future With Microsoft Bing
PPTX
Analytics 101: How to not fail at analytics
PPTX
Are MOOC's past their peak?
PPTX
Opening the doors of the laboratory
PPTX
Open City - Edmonton
PPTX
Unlocking the power of healthcare data
PPT
Checking in on Healthcare Data Analytics
PPTX
Open access and open data: international trends and strategic context
PPTX
I didn't know i was a geomatics company
Cyber Summit 2016: Technology, Education, and Democracy
Cyber Summit 2016: Understanding Users' (In)Secure Behaviour
Cyber Summit 2016: Insider Threat Indicators: Human Behaviour
Cyber Summit 2016: Research Data and the Canadian Innovation Challenge
Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data
Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse
Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...
Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data
Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...
Privacy, Security & Access to Data
Do Universities Dream of Big Data
Predicting the Future With Microsoft Bing
Analytics 101: How to not fail at analytics
Are MOOC's past their peak?
Opening the doors of the laboratory
Open City - Edmonton
Unlocking the power of healthcare data
Checking in on Healthcare Data Analytics
Open access and open data: international trends and strategic context
I didn't know i was a geomatics company

Recently uploaded (20)

PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Advanced IT Governance
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PPTX
MYSQL Presentation for SQL database connectivity
PPT
Teaching material agriculture food technology
PDF
NewMind AI Monthly Chronicles - July 2025
PDF
Machine learning based COVID-19 study performance prediction
PDF
GamePlan Trading System Review: Professional Trader's Honest Take
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
Cloud computing and distributed systems.
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
cuic standard and advanced reporting.pdf
20250228 LYD VKU AI Blended-Learning.pptx
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Advanced IT Governance
Spectral efficient network and resource selection model in 5G networks
Dropbox Q2 2025 Financial Results & Investor Presentation
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
MYSQL Presentation for SQL database connectivity
Teaching material agriculture food technology
NewMind AI Monthly Chronicles - July 2025
Machine learning based COVID-19 study performance prediction
GamePlan Trading System Review: Professional Trader's Honest Take
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Network Security Unit 5.pdf for BCA BBA.
Cloud computing and distributed systems.
Reach Out and Touch Someone: Haptics and Empathic Computing
NewMind AI Weekly Chronicles - August'25 Week I
cuic standard and advanced reporting.pdf

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

  • 1. Establishing an Ethics Framework for Predictive Analytics in Higher Education Cyber Summit 2016, Banff Stephen Childs, Institutional Analyst October 27, 2016
  • 2. Disclamer  The content of this presentation represents my views only. and not that of my employer, the University of Calgary.  I am not qualified to accurately describe University of Calgary policy in the areas discussed in this talk.  Please contact the University if you have policy questions.
  • 3. Data Abundance in Higher Education 3
  • 4. Big Data, Big Problems  Advancing technology —Better data collection —Handle more data —Apply algorithms to data  We know more about our students  Can make predictions about their behavior  Very few guidelines about this practice
  • 5. Solutions  Develop an ethics framework around student data.  Build on existing guidelines.  Build on the norms of service to students  Do this now while these practices are new.
  • 6. Outline  Introduction  Students and Student Data  Predictive Analytics  Existing Frameworks  Next Steps
  • 8. About My Office  Office of Institutional Analysis  https://oia.ucalgary.ca/
  • 12. Student Data  Application  Student Information System  LMS  Unicard  Surveys  Residence  Facilities  Awarding Degrees  Grades  USRI  IT usage  Others…
  • 13. Student Data  Students can opt out of some data collection, but not all  Student give us their data because they trust us  We need to deserve that trust! —Respect student privacy —Transparency about how data is used —Accountability —Consultation —Consider the Consequences
  • 16. Transparency and Accountability  Internalize norms is not enough!  How Universities use data should be known —We aren’t corporations with competitive secrets —We need to set up ways to report and share  We need to be able describe what happened!  Log events  Version control your software  Develop reporting methods
  • 18. Consider the Consequences  Moving from institutional decision making to acting on individual data  Lathe of Heaven – a mad social scientist
  • 20. Best Practices using Predictive Analytics  Have to carefully present information to students —Present a positive outlook —Don’t personalize it – talk about a group of similar students.  The factors in the model may be less deterministic than unobserved factors.  Difference between causality and correlation.  Beware the self-fulfilling prophecy
  • 21. Cathy O’Neil  @mathbabe, mathbabe.org  Mathematician, former hedge-fund quant
  • 22. Weapons of Math Destruction  Three factors make a model a WMD: —Is the participant aware of the model? Is the model opaque or invisible? —Does the model work against the participant’s interest? Is it unfair? Does it create feedback loops? —Can the model scale?
  • 23. Student Data Principles  http://studentdataprinciples.org/  Purpose and use of student data  Timely access to data  Data should not replace professional judgement.  Data governance, security, breach notification
  • 24. Student Data Pledge  http://www.edtechmagazine.com/k12/article/2015/03/prote ct-personal-student-information-pair-organizations- recommends-commitment  Don’t sell student data, use data to target ads, or profile students for non-educational purposes  Don’t collect more information or retain information longer than necessary.  Do disclose how, what and why
  • 25. uCalgary Data Rules  Freedom of Information and Privacy Act (1999) —Students must be able to correct own info —University must provide own info upon confirmation of ID  Categories of Data Confidentiality  Research Ethics Boards —Data collection for University operations does not generally fall under REB jurisdiction.
  • 26. Financial Modeler’s Manifesto  https://www.wilmott.com/financial-modelers-manifesto/  Emanuel Derman and Paul Wilmott – January 7, 2009  The Modelers’ Hippocratic Oath — I will remember that I didn’t make the world, and it doesn’t satisfy my equations. — Though I will use models boldly to estimate value, I will not be overly impressed by mathematics. — I will never sacrifice reality for elegance without explaining why I have done so. — Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights. — I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
  • 27. Responsible Use of Student Data in Higher Education  http://gsd.su.domains/  Opportunity to understand student learning and enhance educational attainment.  New questions about the ethical collection, use, and sharing of information.  Commitments to honor the integrity, discretion, and humanity of students.  Improve practice in light of accumulating information and knowledge.
  • 28. Maciej Cegłowski  https://pinboard.in and @pinboard  http://idlewords.com/talks/  Two talks on Data in particular: —http://idlewords.com/talks/deep_fried_data.htm —http://idlewords.com/talks/haunted_by_data.htm
  • 29. Basic Framework  Safeguard Student Privacy —Vendors; Monetizing Data  Strong internal norms around data  Consider and Measure Outcomes  Work with Data Owners and Stewards  Responsibility to Educate  Consult with Students and Stakeholders  Data should have a clear purpose
  • 30. Next Steps  Write down your norms/expectations for working with Student data  Set up a discussion with your co-workers about it.  Seek out others who perform a similar role and discuss it.  Discuss with the Student Data Steward at your institution.  Send me your comments!
  • 31. Continue the Conversation  Follow me on twitter: @sechilds  Stephen.Childs@ucalgary.ca or sechilds@gmail.com  https://oia.ucalgary.ca/Contact  https://www.meetup.com/PyData-Calgary/

Editor's Notes

  • #4: Higher Education Institutions have a lot of data. We have a lot of data relating to our students. Asymmetric power and information relationship
  • #5: Research Ethics Boards – don’t typically cover this kind of work. How do we know if our models are BAD and if they are harming students?
  • #6: Talk Objectives: Understand why responsible use of student data matters Give you resources and language to talk about it Start a conversation about student data
  • #8: Analyst and Researcher - MA Economics from WLU Handle Student Data and Build models of student behavior Parts of my job are becoming easier Better data science tools – mention PyData Calgary Not much guidance
  • #9: Office of Institutional Analysis Reports to Vice-Provost (Planning & Resource Allocation) …Who reports to the Provost & VP (Academic) …Who reports to the President
  • #10: We don’t own the institution’s data – but we become experts in it. (Particularly student data) Create reports for: SLT, Internal stakeholders, Government, Public Fact Book, USRI (instruction ratings), University Rankings, U-15 Institutional research is great – work with data, wide variety of tasks
  • #11: 50th Anniversary Year Eyes High Energizing Eyes High Better serve students and community
  • #12: Central focus of the University Attend for various reasons: job, skills, education -- University is one of the best paths to the life they want. Student pay the University to act as a gatekeeper Demographic changes, Different support needs Asymmetrical Power and Information
  • #13: Enter the system as soon as they apply
  • #15: Privacy is just the starting point Employees should only have access to data they need Security – need to mention @SwiftOnSecurity De-Identify Data where necessary
  • #16: Access to data his a hard problem To restrictive and nothing gets done To open and problems result Employees need to internalize norms about data Highly specific to institutional culture Need employees to use their own judgment
  • #17: Version control - Software Carpentry
  • #18: Are we hearing the students about how we use their data? Do they care about this? How can we consult with students and get a useful result?
  • #19: Worth reading - but don't watch the TV movie.
  • #20: Technology used in the business world Statistics, Econometrics, Machine Learning Universities have experts on all three. Expertise starting to move into admin Bad use of aggregate statistics can lead to bad policy. Bad use of individual data can lead to bad individual outcomes Individual outcomes can be systematically bad https://ischool.syr.edu/infospace/2013/11/13/using-predictive-analytics-to-understand-your-business/ https://ischool.syr.edu/infospace/wp-content/files/2013/12/crystal-ball-e1385997891512.jpg
  • #23: Talk about examples of WMD at this point: prison recidivism models used in sentencing,