MedU Plenary Session on Analytics

Editor's Notes

• #3: 2012 – Georgia State – $2M investment nets $10M extra tuition a year, increased URM enrolment (22% hispanic, 15% black male) and increased graduation rate (226% hispanic and 111% in black males). 2013 – LAK in Leuven, Belgium 2014 - Anything from Educause? # of presentations at GIR
• #4: Learning analytics is about what learners are learning, and their contexts they do it in. Education data mining is, in many ways, a quantitative analysis focused on prediction or mining the data first Academic analytics is the business intelligence applied to education data. Here we might move across institutional systems and Student information.
• #5: Learning analytics is about what learners are learning, and their contexts they do it in. Education data mining is, in many ways, a quantitative analysis focused on prediction or mining the data first Academic analytics is the business intelligence applied to education data. Here we might move across institutional systems and Student information.
• #9: Today’s session is about thinking about the data in MedU, and what that could mean to integrate the educational concepts from this morning, with what data MedU collects on learners. We will take a high-level look at examples, and then break for you to work in groups to pick a concept, and disuss its implications for VPs with MedU.
• #10: Any visualized dashbaord is a form of feedback, to whatever user is looking at the dashboard. But what kind of feedback is most optimal for Entrada to take on?
• #11: This is Purdue Course signals, started in 2007 The ‘signals’ are traffic signals, which tell learners if they are at risk or not of doing poorly in a course The metrics (# of assignments done/not done, # of quizzes done/not done) are used to calculate how a student is progressing. Any dashboard is fundamentally feedback. This is feedback, but it’s not making learning visible.
• #12: Let’s make it about learning. Here are final exam scores, reported to learners on a dashboard. What are your thoughts? Turn to the person next to you, behind you, in front of you and discuss for 2 mins.
• #13: This is Vanderbilt VSTAR On this dashboard, we see a series of notices: There is a ‘past due’ task IF learners have started (or not) Any notes the faculty member can make about the learner progress
• #14: Here is another dashboard from Vanderbilt, where learners report their goals against the competencies the program monitors. The learner and their faculty coach see this dashboard, so they can monitor progress.
• #15: The MedU dashboard, at it’s most basic feature, is visualizing learning. What does this mean? It is a type of feedback to learners and faculty. But what do they undersatnd?
• #16: The biggest change was to make it more readable, to provide the necessary information right at faculty members’ fingertips. This is another example of how the dashboard impacts the system.
• #17: This means: Time lag between feedback & action We don’t know what the target user took away A systems level change – more hires, more training
• #20: eedback prompts, outcome measures, reporting to learner, reporting to faculty) Move into groups of 4-6 people Decide which element you want to do What would it look like? What metrics matter to you?
• #22: This is an example of academic analytics. This screen shot is an example of predicting learner graduation rate, based on how they are progressing in a course. What does this look like in VPs?
• #24: The most impactful is making student learning visible. This means, giving the learners an explanation for their answer, key feedback, and the opportunity to practice again.
• #25: Could we predict based on a student’s early progress in the case, based on certain choices, a DDx list, answers to any questions …on how they will do in the rest of the case? Other similar cases? Other different cases? A Learning analytics mindset asks: What are the elements of VPs that you, the MedU community, agree are important features to get right? Do those variables play out as the predictors? Education data mining mindset asks: What data do predict performance within a case, across cases? Do these align with what we know is valuable in clinical reasoning with real patients?
• #27: 40 million nudges a week in 2017?
• #28: The goal is to encourage positive behaviors. We don’t want to shove (push) or slap (cut off) the learner.
• #29: Coaching is the approach that we know your performance – how can we get you to mastery, excellence – or merely on the path you want to be on?
• #30: What about their sequence of elements they got correct & incorrect in the VP predicts performance on future cases?
• #31: What would you add across a series of virtual patients to make learning transparent? e.g. feedback prompts, outcome measures, reporting to learner, reporting to faculty)
• #32: 45 mins
• #34: MedU, however, is primed in the medical education space to provide analysis across institutions, and within insitutions. Across cases, you have the power to analyze learner progress & feedback.
• #36: The signals system sat on top of data feeds from Blackboard. Data signals then sent warnings in 3 colors – green, yellow, red – to learners. Faculty would also compose emails with standard messages, and the emails would send to learners based on their signal color. This approach had a very positive impact on Purdue. Due to Blackboard making changes to their system, Signals will no longer exist after Dec 2017. Purdue recently purchased Kaplan’s eLearning division.
• #38: When doing my PhD, watching learners do the VPs in groups, it was clear that VPs were contagious. MedU doesn’t have control over how the VPs are implemented. Some schools will have learners in small groups. Most will have learners do these independently. But in 2017 – are the learners truly independent? What other tools are they using while doing MedU VPs? Can the coaching tips extend to what resources the learners might use? Which peers to consult? How to critically appriase other resources? OR – said another way – in what ways does bringing in
• #43: Aquifer has key elements: Smart people who ‘get’ med ed Fantastic thinkers who can integrate evidence-based learning principles Large, cross-institutional, longitudinal data set
• #45: End with 10 mins for Q&A