Initial consultation event: Timing and publication
Download as PPTX, PDF0 likes553 views
The document outlines a feedback survey for postgraduate taught (PGT) students, detailing aspects such as survey coverage, timing, data linking, and publication. It emphasizes the importance of a multi-point survey approach to capture representative data and discusses methods for publication and analysis of the survey results. Key considerations include ensuring meaningful breakdowns of responses and establishing benchmarks for comparison with existing performance indicators.
Initial consultation event: Timing and publication
- 1. A feedback survey for taught postgraduates Workshop 2: Timing and publication Catherine Cameron Senior HE Policy Adviser, HEFCE Richard Puttock Head of Data and Management Information, HEFCE Future Inn, Bristol City Centre 21 September 2017
- 2. Session Outline 1. Survey coverage (10 mins) 2. Survey timing (20 mins) 3. Data linking (10 mins) 4. Publication levels (20 mins)
- 3. Survey coverage Initial starting point was whether there were any groups of PGT students who should be excluded Our proposal All students on PGT courses which lead to a qualification Excepting those on integrated Masters Questions or comments?
- 4. Survey timing Key consideration is to provide representative and meaningful information Options • Single survey window • Post completion survey • Multi-point survey • Rolling survey
- 5. Survey timing End date analysis Whilst June/July and September/October cover most students, there are differences for: • Particular providers • Some students on unstructured courses • Some students studying particular subjects End month Percentage of students Jan 3% Feb 2% Mar 3% Apr 2% May 4% Jun 12% Jul 16% Aug 5% Sep 34% Oct 10% Nov 6% Dec 3%
- 6. Survey timing Conclusion • A multi-point or rolling survey is the only way to achieve responses which reflect the majority of a students’ academic experience and a sufficiently high response rate. • Running the survey at four points in the year should be sufficient to capture a significant majority of the PGT population close to the end of their period of study.
- 7. Survey timing - questions Your questions …. Our questions a. What are the challenges for institutions in delivering a multi-point survey? b. Is there anything we haven’t considered that we should have?
- 8. Data linking • In order to be able to analyse and represent survey responses by student characteristics we must be able to link back to individualised student data • This will also allow us to link to other datasets: NSS, Graduate Outcomes for analysis • Data futures – will introduce in-year data collection and potentially allow us to generate target lists Questions or comments?
- 9. Publication • Intend to consult on some elements of publication approach now and some following successful piloting • The key things that we think that we will need to consider are: • Publication routes • Publication thresholds • Publication level • Breakdown of responses • Benchmarking of responses • Detailed approach to presentation of information
- 10. Publication Publication routes Envisage publishing outcomes in similar way to NSS: • Publishing responses on the Office for Students (OfS) website • Publishing on the central information source to support decision-making being provided by the Office for Students and the other funding bodies (assuming continuation) • Publishing an open dataset so that other information providers can make use of this information • Making responses available to providers through a secure system.
- 11. Publication Publication thresholds Propose to adopt same thresholds as used for publication of NSS • At least 10 students • 50 per cent of sample Question Our analysis is based on these thresholds and we propose not to consult on them. Are there any concerns about this?
- 12. Publication Potential coverage However Expected gaps for some particular types of provision, particularly at FECs, due to very small cohort sizes Publication level Estimated coverage (70% response rate) Estimated number of students included Course 19% 67% HECoS CAH level 3 57% 94% HECoS CAH level 2 70% 98%
- 13. Publication Breakdown of responses We also need to consider the course characteristics by which the survey responses should be broken down to best meet its purposes For example, mode of study. But …. we must be careful not to fragment to an extent where we cannot publish data Questions • Are there ways in which is would be essential to break down the data to make it meaningful? • Are there ways in which it would be desirable to break it down?
- 14. Publication Benchmarking • We will consider benchmarking as part of the second stage of developing proposals. • HEFCE is currently leading a fundamental review of the approach to benchmarking used in the UK HE Performance Indicators and the TEF to ensure it meets best practice. • This will influence our approach to benchmarking of survey outcomes.
- 16. How to find out more website www.hefce.ac.uk/lt/PGT/