![48
S1: We’ve had some modules where they've told us what would be coming up and we
would go “ok, I’ll learn these five topics and not learn the other six.” … exams take a lot of
hard work anyways so why would you waste energy on a part that you know is not going to
get you marks.
S2: When you’ve got 5 exams, you don't really think whether what you are doing is going to
benefit your future.
S3: I think when it comes to course work, we have the freedom to choose which lecture we
want to focus on, so we make the choice and study it in detail.
I) Do you feel that you would lose out on anything if you left a topic altogether?
S1: Not really, because we hardly learn anything we learnt at university. It’s more of
maturity that we learn at university rather than the detailed skills.
S2: Team work, you’re writing skills and stuff like that. It’s more general knowledge.
S3: It depends on what field you will go and work in. For example, I was working in a
consultancy firm and over there, it helped to know more because it had a lot to do with
finance and I had done the finance module. So I thought to myself, “… only if I had listened
some more!”. So maybe if we get a bit more specialized and learn a bit more in our area of
interest, we will think “we should have paid more attention in our coursework”.
S2: It depends on practicality. Finance is about numbers and using Excel but in subjects
like marketing, I don’t think I used any marketing theories at all. What would have helped
me if I had done a module where I had put together a marketing plan rather than being
assessed on marketing theory. That would have been more transferable in terms of what I
did in my placement.
S1: We had one [module] this semester [Business Planning] which was very practical. We
still had to do research on how we would have to do the thing [practical work] but it didn’t
have big theories. There were theories that were useful, but it was all about applying them.
S3: We learnt by doing it.
I) Was the guidance given by lecturers consistent across the programme?
S1: Some lecturers are very by the book and some give you the topics. Others say that
they’d rather have reports from consultancies rather than articles and research.
S2: Some say [in assessments] they want names and reference, other say that they want
the name, reference and date. So it is very very inconsistent.
I) Do you feel the need for consistency?
S1: I think consistency in things like referencing, things like how we go about structuring an
essay because it quite be quite difficult. I think our International HR module is the hardest,
because their way of structuring the essay, I have never seen in my entire life. It was
completely backwards to the way we usually do it. You read a paper and do you
coursework on that, in other modules, you look at you lecture notes and support your
coursework.](https://image.slidesharecdn.com/05afc849-7898-4182-8b66-19eea753eeb6-161216051313/85/IB-TESTA-Report3-48-320.jpg)
![49
S2: The difficulty was because some of the articles were really complex to read. So we
didn’t really get what it was trying to say. So if we understood a tiny bit wrong, then it was
so clear on the exam.
S3: It was challenging!
I) What assessment related factors motivated you to take your optional modules?
S1: Balancing coursework and exams.
S2: When you do placement year, you come back and think that you know what you can
and cannot do. Having done 1st year, you know what modules you get bad marks in. So I
am not going to risk it going into my final year. For instance, Maths is not really my strong
point, so I am not going to risk doing a Math module. On the other hand, I’d prefer a
Marketing module because I always think, “What do I enjoy? What am I good at?”
S3: I like doing presentations. So I’d pick modules that have more presentations. Though
there are not many who do presentations, so I guess that could be improved.
I) Does the length of the lecture slides or material given make a difference in the modules
you would choose?
S1: Potentially, yes. I think I’d prefer the more because then we’d have more information.
S2: This is more of a grey area. I am a bit lazy, so if the lecturer gives all the slides and
notes, I would have to make less. Coming into the final year, what I realized is that they are
not looking for you to rely on lecture notes because they don’t want you rely.
S3: I think that when there is a bit less, you have the chance to read around the subject and
write from what you learn. I felt that I was stronger in those exams.
S1: Sometimes I feel that if they give you loads of notes, lecturers will feel that there is
replication because that’s what they’ve given you.
I) Something that I picked up from the questionnaires that we distributed: would you like to
have exemplars?
S1: Yeah, it would be great if they could do that, Anyways they are not going to have the
same questions that they gave the previous years, so it’s not like you are going to copy it
straight out. So it would be nice to have exemplars.
I) Do you feel like you learnt anything over the course of your programme?
S1: Yeah.
S2: Yeah.
S3: Yeah.
S1: I remember what I did in my coursework [over the course of the programme]. Group
work is also something I would remember because it was an extra challenge. A lot of
people don’t like them because someone’s …
S2: lacking](https://image.slidesharecdn.com/05afc849-7898-4182-8b66-19eea753eeb6-161216051313/85/IB-TESTA-Report3-49-320.jpg)
IB TESTA Report3
- 1. 1 International Business Programme Audit TESTA Transforming the Experience of Students through Assessment By The Centre for Academic Practice Dr Hardeep Basra Rounaq Nayak Shilpa Khobragarde
- 2. 2 Table of Contents 1. TESTA background 3 2. Key findings 4 3. Programme audit 5 4. Commentary on programme audit 16 5. AEQ results 17 6. Focus group analysis 20 7. Recommendations 25 References 28 Appendices: Appendix A: Copy of AEQ 30 Appendix B: AEQ Results – Individual question breakdown 32 Appendix C: Open Text Comments on AEQs 42 Appendix D: Transcription of focus group 44
- 3. 3 1. TESTA background TESTA is a methodology designed to address assessment and feedback issues at the programme-level. It is built on a robust, triangulated research methodology with qualitative and quantitative elements, and underpinned by educational principles and research literature. TESTA has been implemented on over 100 programmes across 40 national and international universities. The main aim of TESTA is to enhance the student learning experience from assessment by providing evidence to programme teams about assessment and feedback patterns and helping teams identify ways of improving assessment design in the interests of better learning outcomes. TESTA uses three methods to gather evidence about the typical assessment experience of students: Programme audit (evidence from documents and team) Assessment Experience Questionnaire (based on established principles) Focus groups with students The aim is to collate information that provide an overview of assessment - the quantity of assessment, balance of formative and summative, variety, distribution of assessment and its impact on student effort, feedback practices, the clarity of goals and standards and the relationship between these factors and students’ overall perception of their degree. This case study summarises the evidence derived from these three methods and is for use and discussion by the programme team. You can find out more about TESTA on www.testa.ac.uk. The TESTA project is led by the Centre for Academic Practice by Dr Hardeep Basra but she has been supported in the project by a number of postgraduate research assistants. If you have any questions or queries about the project, please e-mail h.k.basra@lboro.ac.uk.
- 4. 4 2. Key findings There is an issue with assessment bunching. The amount of assessment and timing of submission could be having an impact on the quality of submitted work. Students choose their modules on the basis of the assessment type offered. Students generally hold the notion that being able to memorise and regurgitate important facts was essential for Multiple Choice Exams and also felt this type of assessment did not promote a deep understanding. Students indicated that they would like to see the inclusion of more oral assessments, such as presentations in their overall assessment experience. Students stated their preference for ongoing assessments as this enables them to keep on track with their studies by building upon previous knowledge and continuously assessing their level of understanding. Students noted stronger presence of on-going assessment in Part A with decline in this practice in Part B and C. Students are not clear on course expectations generally and assessment requirements (goals and standards) although this is dependent on the module choice. Formative assessments could to be incorporated (as suggested by the students) to enable them to better identify what is needed from them in every module through a practical rehearsal of the actual work they will have to do in order to successfully complete it. Students felt that marking schemes need to be reviewed, with the aim of reducing generic marking schemes for the adoption of more module specific marking schemes. The distribution of written samples is proposed as an effective way of distinguishing between poor and good quality work. Students reported they had no issues with the current feedback time and in fact they were willing to wait a bit longer in order to receive more quality (i.e. individualized) feedback as opposed to generic feedback. However, the time span between receiving feedback and the submission of the next assessment needs to be reviewed so that students have enough time to incorporate changes proposed to them (i.e. feedforward). Students would like feedback to be more constructive and developmental.
- 5. 5 3. Programme audit 3.1 Defining the Cohort ‘International Business’ which is an undergraduate Business course of School of Business and Economics was audited on 10 Feb 2016. The duration of the programme is six semesters, plus one academic year, and has four streams. Candidates following the ‘Placement Stream’ are required to spend the third academic year undertaking professional training leading to the Diploma in Professional Studies in accordance with Senate Regulation XI. Candidates following the ‘Study Abroad Stream’ or the ‘Language Related Placement Stream’ are required to spend the third academic year at an approved academic institution. Candidates following the ‘Split Stream’ are required to spend six months of the third academic year undertaking professional training, and the other half of the third academic year studying at an approved academic institution. The Study Abroad, the Language Related Placement and the Split Stream lead to the Diploma in International Studies (DIntS) in accordance with Senate Regulation XI. The third academic year (Part I) occurs between Part B and Part C. There are 118 students in Part A, 98 students in Part B, 67 students in Part C and 79 students in Part I. The entry tariff is usually AAB/ABB. But General Studies, Critical Thinking and Communication and Culture are not normally accepted. For students those who have studied SQA AH (Scottish Qualifications Authority Advanced Highers) from Scotland, the entry tariff is AB Advanced Highers plus AAB Highers. The entry tariff is 34 points minimum for students that have done IB (International Baccalaureate Diploma). For others, BTEC Level 3 Extended Diploma: DDD profile in relevant subjects is required. Emphasis is also given to additional criteria of having strong GCSE profile including minimum grade B in Maths and English Language (or equivalent). Approximately 49% of each cohort is female (please see Table 3.1 for Student Composition). There are approximately 46 members of staff (Part A -15, Part B – 16 and Part C – 15) who teach within the International Business programme. Table 3.1: Student Composition International Business Gender Total Male Female Part A 60 58 118 Part B 54 44 98 Part C 38 29 67 Part I 33 46 79 TOTAL 185 177 362
- 6. 6 There are two semesters per year and for every semester there are some compulsory modules and some elective modules. For Part A students, there are Introductory modules, out of which Module BSA007 Skills for Study, Placement and Employment is compulsory for both Semesters 1 and 2 with 20 credits. There are 4 x 10 compulsory credit modules for each semester. There are however, 4 x 10 elective credit modules for Semester 1 and 5 x 10 credit modules for Semester 2, out of which the students have to choose only one for each semester. For Part B students (Degree modules), there are 5 x 10 compulsory credit modules for each semester. There are however, 4 x 10 elective credit modules for Semester 1 and 8 x 10 credit modules for Semester 2, out of which the students have to choose only one for each semester. Students may replace the modules required for Part B Semester 2 with an approved course of study taught in English at a foreign University. They have to then undertake assessed work equivalent to 60 credits, as required by the School of Business and Economics. Students who select this option must ensure that they have taken a total of 60 credits in Semester One. For Part C students (Degree modules), there are 3 x 10 compulsory credit modules for Semester 1. There are 1 x 10 and 1 x 20 compulsory credit modules for Semester 2. There are however, 19 x 10 elective credit modules for Semester 1 and 25 x 10 credit modules for Semester 2, out of which the students have to choose any three for each semester. Please see Tables 3.2 (a) and (b) for Typical Programme Composition. Table 3.2 (a): Typical Programme Composition Year Module Part A 20 10 10 10 10 10 10 10 10 10 10 Part B 10 10 10 10 10 10 10 10 10 10 10 10 Part C 10 10 10 10 20 10 10 10 10 10 10 Table 3.2 (b): Typical Programme Composition Year Module A 20 15BSA007 - Skills for Study, Placement and Employment 10 15BSA005 - Organisational Behaviour 10 15BSA055 - Principles of Marketing 10 15BSA070 - Financial Framework 10 15BSA085 - Quantitative Methods for Business B 10 15BSA005 - Organisations in the International Context 10 15BSA057 - The Marketing Mix 10 15BSA060 - Micro- economics for Business 10 15BSA115 - Business Modelling B 10 15BSA050 – Introduction to Management 10 15BSA026 – Principles of Law B 10 15BSB010 - Business Information Management 10 15BSB023 - Macro- economics for Business 10 15BSB040 - Human Resource Management 10 15BSB050 - International Business 10 15BSB137 - Data Analysis for Marketing Decisions 10 15BSB024 - International Economics 10 15BSB052 - International Management 10 15BSB070 - Accounting for Management Decisions 10 15BSB086 - International Employment Relations 10 15BSB105 - Brand Management 10 15BSB035 – Sales Management 10 15BSB135 – Consumer Behaviour C 10 15BSC051 - International Corporate Governance and Firms. 10 15BSC105 - International Human Resource Management 10 15BSC115 - International Marketing 10 15BSC053 - International Negotiations 20 15BSC055 - Strategic Management 10 15BSC124 – Marketing Communcation 10 15BSC132 – Small Business Issues 10 15BSC100 – Advanced Interpersonal Skills 10 15BSC085 – The Changing Work Organisation 10 15BSC070 – Technology in Business and Society 10 15BSC134 – Business Planning for New Ventures
- 7. 7 3.2. Assessment on the Sport and Exercise Science Programme The audit shows the following patterns of assessment across the whole programme: Variable “International Business” programme Total number of assessments1 57 Number of summative tasks 47 Number of formative tasks 10 Varieties of assessments 10 Average Assessment per module 3.46 Proportion of exams2 55.55% (25/45) Proportion of degree weighting on exam 64.02% Proportion of group assignments 13.33% (6/45) Time to return3 21 days Volume of written feedback 9231.34 words (101.44 per script) Volume of oral feedback4 Unspecified due to large proportion of informal feedback 3.3. Comparison with 23 TESTA Undergraduate Degrees A comparison of the undergraduate International Business programme audit compared with audit data from 23 TESTA programmes in eight UK Universities shows some distinctive features of the International Business programs. Variable “International Business” programme Testa Bachelors (Average) Total assessments 57 51 Number of summative 47 38 Number of formative 10 14 Varieties of assessments 10 13 Proportion of exams 55.55% 15% Time to return 21 days 21 days Volume of written feedback 9231.34 words (101 per script) 5808 words (153 per script) The International Business programme has total assessments almost twice that of TESTA sample. Also, there is slightly higher number of summative assessments compared to the TESTA sample mean. The numbers of formative assessments are about five times the TESTA sample. The average return time of assessment and feedback is same. It is quite difficult to compare the amount of oral feedback given due to the informal nature of office hours and contact time with students. 1 The assessment total is based on the information available on LEARN page of Loughborough University. 2 The information here is based on summative assessments only. 3 Average return time of assessment w as calculated taking average of the return times across the 3 years. 4 Oral feedbackw asdifficult to quantify due to the open door policy of lecturing staff and the close contact w ith students during lab sessions increasing the opportunity for staff to offer informal feedback.
- 8. 8 3.4. Summary Table Showing Assessment Breakdown by Year 3.4.1. Summary Table Showing Assessment Breakdown by Year The table below illustrates the assessment breakdown for the typical “International Business” student. This provides information on the varieties of assessment, summative assignments, formative assignments, written feedback and return times of feedback. Year Typesof Assessment Varieties Summative Formative Average Assessment Exams Proportionof Weightingon Exam Written Feedback ReturnTimes A Exam, Excel Assessment, Individual written coursework or report, Coursework test, Group coursework or report, Group presentation, Workshops,Online quiz 8 19 4 6.09 8/18 60.83% 92.0 23 days B Exam, Tutorial presentation,Group coursework or report, Group presentation. 4 15 5 1.83 10/15 75% 101.4 21 days C Exam, Essay, Individual written coursework or report, Group coursework or report, Group presentation. 5 13 1 2.45 7/12 56.25% 93.6 19 days Total 47 10 3.465 25/45 64.02%6 2965.77 21 days8 5 Average assessment per module w as calculated by taking total number of assessment and then dividing it by number of modules across the 3 years. 6 Exam w eightings for each module w ere totalled and then divided by 12 for each year to get an average exam w eighting. The average for each year w ere added together and divided by three for an average for the programme. 7 The average for each year is added up (92.0 + 101.4 + 93.6) divided by 3 to get an average across the years (95.67); multiplied by number of tasks eliciting feedback (summative + formative – exams) = (56-25) x 95.67 = 2965.7. 8 Average return time of assessment w as calculated taking average of the return times across the 3 years.
- 9. 9 3.4.2. Assessment Per Module for the typical “International Business” student The table below shows the assessment for each module that a student in the International Business programme would take; this is based on summative assessment only. Information on assessment weighting is also provided. Part A Type of Assessment (% assessment weighting) Total Coursework In-Class Test Exams Module Code Credit Group Presentation Group Report Tutorial Presentation Individual Coursework/ Report Excel test Course- work test Exams BSA007 20 X (10) X (30) X (40) X (20) 4 BSA005 10 X (100) 1 BSA055 10 X (100) 1 BSA070 10 X (100) 1 BSA085 10 X (20) X (5) X (75) 3 BSA053 10 X (100) 1 BSA057 10 X (100) 1 BSA060 10 X (100) 1 BSA115 10 X (20) X (5) X (75) 3 BSA050 10 X (100) 1 BSA026 10 X (20) X (80) 2 TOTAL 120 1 3 2 3 1 1 8 19
- 10. 10 Part B Type of Assessment (% assessment weighting) Total Coursework In-Class Test Exams Module Code Credit Group Presentation Group Report Tutorial Presentation Individual Coursework/ Report Excel test Course- work test Exams BSB010 10 X (50) X (50) 2 BSB023 10 X (100) 1 BSB040 10 X (100) 1 BSB050 10 X (100) 1 BSB137 10 X (25) X (25) X (50) 3 BSB024 10 X (100) 1 BSB052 10 X (100) 1 BSB070 10 X (100) 1 BSB086 10 X (100) 1 BSB105 10 X (100) 1 BSB035 10 X (100) 1 BSB135 10 X (100) 1 TOTAL 120 1 2 1 1 0 0 10 15 Part C Type of Assessment (% assessment weighting) Total Coursework In-Class Test Exams Module Code Credit Group Presentation Group Report Tutorial Presentation Individual Coursework/ Report Excel test Course- work test Exams BSC051 10 X (100) 1 BSC105 10 X (100) 1 BSC115 10 X (100) 1 BSC053 10 X (100) 1 BSC055 20 X (25) X (75) 2 BSC070 10 X (100) 1 BSC100 10 X (100) 1 BSC132 10 X (100) 1 BSC085 10 X (100) 1 BSC124 10 X (100) 1 BSC134 10 X (25) X (75) 2 TOTAL 120 1 1 0 4 0 0 7 13
- 11. 11 3.4.3. Contact & Self-Guided Timelines The tables below show the contact and self-guided timelines expected of students throughout their degree program. This information was gathered from the specific Module cards of the International Business programme modules, that are available on LEARN. Contact time may be defined as a time period where there is contact between the students and the lecturer. For example this may involve the lecturer giving instruction to students during a seminar or tutorial or a laboratory session. Furthermore, contact time may also take a virtual from either via email, virtual learning environments, such as using LEARN, face-to-face feedback, peer learning and many others. Additionally, differentiations in contact time also occur on the basis of the person providing the contact. Overall the contact time outlines includes the form of the contact time, whether this is scheduled or not, the person providing the contact and the intended outcome of this contact. Self-guided work may be defined as work which students have to do on their own. This is to aid in their understanding and learning development process.
- 12. 12 Year A Duration Module Code Credit Expected time on assessment Self-guided work (hrs) Contact time (hrs) BSA007 20 a. Practical classes and workshops - 10 hrs 156 44 b. Tutorial (1 hr once/wk) - 10 hrs c. Lecture - 24 hrs BSA005 10 a. Lectures (2 hrs/wk) - 23 hrs 73 27 b. Workshop (1 hr, once/fortnight) - 4 hrs BSA055 10 a. Tutorial x3 (1 hr every 3 wks) - 3 hrs 75 25 b. Lectures (1 hr/wk) - 22 hrs c. Discussion classes (1 hr/wk) BSA070 10 a. Lectures (2 hrs/wk) - 22 hrs 74 26 b. Tutorial x4 (1 hr once/fortnight) - 4 hrs BSA085 10 a. Lectures (2 hrs/wk) - 24 hrs 71 29 b. Tutorial x5 (1 hr once/fortnight) - 5 hrs BSA053 10 a. Lectures (2 hrs/wk) - 20 hrs 76 24 b. Practical classes and workshops - 4 hrs BSA057 10 a. Lectures (2 hrs/wk) - 22 hrs 75 25 b. Tutorial x3 (1 hr every 3 wks) - 3 hrs BSA060 10 a. Lectures (2 hrs/wk) - 20 hrs 70 30 b. Tutorial x10 (1 hr/wk) - 10 hrs BSA115 10 a. Lectures (2 hrs/wk) - 24 hrs 71 29 b. Tutorial x5 (1 hr once/fortnight) - 5 hrs BSA050 10 a. Lectures (2 hrs/wk) - 22 hrs 76 24 b. Workshop x2 (1 hr, once/month) - 2 hrs BSA026 10 a. Lectures (2 hrs/wk) - 24 hrs 72 28 b. Tutorial x4 (1 hr once/fortnight) - 4 hrs TOTAL 120 889 311 Year B Duration Module Code Credit Expected time on assessment Self-guided work (hrs) Contact time (hrs) BSB010 10 a. Lectures (2 hrs/wk) - 24 hrs 72 28 b. Practical classes and workshops - 4 hrs BSB023 10 a. Lectures and feedback (2 hrs/wk) - 16 hrs 80 20 b. Practical classes and workshops - 4 hrs BSB040 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 b. Individual and group exercises BSB050 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSB137 10 a. Lectures and seminars (2 hrs/wk) - 24 hrs 70 30 b. Practical classes and workshops - 2 hrs c. Tutorials - 4 hrs BSB024 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSB052 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSB070 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSB086 10 a. Lectures including revision (2 hrs/wk) - 22 hrs 78 22 BSB105 10 a. Lectures (1 hr/wk) - 22 hrs 78 22 b. Case-based/student-centered activities (1
- 13. 13 hr/wk) BSB035 10 a. Lectures (1 hr/wk) - 22 hrs 78 22 b. Seminars (1 hr/wk) BSB135 10 a. Lectures (1 hr/wk) - 22 hrs 78 22 TOTAL 120 924 276 Year C Duration Module Code Credit Expected time on assessment Self-guided work (hrs) Contact time (hrs) BSC051 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSC105 10 a. Lectures (2 hrs/wk) - 24 hrs 76 24 b. Tutorials and individual & group exercises BSC115 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSC053 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 b. Participative class exercises BSC055 20 a. Lectures (4 hrs/wk) - 38 hrs 158 42 b. Practical classes and workshops - 4 hrs BSC070 10 a. Lectures (2 hrs/wk) - 20 hrs 80 20 BSC100 10 a. Lectures (2 hrs/wk) - 22 hrs 77 23 b. Tutorial x1 Each studentreceives at least one 30 minute one-on-one tutorial BSC132 10 a. Lectures (2 hrs/wk) - 24 hrs 76 24 BSC085 10 a. Lectures (2 hrs/wk) - 20 hrs 80 20 BSC124 10 a. Lectures (2 hrs/wk) - 22 hrs 78 22 BSC134 10 a. Lectures - 10 hrs 76 24 b. Practical classes and workshops - 14 hrs TOTAL 120 935 265
- 14. 14 3.4.4. Assessment Timelines for “International Business Programme" students Table 3.9:
- 15. 15
- 16. 16
- 17. 17 4. Commentary on Programme Audit Students in this programme undertake more formative assessments than summative assessments as seen in the programme summary above. A higher amount of formative assessments could probably aids the students to actually learning the module rather than focus only on marks to be obtained. Additionally, the high amount of formative assessment seems to suggest that there are ample of opportunities for students to practice tasks and receive feedback to clarify their understanding of the material taught. Students in the International Business programme experience about 10 different varieties of assessment. This is slightly less than as much as the TESTA average. The proportion of exams is 55.5%, which is almost thrice than the TESTA average of 15%. The percentage proportion weighting of exams for students are 60.83%, 75% and 56.25% for year A, B and C respectively. This indicates that the students are mostly assessed through exams and there is little coursework assessment. There is need to have balance between assessment through exams and coursework, so that it relieves the pressure on the students during exam days. The amount of oral feedback was difficult to quantify mostly because this type of feedback is informal; and students seek additional feedback when they want. The average time period within which students in the programme receive marks and feedback is 21 days. The total amount of written feedback given is higher in comparison to the TESTA sample. The audit data seems to pinpoint that marks and feedback received by students would help them to understand the goals and standards of assessed tasks.
- 18. 18 5. AEQ Results 71 students from the International Business Programme completed the Assessment Experience Questionnaire (AEQ) (see Appendix 1). The AEQ contains 9 scales with 2 to 4 related questions each one and one overall satisfaction score. The scales relate to conditions of assessment which promote student learning. Students answer on a 5-point scale where: 1 = strongly disagree, 2 = disagree, 3= neutral, 4 = agree and 5 = strongly agree. A good score is = or >4, aside from the Surface Approach scale, in which case a good score is = or <2. The table below shows the International Business programme mean scores compared to the TESTA means where (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) Table 5.1: International Business mean sore compared to TESTA mean Scales Quantityof effort Coverageof syllabus Quantity& qualityof feedback Useof feedback Appropriate assessment Cleargoals& standards Surface approach Deep approach Learningform the examination Overall satisfaction TESTA (n=23) 3.69 2.93 3.30 3.73 3.75 3.29 3.30 3.82 3.08 3.86 International Business 3.51 3.29 3.29 3.30 2.85 3.12 3.69 3.60 3.62 3.77 The International Business programme mean scores range between 2.85 and 3.77, which indicates that, in general, students are neutral concerning their experience with assessment in their course. Students felt that they had to give equal weightage to memorizing certain details and understanding the topics further in order to successfully complete the programme. For example, Deep Approach gets the higher mean score (3.60) and at the same time Surface Approach to learning also seems to have been adopted by the students (3.69, where a good score is considered to be = or <2). A low score was given to appropriateness of assessment (2.85) and a higher score to learning from examinations (3.62), which indicates that students felt that lecturers did not solely rely on judging students’ ability to memorize in assessments, but gave importance to them understanding the topic at a deeper level. They felt that their ability to memorize important topics was assessed in the exams, and that a combination of assessment and exams would help them in the real world. This might have led to students adopting the two approaches in order to complete the programme with a good grade.
- 19. 19 As far as the rest of the scores are concerned, International Business students indicated that the assessment pattern can ensure they distribute effort across the module as much as it can require them to cramp through the end of the module (Quantity of effort = 3.51). Students stated that they are able to be selective about the topics they study (Coverage of syllabus = 3.29) and were not impressed the quantity and quality of feedback they received (3.29). They also did not seem to find the feedback received useful (Use of feedback = 3.30). Furthermore, students seem to be quite unclear about the goals and standards that they have to fulfil in order to achieve a good grade (Clear goals & standards = 3.12). This was mainly because students did not find much input from lecturers in their feedbacks and also felt that lecturers should provide them with what they expect of students at the beginning of the modules with the programme. All in all, students’ overall satisfaction with their course appears to be neutral (due to issues with feedback time and quality) with a tendency to higher satisfaction (Overall satisfaction = 3.77). Correlation analysis demonstrates statistical relationship between one scale and another. When using the Spearman’s rho tests, there are 19 strong correlations and 4 moderate correlations in the data. Correlations are not causal but they do demonstrate a relationship between one scale and another. The most important strong correlations observed are between: Quantity and quality of feedback and the Use of feedback (r=0.243, p<0.01), which translates into: the better the quantity and the quality of feedback is, the more students use it). Quantity and quality of feedback and Appropriate assessment (r=0.364, p<0.01), which translates into: the better the feedback is, the more assessment helps students to learn for understanding). Quantity and quality of feedback and Clear goals and standards (r=0.451, p<0.01), which translates into: the better the feedback, the clearer students are about goals and standards). Quantity and quality of feedback and Surface approach (r=0.299, p<0.01), which translates into: the better the feedback, the less students take a surface approach to learning). Appropriate assessment and Surface approach (r=0.0446, p<0.01), which translates into: the more assessments test understanding, the less students take a surface approach to learning). Clear goals and standards and Overall satisfaction (r=0.310, p<0.01), which translates into: the clearer students are about goals and standards, the more satisfied they are). In all the TESTA data, there is a golden thread between the quantity and quality of feedback, clear goals and standards and overall satisfaction. Where feedback works well, students are clearer about goals and standards and are more satisfied with the whole course experience.
- 20. 20 Bar charts providing a breakdown of each individual question asked on the AEQ can be found in Appendix B. These provide additional insight into the different conditions under which assessment supports learning. The focus group data will also provide further texture and explanation for these scores.
- 21. 21 6. Focus Group Analysis The focus group discussion was conducted on May 19th 2015 and lasted for 37 minutes. The session was facilitated by one of the Research Assistants of the TESTA program. Three final year students (3 females) from the International Business programme volunteered to participate in the discussion. All of them gave their consent to be recorded and for their answers to be used for the purpose of the focus group discussion analysis. In order to protect the anonymity of students (and staff) names have been removed. Proximity to exams was cited as the reason behind more students not volunteering for the focus group study. The discussion focused on the students’ overall experience with assessment in their studies. The following section presents the thematic analysis of the focus group discussion, presenting the main points risen by the students. 6.1. Exams vs. Coursework Students recognised two to three forms of assessments on their programme (exams, coursework and presentations) and felt that there was balance between exams and coursework in their programme. However, they felt that they were not given enough opportunities to give presentations and seminars. Students felt that it would make more sense to develop their presentation skills as they were made to present during their placement years. Moreover, as they progressed in their years of study, they noticed that there is an emphasis for literature review and review of journals. They also noticed an increase in the demand with regards to the quality of work and felt that there was a massive jump in this regard between the 2nd and 3rd year. The fact they the students had a year break (placement year) only made this ‘jump’ seem even bigger. “Obviously, the more you go towards your final year, the stress on literature review and reading journals increases. There seems to be a massive jump in their (lecturers’) demands and evaluation criteria.” (A1) All students agreed that, in general, all of the module leaders are very good at outlining at the start of the module the type of assessment that students will have to undergo and the evaluation criteria. Students felt that when they know how much weightage a coursework carries, they are able to judge how many hours they would put in coursework and how many hours they would put in preparing for exams for that module. “… throughout the programme, we were told how we were to be assessed and how much the coursework was worth. We were also well informed about how much weightage a coursework carried and so were able to decide how much time we would spend on a particular coursework and how much time we would spend studying for an exam (for the same module).” (A1)
- 22. 22 Students chose optional modules based on two criteria: (1) what they thought would benefit them with regards to their careers and (2) only if they felt they would be interested in the topics that would be covered in the module. They did not base their choice on what previous students (who had taken the module before) had told them or on the types of assessment. “I chose modules based on which topics I felt interested me. Types of assessment or length of lecture notes and slides did not have any effect on the option modules I opted for. I thought of the area I wanted to work in once I graduated and selected my optional modules.” (A1) “… if lecturers gave more notes, I would have to go through them, yes, but if they gave lesser notes, I would have had to look for those topics on my own. So I did not think about the quantity of material to study from, but looked at the content of the modules.” (A2) Students felt that exams encouraged surface learning and that they did not contribute in any way to real life situation, such as ones they would encounter in their work-place. They felt that they were asked only theoretical questions in their exams and would be much better off if they were asked to come up with a plan instead (e.g. project plan). They said that it only made sense to have theoretical questions in modules which were related to finance. “… questions asked in exams are focussed more towards the theoretical aspect of the module. Lecturers should understand that we are at university to learn how to carry out our work effectively in the real world. … need more focus on designing project plans.” (A1) Students criticized on of the modules that they had in the final year where they were asked to review and refer to journals for their coursework. They said that they were never taught how to do this and the language used in journals was too complicated for them to understand. Students felt that they needed more guidance from their module leaders in such modules. “In one of our modules, we were asked to refer to journals to for our projects. We found this extremely difficult as the language used was very complicated and we had never done this before.” (A1) “The guidance given was also not very clear and so we were worried that the interpretation we did of the journals was not correct. When we got the grades for this module, most of the students did not fare well as they were totally confused.” (A2) 6.2. Oral Assessment/Seminars All of the students pointed at the lack of oral assessments and their willingness to undertake more of them throughout their studies. They felt that when they would get jobs, they would be asked to present in front of their colleagues. They felt that if they were asked to present in class in front of the friends, they would be able to build up confidence over time.
- 23. 23 “When I worked in my placement year, I was asked to present my work in front of my colleagues. I was nervous as I hadn’t done this before. I wish my lecturers had asked me to present in front of a small group first and then slowly increased the size of the group in subsequent modules.” (A1) “It is important to develop presentation skills as a large part of what we are supposed to learn at university is personal development and communication skills.” (A2) 6.3. Feedback Students felt that lecturers took too long to give them feedback on their work. They were worried about this as they said that they would usually forget about what they had written in their coursework (due to the waiting period) and when they got their feedback, it would hardly have any meaning. The drawback of waiting longer to receive feedback is that it prohibits the opportunity to use feedback to feedforward. Students themselves identified this problem. These students stated they did pay attention to the feedback they received and ideally, would have liked to use it to help improve their work. When asked whether they went back to the lecturers to ask them for more feedback, they said that as in addition to waiting for a long time, they would only get a short generic feedback; they would lose interest in the coursework “We never get our feedbacks on time. Due to this, we can never make improvements as we never get to know what mistakes we have made. By the time we get our feedbacks, we would have forgotten about what the coursework was about.” (A1) “There is no point in going back to them (lecturers) as most of the time they say that they are too busy to evaluate what they have already gone through. So in addition to the waiting, we also get (what seems like) disinterested lecturers; so we just lose interest in that coursework.” (A2) Students would value feedback that offered explanations of where marks were lost and how improvements can be made. They felt that some lecturers gave feedbacks only because they ‘had to’. Students felt lost when it came to understanding why they lost their marks. Students felt that there needed to be some consistency with the quality of the feedback. Students stated feedback was not always constructive and did not allow them to feedforward. They also felt that providing them with bullet points on what their coursework lacked did not make any sense to them as most of the time, they would want examples. “There is absolutely no consistency in the quality of the feedback. Some lecturers give one or two lines in the feedback while some provide us with checkboxes. … We would like some more explanations and some examples on what they expect from us.” (A1) “Most of the time, the feedback we get only has bullet points. We end up not understanding these bullet points because after waiting for a very long time (for the feedback), we tend to forget the details of what we had written.” (A2)
- 24. 24 6.4. Effect of method of assessment on studying Students said that they usually only focussed on those topics that lecturers told them were important. They said that it made no sense for them to study all the topics when they were told that they would be quizzed only on a few topics. They also said that the marks they got on their coursework did not affect the way they studied for the exams. “We only study what the lecturers tell us is important; not all of them do, but some do. This helps us get good grades in exams.” (A1) “It has no effect on the way we study for our exams. If anything, it only motivates us both ways as if we get a low mark (in the coursework), we want to study more to get a good overall grade and if we get a high grade (in the coursework), we want to get a 1st class in that module.” (A2) Participants argued that they were not sure about what they had to do in order to achieve a good grade. They stated that lecturers had different preferences concerning structure and writing style and that the generic marking schemes provided were not helpful. Consequently, two suggestions were made in order to achieve greater understanding of how to get a good grade: a) submitting a first draft, receiving feedback on it and then resubmitting it again and b) providing specific examples of sample text abstracts that correspond to every grade category for every module. “Some lecturers allow us to submit a first draft of the coursework and then give their feedbacks on it; but others say that it would not be fair as it would mean that they would be reading the same student’s work twice, making it unfair for others … .” (A1) “It would be extremely useful if they actually gave us a sample work (exemplar) on what they thought was ‘gold standard’ as this would help us understand what they are expecting from us.” (A2) 6.5. Overall experience Students said that they had an average experience about International Business on the whole. They felt that the best thing about the programme was that they were allowed to go on a placement year and that they were provided a lot of support and guidance right from the first year of their programme. “The fact that we were allowed to go on a placement year was the best as it helped us gain a lot of experience. They encouraged us to put our CV together write from the first year.” (A1) Students felt that as the department adopted a theoretical approach in this programme, they were unable to put together a portfolio of business plans they had developed as a part of their modules. One of the participants mentioned that she had lost out on a job (placement year) as the one the company had selected had a portfolio of all the business plans she had drafted as a part of her modules.
- 25. 25 “A major improvement that needs to be made is that the programme needs to have a practical approach to it. We need to develop plans as a part of our coursework as other universities do that and we tend to lose out on experience and jobs because of this.” (A1) Another issue students had was that they felt that their tutors needed to be a lot more involved than they were at present. They felt that it was the tutor’s role to help them understand how to answer essay questions or how to do a literature review. One of the students also said that in her three years at university, she hardly ever seen her tutor. They felt that they were not sure what their tutors could help them with. “… I have never met my tutor more than thrice. Tutors should be able to meet with us and help us understand what literature reviews are or what exactly the general expectations are of students (with regards to coursework or exam answers).” (A1) “I feel that they are our tutors only because they have to be some students’ tutors every year. They really should be a bit more involved with what we do and not just want to meet us when things get out of hand.” (A2)
- 26. 26 7. Recommendations Please note these are not requirements being imposed upon the course team. These are simply some ideas the programme team may wish to consider to address some of the issues and concerns raised by TESTA research. 7.1 Reconsider the number of assessments through exams The proportion of exams is 55.5%, which is almost thrice than the TESTA average of 15%. Most of the modules of International Business programme have 100% assessment through exams. Also, the percentage proportion weighting of exams are on an average 64.02%. This means that the students are mostly assessed through exams and there is little coursework assessment per module. Students do not associate with studying if coursework is taken away from a module. It is reasoned that there is need to assess everything that moves to keep students engaged. Students also prefer coursework and they consider it to be fairer than exams, as it allows them freedom to express and it also measures a greater range of abilities than exams (Kniveton, 1996). Furthermore, coursework marks are a better predictor of long term learning of course content than are exams. Conway et al. (1992) reported about a study assessing the performance of psychology students on a range of tests of their understanding and recall of content of a cognitive psychology course taken many years before. The authors concluded that the kind of learning that coursework involves has long term consequences while the kind of learning involved in revision for exams does not. There is need to have balance between assessment through exams and coursework, so that it relieves the pressure on the students during exam days and they have better learning experience. 7.2 Using exemplars to engage students with assessment criteria and feedback Exemplars have a potentially valuable role within marking teams and they are also very useful for students. Handley et al. have described about the problem of student (dis)engagement with formative feedback, and argued that engagement could be enhanced by time-shifting feedback so that it comes before final assignments are submitted. One method of achieving this is to use exemplars annotated with feedback. Students like being able to see the structure and layout of a previously marked assignment, and to read the feedback in order to understand what was required, and how the assignment could have been improved. To support both tutors and (indirectly) students, exemplars may be more effective by enabling discussion before the module gets underway. Although students are very receptive to exemplars, there are many questions to be considered by tutors as they design exemplar activities: for example, whether to ‘construct’ exemplar assignments, or use authentic student work; whether to use complete assignments or only those parts which illustrate specific criteria; etc.
- 27. 27 7.3 Follow-up after Part I (placement year) The students take the placement in third academic year (Part I) which occurs between Part B and Part C. Students can get disoriented and find it difficult to adjust to the studying environment of the programme after the placement. There should be some study sessions so that they become accustomed to the academic milieu again and cope with the studies better in Part C. 7.4 Improving peer interaction and classroom environment Many modules are common for different programmes of Business School. As a result, students pursuing different programmes attend the same module. This can affect their participation in the class activities as they many times meet unfamiliar students in the classroom. Fleming et al. have observed that the potential impact of the peer environment multiplies with the increasing multitude of roles a student might play when interacting with others who reside and study in the same setting. The variety of students and the frequency of contact among them—in class, at social events, while walking through residence hallways—provide a level of stimulation. Introducing common social activities for students across different programmes can increase their frequency of contact and interaction, thereby improve the classroom environment. 7.5 Timing of revision In general, for the different modules of this programme the revision sessions are taken in the last week before exams which gives students very less time to assimilate and revise properly. To improve the quality of student learning experience, programmes should consider providing more time between the end of teaching and the examination to allow students to use revision for integration and consolidation rather than only last minute memorization. 7.6. More involvement by personal tutors Students felt that they needed more guidance from their personal tutors. They felt that personal tutors existed only for namesake purpose; a lot of the students had only met their tutors for ten minutes, around three times in the three years they were at university. 7.7. Improvement of quality of feedback The major drawback with this programme was the quality of feedback students received on their coursework. Most lecturers gave students one line, generic comments or bullet point recommendations which they did not find very helpful. Students expected a detailed critical review of their work with suggestions on how to improve their approach and answers. They felt that this would help them better their future coursework and exam answers. Students also mentioned that even if they tried to get more feedback from their lecturers, they would only get a reply two weeks after getting the original feedback. 7.8. Time taken to provide feedback
- 28. 28 Students felt that some lecturers took too long to provide them with feedback on their coursework. This delay, in addition to the poor quality of feedback, would make them lose interest in the feedback provided.
- 29. 29 References Astin, W. A. and Antonio, A.L. (2012) Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education, Rowman and Littlefield. Biggs, J (2003): Aligning Teaching and Assessment to Curriculum Objectives, (Imaginative Curriculum Project, LTSN Generic Centre. Biggs, J.B. (1996) Enhancing teaching through constructive alignment, Higher Education, 32: 1–18. Boud, D & Molloy, E (2013) Rethinking models of feedback for learning: the challenge of design, Assessment & Evaluation in Higher Education,38:6, 698-712. Brown, S (2004), Assessment for Learning, Learning and Teaching in Higher Education,1:81-89. Chan, P.E.; Konrad M.; Gonzalez V.; Peters M.T.; Ressa V.A (2014) The Critical Role of Feedback in Formative Instructional Practices, Intervention in School and Clinic. 50:2, 96 - 101. Clark I. (2010) Formative Assessment: ‘There is nothing so practical as good theory’ Australian Journal of Education. 54:3, 341-352. Gibbs, G. & Dunbar-Goddet, H. (2009) Characterising programme-level assessment environments that support learning. Assessment & Evaluation in Higher Education. 34:4, 481-489. Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning, Learning and Teaching in Higher Education. 1:1, 3-31. Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning, Learning and Teaching in Higher Education. 1:1, 3-31. Harland,T,. McLean, A, Wass, R, Miller, E & Sim, K. N., (2014) An assessment arms race and its fallout: high-stakes grading and the case for slow scholarship, Assessment & Evaluation in Higher Education, 40:4, 528-541. Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112. Jessop, T. & Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study, Studies in Higher Education. Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns, Assessment and Evaluation in Higher Education. 39:1, 73-88. Nicol, D (2013) Peer review: putting feedback processes in students’ hands, Perspectives on Pedagogy and Practice, Journal of the Centre for Higher Education Practice, University of Ulster.
- 30. 30 Rust, C, Price, M & O'Donovan, B (2003): Improving Students’ Learning by Developing their Understanding of Assessment Criteria and Processes, Assessment and Evaluation in Higher Education, 28:2, 147-164 Weaver M.R. (2006) Do students value feedback? Student perceptions of tutors’ written responses, Assessment & Evaluation in Higher Education. 31:3, 379 -394.
- 31. 31 Appendices Appendix A: Assessment Experience Questionnaire (V3.3) By filling out this questionnaire I understand that I am agreeing to participate in a research study Please respond to every statement by circling 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree to indicate the strength of your agreement or disagreement Programme of Study: …………………… ……. Biographical Data: (please fill in the circle as appropriate) Please respond with respect to your experience so far of the programme named above, including all its assessment components 1 I used the feedback I received to go back over what I had done in my work 2 The feedback I received prompted me to go back over material covered in the course 3 I received hardly any feedback on my work 4 You had to study the entire syllabus to do well in the assessment 5 The assessment system made it possible to be quite selective about what parts of courses you studied 6 The way the assessment worked you had to put the hours in regularly every week 7 It was always easy to know the standard of work expected 8 I paid careful attention to feedback on my work and tried to understand what it was saying 9 The teachers made it clear from the start what they expected from students 10 The staff seemed more interested in testing what I had memorised than what I stronglyagree agree neutral disagree stronglydisagree
- 32. 32 understood 11 It was possible to be quite strategic about which topics you could afford not to study 12 It was often hard to discover what was expected of me in this course 13 On this course it was necessary to work consistently hard to meet the assessment requirements 14 Too often the staff asked me questions just about facts 15 I didn’t understand some of the feedback on my work 16 Whatever feedback I received on my work came too late to be useful 17 The way the assessment worked on this course you had to study every topic 18 To do well on this course all you really needed was a good memory These questions are about the way you go about your learning on the course 19 When I’m reading I try to memorise important facts which may come in useful later 20 I usually set out to understand thoroughly the meaning of what I am asked to read 21 I generallyput a lot of effort into trying to understand things which initiallyseem difficult 22 I often found myself questioning things that I heard in classes or read in books 23 I find I have to concentrate on memorising a good deal of what we have to learn 24 Often I found I had to study things without having a chance to really understand them Learning from the exam (only to be completed if there were exams on the course) 25 Doing exams brought things together for me 26 I learnt new things while preparing for the exams 27 I understood things better as a result of the exams Overall satisfaction 28 Overall I was satisfied with the quality of this course Additional Questions
- 33. 33 Appendix B: AEQ Results – Individual Question Breakdown 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree I used the feedback I received to go back over whatI had done in my work 0 5 10 15 20 25 Strongly Agree Agree Neutral Disagree Strongly Disagree The feedback I received prompted me to go back over material covered in the course
- 34. 34 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree I received hardly any feedback on my work 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree You had to study the entire syllabus to do well in the assessment 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree The assessmentsystemmadeit possibleto be quite selective about whatparts of courses you studied
- 35. 35 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree The way the assessmentworked you had to put the hours in regularly every week 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree Itwas always easy to know the standard of work expected 0 5 10 15 20 25 30 35 40 Strongly Agree Agree Neutral Disagree Strongly Disagree I paid carefulattention to feedback on my work and tried to understand whatit was saying
- 36. 36 0 5 10 15 20 25 30 35 1 2 3 4 5 The teachers made it clear fromthe startwhat they expected from students 0 5 10 15 20 25 Strongly Agree Agree Neutral Disagree Strongly Disagree The staff seemed moreinterested in testing what I had memorised than whatI understood 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree Itwas possibleto be quite strategic aboutwhich topics you could afford notto study
- 37. 37 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree Itwas possibleto be quite strategic aboutwhich topics you could afford notto study 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree Itwas often hard to discover whatwas expected of me in this course 0 5 10 15 20 25 30 35 40 Strongly Agree Agree Neutral Disagree Strongly Disagree On this courseit was necessary to work consistently hard to meet the assessment requirements
- 38. 38 0 5 10 15 20 25 30 35 40 45 Strongly Agree Agree Neutral Disagree Strongly Disagree Too often the staff asked me questions justabout facts 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree I didn't understand someof the feedback on my work 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree Whatever feedback I received on my work came too late to be useful
- 39. 39 0 5 10 15 20 25 Strongly Agree Agree Neutral Disagree Strongly Disagree The way the assessmentworked on this course you had to study every topic 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree To do well on this courseall you really needed was a good memory 0 10 20 30 40 50 60 Strongly Agree Agree Neutral Disagree Strongly Disagree When I'mreading I try to memoriseimportant facts which may come in usefullater
- 40. 40 0 5 10 15 20 25 30 35 40 45 Strongly Agree Agree Neutral Disagree Strongly Disagree I usually set out to understand thoroughly the meaning of whatI am asked to read 0 5 10 15 20 25 30 35 40 45 50 Strongly Agree Agree Neutral Disagree Strongly Disagree I generally put a lot of effortinto trying to understand things which initially seem difficult 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree I often found myself questioning things that I heard in classes or read in books
- 41. 41 0 5 10 15 20 25 30 35 40 45 Strongly Agree Agree Neutral Disagree Strongly Disagree I find I haveto concentrateon memorising a good deal of whatwe haveto learn 0 5 10 15 20 25 30 35 40 45 Strongly Agree Agree Neutral Disagree Strongly Disagree Often I found I had to study things without having a chance to really understand them 0 5 10 15 20 25 30 35 Strongly Agree Agree Neutral Disagree Strongly Disagree Doing exams broughtthings together for me
- 42. 42 0 5 10 15 20 25 30 35 40 45 50 Strongly Agree Agree Neutral Disagree Strongly Disagree I learnt new things while preparing for the exams 0 5 10 15 20 25 30 35 40 Strongly Agree Agree Neutral Disagree Strongly Disagree I understood things better as a resultof the exams 0 10 20 30 40 50 60 Strongly Agree Agree Neutral Disagree Strongly Disagree OverallI was satisfied with the quality of this course
- 43. 43 Appendix C: Open Text Comments on AEQs Open Text Comments on AEQs Not enough feedback. Feedback on assignments is given once the module is over so cannot learn from it. New lecturers tend to not be as organized. It would have been useful to provide handouts. The quality of the lecturers should be assessed. A lot are sub-par. External lecturers brought in for final year modules should be screened carefully. Some of them clearly don’t know what they are talking about. Feedback times could be improved (e.g. UCLAN have a 2 week turnaround for feedback). Study skills sessions throughout the years (e.g. How to write a lit review, how to do first level thinking). There was a huge jump from what was expected between the 2nd year and final year and yet it was unclear how this was to be achieved. Example answers of what is a 2:1 or 1st would be helpful. Feedback is often slow. Results were released too late to change options and not be behind. Feedback was often very general. Lecturers should give is the opportunity to have an exam answer marked or give us example answers. Good balance between coursework and exams. Enjoyed the ability to tailor the programme as years went on. Would be useful to include sample exam/coursework extract answers to know what to aim for/avoid. Some lecturers did this and proved very useful. Would be useful to have exam papers back, especially if low marks have been achieved. Useful when real life examples are used in lecturers. Results for Semester 1 were released too late to change modules. More quality lecture recording. Feedback from exams. All lecturers use the standard marking scheme for coursework, this needs to be rethought. Feedback should be ongoing. Better lecture recordings. Lectures till 1pm on Wednesdays are not convenient. Most of us have BUCS games and cannot go to them. Feedback given after assessment handed in, hence, it was mostly useless. Very broad programme with touch points in many aspects. But the feedback system could be better. Currently you do exams but don’t get any feedback of what you could have done better. You have to be very timely in emailing asking for feedback, if it is asked too late, they cant give it. There was a significant lack of feedback that would have been very useful. Especially with exams – for some exam, I still am not entirely sure why I achieved the mark I did. One of my modules was marked too strictly to the point of not being fair. Would have benefitted from more feedback throughout the course. The positive feedback is for some of my modules this year. Some modules have been poorly taught and the content is quite literally read from text books. Lectyrers who do this do no enhance our learning or encourage engagement. Due to this, I found significant disparity between the well taught modules and the poorly taught
- 44. 44 ones. There has been a huge step up between the 1st, 2nd year and 3rd year in terms of reaching quality, although not in all cases. I would have liked to see the course be more focused around practical assessment. Whilst exams help you improve your understanding, it often doesn’t help you in practical skills for the working world (e.g. presentation skills). I think feedback could be more detailed and should happen throughout the term rather than the end. I also think lecturers should talk to each other about what is happening in their lectures so we do not continue to repeat the same topics. No seminars/tutorials. Limited time to ask questions and gain more understanding. Very limited feedback on course works to understand what we were meant to improve. If we had seminars, we could understand in more detail how the lecturer wants us to write the coursework/exam. More choice in 2nd year in terms of how you want to write your degree. DPS report should be worth at least 5% of this year. At the moment it is little motivation for people to do it well.
- 45. 45 Appendix D: Transcription of focus group I) In your module, could you tell me how you are assessed? S1: I’d say a majority of the assessment is through examination. We’ve had various coursework modules and mainly most of them have been individual course works. Maybe some presentations. S2: Presentation sometimes I) Do you know what the weightage was when it came to coursework and exams or presentations? S1: Usually presentations are around 25% and it differs either between 100% coursework or 25% coursework and the rest are exams. S2: Sometimes 50% coursework too I) Is there a preference you guys have? Also, do you prefer a difference in the weightage given at present? S1: I quite like the balance. I think half coursework and half exams is quite nice because it means that you can get a certain set of marks before you go for exams. Sometimes you go straight for exams and that’s all it is, your marks from there and that can be quite stressful if you can't go for an exam on the day, as then that is it. Whereas with coursework you can get people to check it, you can review and go back and speak to lecturers and stuff. I quite like the balance. S2: Yeah. I think exams are very much where most people will cram and will not necessarily retain the knowledge. Whereas if you do coursework or present, you’ll have to know what you are talking about. S3: I agree too. I) Which one do you feel would help you in the real world? S1: Coursework. S2: Yeah S3: The same. I) If the weightage for the coursework was just 5%, would that be okay? S1: That would depend on the word count, if it was less than one thousand or two thousand words, I would be okay but anything more than that and I would be worried because I would rather be doing revision notes and preparing for the exam rather than doing this. S2: 5% sounds way too low for me!
- 46. 46 S3: I think I would quite prefer a presentation if it was 5% because that is something you can make in a couple of days and get it done, whereas a coursework would take a little bit longer. Obviously even if it was 5%, you would want to do pretty well. I) With feedback, are you happy with the feedback you get? S1: No. S2: No. S3: No. I) That's an agreement! So what exactly is it that you don't like about this food? S1: I’d say that it is very limited. I guess the generic feedbacks are quite useful to go over S2: Yeah S1: but your own feedback is very limited. They are usually a few scribbles on a piece of paper, if they’ve written anything at all. S3: Yes, I agree with that. S2: Yes, most of them are ticks in a box, which don't mean anything really. Like if you say “you did good on this”, but it’s never ‘how would you have gotten to excellent’. You get the feedback and it’s usually “oh you did this really well”, but it never says “to have got to a first or to have got to a higher mark, you should have actually…”. The usefulness of that is now very limited. S3: I think in the final year modules, there have been quite a few sessions where there has been reviewing between good and bad. This I think has been quite useful to see because previously, we never really got that much information. There was only 1 module I think that was useful in that, they went over coursework and exam and what was good and what was bad for us. I) Were all the modules the same then when it came to feedbacks? S1: No. S2: No. S3: No S1: I think in some modules you get no feedback at all. I had two modules last semester, one I got no feedback and in the other I got maybe like three lines. If you read it through, you realize that it hasn't helped me. But the generic feedbacks from exams are quite good. On one of my modules they said that some of the answers were excellent because they did some further reading, full stop. That's not helped me. S2: I guess if you want exam feedback, more personal and individual, we’d have to chase for it. It’s not a simple or easy procedure. You have to chase the lecturer who will take two weeks to answer and by that time, it is usually forgotten.
- 47. 47 S3: There was one lecture where we were able to do a mock question and then give it in. that was very useful because they gave us the feedback and we were able to make improvements based on the suggestions. I) So was this before the exam? S2: Yes, they said that you can pick any topic you want from the past paper questions and then just write it up and submit it. S3: I think the inconsistency factor comes in here where lectures call it double marking. Some lecturers would say “you can give me a plan; I’ll review the plan and give it back to you”. On the other hand, some lecturers say “we can't do that because that would mean that I would have seen you work twice”. So the ones where you can, great, but it does get quite confusing because you don't actually know what the lecturer is looking for. You would think that it is the same business school so there would be a lot more consistency. The ones where we could hand in a plan are the modules I feel more confident and much stronger in. I) If you are not happy with the feedback, do you go back to the lecturer? **laughter** S1: No. S2: No. S3: No, we probably should go back! It just feels like there is no point in doing it because they usually say “this is what I am going to give you”. S2: The answers they give is very generic as well, so we really don't see the point in wasting our time. S1: From their feedback, we always end up thinking “okay, so what should we do more?”. In trying to not help one student more than the other, they end up not helping any students at all. I) Are you aware of the criteria of the assessments? S1: Yes. S2: Yes. S3: Yes. I) Do the assessment results affect the way you study? S1: I think so. It is more like a mental effect. When something goes bad, it can go either way: either you can get more motivated to work harder for your exam or you get demotivated and think “I’m not good enough”. S2: I kind of think I try hard anyways. I don't think I would try any extra hard. I) Do you base your study based on the marking criteria?
- 48. 48 S1: We’ve had some modules where they've told us what would be coming up and we would go “ok, I’ll learn these five topics and not learn the other six.” … exams take a lot of hard work anyways so why would you waste energy on a part that you know is not going to get you marks. S2: When you’ve got 5 exams, you don't really think whether what you are doing is going to benefit your future. S3: I think when it comes to course work, we have the freedom to choose which lecture we want to focus on, so we make the choice and study it in detail. I) Do you feel that you would lose out on anything if you left a topic altogether? S1: Not really, because we hardly learn anything we learnt at university. It’s more of maturity that we learn at university rather than the detailed skills. S2: Team work, you’re writing skills and stuff like that. It’s more general knowledge. S3: It depends on what field you will go and work in. For example, I was working in a consultancy firm and over there, it helped to know more because it had a lot to do with finance and I had done the finance module. So I thought to myself, “… only if I had listened some more!”. So maybe if we get a bit more specialized and learn a bit more in our area of interest, we will think “we should have paid more attention in our coursework”. S2: It depends on practicality. Finance is about numbers and using Excel but in subjects like marketing, I don’t think I used any marketing theories at all. What would have helped me if I had done a module where I had put together a marketing plan rather than being assessed on marketing theory. That would have been more transferable in terms of what I did in my placement. S1: We had one [module] this semester [Business Planning] which was very practical. We still had to do research on how we would have to do the thing [practical work] but it didn’t have big theories. There were theories that were useful, but it was all about applying them. S3: We learnt by doing it. I) Was the guidance given by lecturers consistent across the programme? S1: Some lecturers are very by the book and some give you the topics. Others say that they’d rather have reports from consultancies rather than articles and research. S2: Some say [in assessments] they want names and reference, other say that they want the name, reference and date. So it is very very inconsistent. I) Do you feel the need for consistency? S1: I think consistency in things like referencing, things like how we go about structuring an essay because it quite be quite difficult. I think our International HR module is the hardest, because their way of structuring the essay, I have never seen in my entire life. It was completely backwards to the way we usually do it. You read a paper and do you coursework on that, in other modules, you look at you lecture notes and support your coursework.
- 49. 49 S2: The difficulty was because some of the articles were really complex to read. So we didn’t really get what it was trying to say. So if we understood a tiny bit wrong, then it was so clear on the exam. S3: It was challenging! I) What assessment related factors motivated you to take your optional modules? S1: Balancing coursework and exams. S2: When you do placement year, you come back and think that you know what you can and cannot do. Having done 1st year, you know what modules you get bad marks in. So I am not going to risk it going into my final year. For instance, Maths is not really my strong point, so I am not going to risk doing a Math module. On the other hand, I’d prefer a Marketing module because I always think, “What do I enjoy? What am I good at?” S3: I like doing presentations. So I’d pick modules that have more presentations. Though there are not many who do presentations, so I guess that could be improved. I) Does the length of the lecture slides or material given make a difference in the modules you would choose? S1: Potentially, yes. I think I’d prefer the more because then we’d have more information. S2: This is more of a grey area. I am a bit lazy, so if the lecturer gives all the slides and notes, I would have to make less. Coming into the final year, what I realized is that they are not looking for you to rely on lecture notes because they don’t want you rely. S3: I think that when there is a bit less, you have the chance to read around the subject and write from what you learn. I felt that I was stronger in those exams. S1: Sometimes I feel that if they give you loads of notes, lecturers will feel that there is replication because that’s what they’ve given you. I) Something that I picked up from the questionnaires that we distributed: would you like to have exemplars? S1: Yeah, it would be great if they could do that, Anyways they are not going to have the same questions that they gave the previous years, so it’s not like you are going to copy it straight out. So it would be nice to have exemplars. I) Do you feel like you learnt anything over the course of your programme? S1: Yeah. S2: Yeah. S3: Yeah. S1: I remember what I did in my coursework [over the course of the programme]. Group work is also something I would remember because it was an extra challenge. A lot of people don’t like them because someone’s … S2: lacking
- 50. 50 S1: yeah, but I guess that’s something to do with the real world. I) Has the coursework helped you in exams? S1: Yeah. I think the way you write an answer. The way you structure it, I have learnt it in my coursework. I used this in my exams and it seems to work a lot better than when I used to go straight for exams. I) In hindsight, would you want any changes in the way the programme was planned? S1: Having done placement for a year, I was completely out of practice when it came to exams or coursework. I felt very shaky going into exams because the jump from second year to final year was something I found massive. I was getting a first in the first two years without having to read journals and all of a sudden in the final year, they were like “you won’t get a 2:1 if you don’t read journals.” That jump was quite big, so I think maybe some study skills would help. Something else that would be quite useful would be how to write a literature review. S2: Yeah. S3: I agree. S1: I had two modules of literature review and I have never written one. I still have no idea on how to write one. A little more explanation in how to use journals would also be great because in modules such as International HR, I did not do well as we were not told how to use journals. S2: In Cambridge, they have tutors that will guide students on how to write essays. I have never met my tutor more than thrice. Tutors should be able to meet with us and help us understand what literature reviews are or what exactly the general expectations are of students (with regards to coursework or exam answers). I feel that they are our tutors only because they have to be some students’ tutors every year. They really should be a bit more involved with what we do and not just want to meet us when things get out of hand. S1: Also, we’ve never seen the programme overseer. He’s taught us a couple of modules in Strategic Management but I don’t know why he wouldn’t help in exam techniques or in general such as essay writing or something. I) Were you satisfied with the way the programme was conducted? S1: Yeah. I think it was okay. There was a LOT of repetition, so although knowledge went in, there was so much repetition about culture … S2: Oh my God! S3: We just had to learn the same thing over and over again, so yes, although information does go in, there is more material out there that we could be learning. I feel that there is a lack of connection between the lecturers to know what each other is teaching. When we go in the following week, we would have basically the same lecture but in a different module. This leads to half of the students leaving and the lecturers wonder why the students have left. We leave because we get bored.
- 51. 51 S2: We want to be enthusiastic about what they are teaching us, but it becomes hard because we would’ve already heard it before. I) Alright! That is the end of the focus group interview. Thank you for taking the time to take part in this study. All the information you gave us has helped us a lot and will also help your school a great dea!