Assessment of Practicing skills
4 likes1,676 views
The document provides a comprehensive overview of evaluation methods in nursing education, focusing on checklists, rating scales, objective structured clinical examinations (OSCE), objective structured practical examinations (OSPE), and anecdotal records. It outlines the construction, advantages, and disadvantages of these assessment tools, emphasizing their role in measuring clinical competence and student performance. By detailing the steps involved in creating and implementing these instruments, the document aims to enhance the reliability and objectivity of evaluations in health professional education.
Assessment of Practicing skills
- 3. Checklists Rating Scale OSCE OSPE ANECDOTAL RECORDS Susmita Halder M. Sc. Nursing 1st Year CON BSMCH
- 5. Checklist A checklist is a simple instrument consisting prepared list of expected items of items of performance or attributes, which are checked by a evaluator for their presence or absence
- 6. Meaning British Columbia Institute of Technology- A checklist is a tool for identifying the presence or absence of conceptual knowledge, skills or behaviors. A checklist itemizes task descriptions, in one coloumn and provides a space beside each item, in a second coloumn to to check off the completion of a task
- 7. Characteristics Observe one respondent at a time Clearly specify the characteristics or behaviour to be observed. Use only carefully prepared checklist with simple traits Observed should be competent to use that tool Checklist should use to calculate only a particular characteristics.
- 8. Construction of a Checklist Express each item in clear and simple language An intensive survey of the literature should be made to determine the type of checklist to be used in particular assessment or evaluation The list of items in checklist may be continuous or divided into related items These lists are formulated on the basis of judgement of experts, and each item is evaluated with respect to number of favorable and unfavorable response Avoid negative statements whenever possible Avoid lifting statements verbatim from text Ensure each item has a clear response, yes/no, true/false Review the items independently Checklists must have the quality of completeness and comprehensiveness
- 9. Suggestions when using Checklists The checklist should directly relate to learning objectives Checklist need to be confined to performance, areas that can be assessed scientifically by examining positive and negative criteria and when sufficient opportunity for observation exists. Use checklist only when you are interested in ascertaining if a particular trait is present or absent. Clearly specify the traits to be observed Have a separate checklist for each candidate Observer must be trained how to observe. Multiple observations provide more accurate assessment than single observation Students should be evaluated in the natural setting or as closely as possible in a real situation
- 10. Steps oIdentify each of the specific actions desired in the performance Add to list those actions, that represent common errors Arrange the desired action Provide a simple procedure for for checking each action as it occurs.
- 12. Advantages Checklists allow inter individual comparisons They provide a simple method to record observations They are adaptable to subject matter areas Checklists are useful in evaluating learning activities expected to be performed They are helpful in evaluating procedure work Properly prepared checklists allow the observer to constrain the direct attention Checklists have objectivity to evaluate characteristics Useful for evaluating the process that can be subdivided into series of actions Decreases the chances of errors in observation
- 13. Disadvantages Do not indicate quality of performance so that the usefulness of checklists is limited Only a limited component of overall clinical performance can be evaluated Only the presence or absence of an attribute behaviour or performance parameter may be assessed. However, the degree of accuracy of performance can be assessed. It has limited use in qualitative observations Checklists are not easy to prepare.
- 14. Rating Scale
- 16. RATING SCALE According to Barr:-Rating is a term applied to an expression of opinion or judgments regarding some situation, object or character opinions are usually expressed on a scale of values. According to Ruth strength:- ‘Rating is, in essence, directed observation.’
- 17. Example Excellent Very good Good Poor Avarage
- 18. Types of Rating Scale Numerical scale Graphic scale Descriptive scale Comparative rating scale Forced choice
- 19. Numerical Scale
- 20. Graphic Scale
- 23. Others Comparative Scale Forced Choice Scale Paired Comparison Scaling Rank Order scaling
- 25. Others FORCED CHOICE SCALE An example by Guilford Careless Serious-minded Energetic Snobbish
- 26. Others COMPARATIVE RATING SCALE EX- Observer is asked to specify the comparative ability of providing health education of two students in a class
- 27. PAIRED COMPARATIVE RATING SCALE
- 30. RELEVANCE CLARITY Characteristics of Rating Scale OBJECTIVITY UNIQUENESS VARIETY
- 31. USES OF RATING SCALE Proceed evaluation Product evaluation Evaluating personal social development
- 32. Principles of preparing rating scale:- It directly relates to learning objectives. Needs to be confined to performances areas that can be observed. Clearly define the specific trait or mode of behaviour. The trait or behaviour should be readily observable, it should be observable in a number of situations. Allow some space in the rating scale card for the rater to give supplementary remarks. 3 - 7 rating positions may need to be provided. There should be provision to omit items as sometimes teacher fails to judge properly. Pooled ratings from more than one observer participation in instrument development will make this more objective, clear, valid, and reliable. All raters should be oriented to the specific scale as well as the process of rating in general. Consider evaluation setting feedback and student participation. The rater should be unbiased and trained. Have expert and well informed raters.
- 33. Steps in Constructing Rating Scale Selection of area Characteris tics to be measured in each area Define a range for each characteris tics Arrange items in scale Design directions Carry out Pilot test Make required modificatio ns
- 34. Advantages of rating Scales:- Checklists allow inter individual comparisons They provide a simple method to record observations They are adaptable to subject matter areas Checklists are useful in evaluating learning activities expected to be performed They are helpful in evaluating procedure work Properly prepared checklists allow the observer to constrain the direct attention Checklists have objectivity to evaluate characteristics Useful for evaluating the process that can be subdivided into series of actions Decreases the chances of errors in observation
- 35. Disadvantages of rating Scales:- •It is difficult or dangerous to fix up rating about many aspects of an individual. •misuse can result in a decrease in Objectivity. •There are chances of subjective evaluation thud the scales may become unscientific and unreliable. •Their is a lack of uniformity with which terms are interpreted by evaluators. •There are saveral common sources of errors in rating scales.
- 36. Defects of Rating Scale Error of Leniency Personal Bias Error Error of Central Tendency Halo effect Logical Error Contrast Error The proximity Error
- 38. MEANING Objective because examiners use a checklist for evaluating trainers Structured The marking scheme for each station is structured Clinical skills or procedure is typically broken down into component parts in a very structured way Structured interactions between an examiner and student Stations are short numerous and highly focused Clinical because the tasks are representative of those faced in real clinical situations Examination because the skills are assist in the form of examination
- 39. It is a form of multi-station examination of clinical subjects. It was first described by Harden et al. 1975. OSCE is a modern type of examination used in health sciences A framework for development for development of clinical competence has been described Miller, 1990, which outlines four levels of at which learner can be assessed- The OSCE conforms to third level of the Miller’s pyramid which focuses on assessment of performance of specific skills in controlled environment.
- 40. The development of Clinical Skills/ Competence/performance Adapted from MILLER 1990 Factual recall: MCQ, Essay type, Oral test KNOW KNOWS HOW SHOWS HOW DOES Context based assessment: MCQ, Essay type, Oral test Assessment of competence: Simulation, OSCE Assessment of competence: Observation in real setting
- 41. DEFINITION OSCE is an assessment tool in which the components of clinical competence such as history taking physical examination simple procedures interpretation of lab tests patient management problems communication attitude etc attested using agreed checklist and rotating the student round a number of stations some of which have observations with checklists. – N. AnanthaKrishnan
- 42. PURPOSES OF OSCE Used in both formative and summative assessment in health professional education Identify objective performance criteria for the skill being examined Structure the performance, criteria in a checklist to facilitate the identification of desired clinical skills A requirement for accreditation in many health professional programs
- 43. Structure of OSCE Practice-based Students are given written instruction and have to carry out a procedure Question-based Students have to answer questions about their findings at previous station, and interpret these findings.
- 44. USES Interpersonal and communication skills History taking skills Physical examination of specific body systems Mental health assessment Clinical decision- making including formation of differential diagnosis Clinical problem- solving skills Interpretation of clinical findings and investigations Management of a clinical situation including treatment and referral Patient education Health promotion Acting safely and appropriately in a urgent clinical situation Basic and advanced nursing care procedure practices
- 45. TIPS FOR ORGANISING ON OBJECTIVE STRUCTURED CLINICAL EXAMINATION Assessment Duration of Station No. of Stations Range of Approaches New Stations Organization of examinations Resource Requirements Plan examination and directions Change signal Records
- 46. Registration Orientation Escorting the students to exam position SYSTEMATIC STEPS IN ORGANISING AN OBJECTIVE STRUCTURED CLINICAL EXAMINATION
- 47. Station instruction time The encounter Escorting to the dismissal area SYSTEMATIC STEPS IN ORGANISING AN OBJECTIVE STRUCTURED CLINICAL EXAMINATION
- 48. EXAMPLE A MODEL OSCE FOR CRITICAL CARE NURSING STUDENTS PROCEDURE STATION – I Assess the patient X, Identify three priority nursing needs i. ……………………………... ii. ……………………………... iii.…………………………..... PROCEDURE STATION – II Perform chest auscultation of A and identify findings i. ………………………………………………. ii. …………………………………………………
- 49. EXAMPLE A MODEL OSCE FOR CRITICAL CARE NURSING STUDENTS PROCEDURE STATION – III Carry out ET suctioning of Y and record the procedure PROCEDURE STATION – IV Change FiO2 from 30% to 50% who is on ventilator
- 50. EXAMPLE A MODEL OSCE FOR CRITICAL CARE NURSING STUDENTS RESPONSE STATION – V Interpret the ABG Report pH=7.27 PaCO2= 49 HCO3=24 PaO2=92 Name the condition …………………………………………….. RESPONSE STATION – VI Mention indications, dose and three nursing considerations i. ………………………………………………. ii. ………………………………………………… iii.…………………………………………………
- 51. SIMULATED VERSUS REAL LIFE OSCE STATIONS The positive aspects of simulated objective structured clinical examination stations are as follows They are controlled and safe Feedback from modern sophisticated simulators can be obtained Simulators are readily available when required Simulated actions can be tailored to the level of skill to be assessed Sceneries that are distressing to real patients can be simulated In simulated stations the patient variable in examination is uniform across trainees
- 52. The positive aspects of real life OSCE stations are as follows Real life stations provide actual competence of a person on performance because idealized textbook scenarios may not mimic real life situations The allow assessment of complex skills which may not be possible at simulated stations Real life situations may be more cost effective
- 53. PROBLEMS OF USING OBJECTIVE STRUCTURED CLINICAL EXAMINATION IN THE INDIAN SCENARIO 1. Lack of feasibility due to time constraints 2. shortage of training for use of objective structured clinical examination 3. Shortage of observers or examiners 4. Lack of Infosys guidelines for practical examination by universities
- 54. ADVANTAGES More value than the traditional approach to clinical examinations Examination can decide in advance what is to be tested and can design the examination to test these competencies Examination can have better control on the content and complexities emphasis can be moved away from testing factual knowledge to testing a wide range of skills including advanced clinical skills more reliable because variable is of examiner and the patient removed to a large extent the use of checklists by examiners and the use of multiple choice questions results in a more objective examination More practical because it can be used with large number of students
- 55. DISADVANTAGES Students knowledge and skills are tested in compartments and they are not tested on their ability to look at the patient as a whole Demanding for both examiners and patients examiners are required to pay close attention to students repeating the same task on a number of occasions the time involved in setting up the examination is greater than for the traditional examination Maintaining uniform difficulty levels is not always possible.
- 56. OSPE
- 57. It is an assessment tool in which competence of a student is evaluated for general experiments in terms of identification of equipment or accessories of an experiment procedure of the experiment handling of the instruments making observations results interpretation of results and conclusion. New pattern of practical examination in which is component of clinical competence is tested uniformity and objectivity for all students Objective Structured Practical Examination
- 58. Features Assessment of process and product through observation of performance and assessment of result Adequate sampling of skills and content to the test An analytical approach to the assessment Objectivity Feedback to teacher and student
- 59. Procedure station This station requires students to perform a task such as monitoring of oral temperature. When a student performs the tasks simultaneously she is observed and marked against a checklist being prepared in advance so the students get score according to the skill demonstrated by her The question station or the response station Student answer the question being asked on the answer sheet provided and leave it in the place specified. Usually the question station may have a question related to the procedure station. Types
- 60. Methodology of objective structured practical examination During exam student pass through a number of stations Objective structured practical examination consists of 15-20 stations for exam of a particular course. Each station should not be less than 4 minutes. All the station should be completed in the same period of time where students are rotated through all the station and move to the next station on the ring of the bell. At some stations called procedure stations students are given tasks to perform on subjects at all stations there is observed with checklist to mark the student performance. At some station where student give responses unknown as response station where their findings of previous procedure station
- 61. Steps Demonstrate practical skills Make accurate observations Analyze and interpret data Identify the patients problems Plan alternative nursing interventions
- 62. Sample OSPE Station Question Method of scoring Procedure station Take oral temperature and record it Observed and scored by examiner using checklist Question station Convert 40 C into F using Formula Student answer on a sheet provided
- 63. Advantages It is more objective reliable and valid than the traditional system of examination All students are subject to the same standardized test the emphasis is shifted from testing factual knowledge to testing skills that to in a short time It has to ensure avoid coverage of all practical skills It ensures interaction of teaching and learning There is a increased faculty students interaction A large number of students can be tested within a restricted period of time
- 64. Disadvantages The simulator situation may not reflect real life situation Students cannot be assessed for different skills searches interpersonal relationship communication skills and dexterity in handling equipments Empathy towards the patients cannot be evaluated The skill of student in providing holistic nursing care cannot be assessed it may be time consuming to construct an objective structured practical examination it cannot be used a single person it needs more resources in terms of manpower time and money There is no interaction between the examiner and the student There is a risk of fatigue Breaking clinical skills to individual competencies maybe artificial
- 65. ANECDOTAL RECORD
- 66. Definition It is a brief description of observed behaviour that appears significant for evaluation purposes A factual record of an observation of a single specific significant incident in the behavior of a student an objective description by the teacher of a significant occurrence or an episode in life of the pupil
- 67. PURPOSES It provides an opportunity for healthy pupil teacher relationship It can be maintained in the areas of behaviour that cannot be evaluated by other systematic methods Help the students to improve their behaviour Useful in supplementing and validity observations made by other means. Can be used by students for self appraisal and peer assessment.
- 68. PURPOSES To furnish the multiplicity of evidence needed for good cumulative record to substitute for wake generalizations about students specific exact description of behaviour to stimulate teachers to look for information that is pertinent in helping each student realise good self adjustment to understand individuals basic personality pattern and his reactions in different situations The teacher is able to understand her pupil in a realistic manner
- 69. CHARACTERISTICS A factual description of an event Each anecdotal record contain a record of a single incident. The incident recorded should be one that is considered to be significant to the pupils growth and development
- 70. TYPES OF ANECDOTAL RECORD GENERAL SPECIFIC EVENT
- 71. Items in anecdotal record To relate the incident correctly for drawing inferences the following items to be incorporated 1. Name of the student class and school date of observation 2. Setting background of the incident 3. Signature of the observer 4. Interpretation of the behaviour 5. Recommendations concerning the behaviour
- 72. Behaviour number in item What happened: record incidence behavior and consequences Signature
- 74. Thank You