Assessing and evaluating

Assessing and Evaluating English Language Teacher Education 1
Assessing and Evaluating
English Language Teacher Education,
Teaching and Learning
Selected papers from the Second International
Conference of English Language Teacher
Educators held in Hyderabad, India
3 - 5 March 2012
Edited by
Dr Philip Powell-Davies
With an Introduction by
Professor Paul Gunashekar

Assessing and Evaluating English Language Teacher Education2
© British Council, 2013
ISBN: 978-81-922126-9-2
Published by
British Council
17 Kasturba Gandhi Marg
New Delhi 110 001
Printed by
Ishtihaar
511 Surya Kiran Building
19 Kasturba Gandhi Marg
New Delhi 110 001

CONTENTS
Introduction
Paul Gunashekar 5
Dilemmas in Observing, Supervising and Assessing Teachers
Rod Bolitho 7
Peer Observation and Feedback in ELT teacher training
programmes: A Constructive Model
Bose Vasudevan 13
How to Integrate the Planning, Implementation and
Evaluation of Trainer Training Courses
Cholpon Musaeva 19
Ripeness is all: a trainer’s narrative of lessons learnt
K. Padmini Shankar 26
Open-book examinations: The Need of the Hour
in Teacher Education
Geetha Durairajan 35
Evolving a process for Continuing Professional Development
of Teachers in Remote areas through SMS
Nivedita V Bedadur 45
Using classroom presentations as a tool for self-assessment
Padmini Boruah 53
Assessing the Critical Language Awareness of
ESL School Teachers
Deepesh Chandrasekharan 58

Evaluation of the Class X Course Books by Classroom
Teachers in Bangladesh
Mian Md. Naushaad Kabir 64
Training Needs Assessment of Secondary School English
Teachers in Aligarh
Raashid Nehal 72
Portfolio Assessment: Do Teachers and Experts Speak
the Same Language?
Ravinarayan Chakrakodi 81
Locating Continuous Comprehensive Evaluation in a
Pedagogic Frame
Jacob Tharu 96

Introduction
This volume is an anthology of twelve selected presentations on the theme of
Assessing and Evaluating English Language Teacher Education, Teaching and
Learning. This was the focus of the International Conference of English Language
Teacher Educators held in Hyderabad, India from 3 to 5 March 2012.
The central focus of the Conference was on the importance of assessing and
evaluating ideas and practices in the training classroom as well as the efficacy
of teacher education programmes and models. Against this backdrop, the
Conference focused on five themes – all of which are represented in this selection
of papers:
• Programme Evaluation: Padmini Shankar provides an insider
perspective on a postgraduate teacher training course she has been
offering; Cholpon Musaeva focuses on the qualitative differences
between a teacher- and a trainer-training programme; Naushaad Kabir
makes a case for including materials evaluation as a necessary
component of teacher orientation programmes; and Raashid Nehal
shares the findings of a study of a large-scale teacher training
programme in a rural setting.
• Assessing Language Capability: Geetha Durairajan posits that teacher
education programmes which value teacher reflection should include
enabling evaluation practices such as open book examinations; Deepesh
Chandrasekharan analyses the results of a study of the critical language
awareness of English teachers in schools and its implications for learner
empowerment; Padmini Boruah shares a strategy that encourages ELT
postgraduate students to develop self-assessment tools; and
Ravinarayan Chakrakodi presents a research study that attempts to
develop indigenous criteria for assessing writing portfolios in a teacher
education programme.
• Assessing and Evaluating Continuous Professional Development
(CPD): In the context of using technology for CPD, Nivedita Bedadur
offers mobile phone texting as a viable tool for promoting professional
development in rural areas where opportunities for direct engagement
with teachers of English are minimal.
• Evaluating Teacher Performance: Rod Bolitho examines the tensions
and dilemmas surrounding the assessment of practice teaching for both
observers and observees; and Bose Vasudevan looks at the possibilities
of using a task-based approach to peer observation.
• Continuous Comprehensive Evaluation (CCE): Against the background
of current attempts to implement and institutionalise CCE in India, Jacob

Tharu positions it on the continuum of general examination reforms in
the country and then highlights its potential for enriching pedagogy.
This collection of papers will give the reader a flavour of the spread of topics
explored at the Conference, and offer an overview of ideas that continue to
influence contemporary research in teacher assessment and programme
evaluation. We hope this publication will encourage teachers of English and
language teacher educators to contribute to the development of reliable
monitoring and evaluation systems and thereby influence quality and
accountability.
Professor Paul Gunashekar
Dean, School of English Language Education, EFL University, Hyderabad

Dilemmas in Observing, Supervising and Assessing
Teachers
Rod Bolitho, Norwich Institute for Language Education, UK
Readers who are in any way concerned with the observation, supervision and
assessment of teachers and teaching will recognise that these processes involve
a complex web of issues and dilemmas for both observers and observees. This
paper looks at some of these issues, largely from an observer/supervisor/assessor
perspective, under four main headings and also includes some recommendations
for improvements in our practices in this important area of our profession. A fair
amount has been written on supervision cycles (Gaies and Bowers 1990, Randall
with Thornton 2001), and on models of the supervision process (Freeman 1982,
Gebhard 1990), stressing the need to differentiate between degrees of
directiveness in feedback, but there is relatively little on attitudes, affect and on
the psychological factors at play in the observation encounter. This article sets
out to redress that balance.
Introduction
Very few teachers can put hand on heart and say that they enjoy being observed.
But why is this? There may be several reasons, but one of the most likely ones is
that it is perceived by an observee as an extremely high stakes event. The decision
to become a teacher is an existentially significant one, and the thought that
one’s professional judgement and skills are under scrutiny by an ‘outsider’ to
the class community one has striven to mould and develop is usually seen as
scary, perhaps even more so because a single observation can never be more
than just a ‘snapshot’ of the longer term endeavours which are crucial in teaching
and learning. The potential for misunderstandings seems, in the mind of the
observee, to be immense. Yet it is the observer who is the ‘senior partner’ in this
process and I will now look at ways in which observers can be more self-aware
and better prepared for the task they are charged with, whether they are
observing for assessment, for training, for research or for professional
development (cf. Maingay 1988). I see four main categories of dilemma, each of
which needs to be explored and understood at a deeper level.
Dilemma 1: Observer ‘Baggage’
When we go into a classroom to observe a teacher we are very seldom able to
do this in a completely non-judgemental way. Let’s look at what we take in with
us:
i Our own view of what constitutes good teaching. Most observers are
experienced teachers with their own ideas about how to teach and
how to behave as a teacher;

ii Our own view of what a classroom should look like, and of the ‘proper’
relationship between teachers and learners. These assumptions are
often culturally rooted and they are of particular importance in cross-
cultural observation encounters, for example, when a British teacher
educator observes a class in a developing world context;
iii i & ii above mean that we are in constant danger of thinking, “I wouldn’t
do it like that” or something similar;
iv In many contexts, the observer may know the teacher, the class or
both, with the result that he/she takes preconceptions into the
classroom. “I know she isn’t up to teaching at advanced level”; “I don’t
expect to see a brilliant lesson from X” etc; and
v We may go into an observation stressed by our last private or
professional encounters, with our mind on other priorities, maybe feeling
unwell or tired, or just in a bad mood after a terrible journey to work or
a disagreement with a family member or colleague.
None of this ‘baggage’ is easy to deal with, but awareness of it is half the battle.
Here are some of the things I tell myself before any and every observation:
• “I am going in to observe this teacher and her class on her terms and
not on mine. I need to clear my head of my own preferences and
practices and try to see what this teacher is trying to do and why. What
I would do in her situation is simply not relevant.”
• “In order to be fair to this teacher, I have to understand what she is
aiming to do before I go in. I need to have a chat with her and allow her
to tell me all about the class from her point of view. Any advice I give,
directly or indirectly, needs to be based on this understanding.”
• “I need to go into this class with an open and curious mind, and learn
from what I see and hear. I need to remember that there are many ways
of being a good teacher.”
• “I need to clear my head of all the preconditioned noise it contains
before I enter the classroom so that I can be fully ‘present’ and attentive
during the lesson.”
Dilemma 2: The Perception Gap
Any detective investigating a crime will tell you that there are usually as many
versions of events as there are witnesses, and that these all have to be sifted
and sorted in order to arrive at the common ground that is needed to move the
investigation forward. So it is with classroom observation. A teacher, managing
the class and engaging with his/her learners through spoken interaction and

eye contact, often from a standing position and from the front of the classroom,
is likely to perceive classroom incidents differently from an observer sitting at
the back or the side of the classroom, remaining passive and avoiding eye contact
with learners. The teacher will notice things that the observer fails to see, and
vice versa. And there is a third perspective which in certain situations will also
need to be taken into account: that of the learners themselves. Left unexplored
and unarticulated, these differences in perception may lead to misunderstanding
and resentment. Properly broached, for example, through questions in a post-
lesson feedback session, they become learning opportunities and also a chance
for the teacher to explain the rationale of a particular course of action. An
experienced observer or assessor will be aware of the potential for learning
and understanding if the perception gap becomes the focus of dialogue between
him/her and the observee. Productive exchanges such as these may then ensue:
Observer: “I noticed that most of your questions were directed at stronger
learners, and that some of the others seemed bored.”
Teacher: “Was I really doing that? It definitely wasn’t deliberate.”
Observer: “Did you know that some of the kids at the back were texting
on their mobile phones?”
Teacher: “To be honest, I didn’t. It’s quite difficult for me to see what the
pupils at the back are doing in such a crowded classroom.”
Observer: “Any ideas on how to deal with that in your next class?”
Teacher: “Not at the moment, but I’ll give it some thought.”
Teacher: “I was a bit concerned that I was too hard on students when
they made mistakes. I think I jump in too quickly with corrections.”
Observer: “I thought you handled the mistakes pretty effectively when
they were giving their summaries. Sometimes you need to intervene briefly
and supportively, and I thought you got that about right, to be honest.”
Teacher: “Thanks. Maybe I’ll also ask them how they felt about that when
I see them next time. I don’t want to discourage them.”
Observer: “Good idea!”
Focussing selectively, from different perspectives, on three or four incidents in
a lesson can lead to concrete action and improvement in a teacher. We need to
remember that this is often more helpful to a teacher than to be confronted with
‘big picture’ comments on the lesson as a whole, which may leave her wondering
where to start when planning the next lesson. It also encourages dialogue on an
equal footing between teacher and observer, and in the context of an assessed
lesson, may help the observer-assessor to understand the reasons behind a
teaching decision and find out if the teacher has acted in a principled way – or
not.

One consequence of the existence of the ‘perception gap’ as a potentially fruitful
area to explore through dialogue is that observers may need training in
supervision procedures and may need to be observed giving feedback in order
to become more aware of the value and impact of the different options available
to both parties in the post-lesson discussion (cf Freeman 1982, Gebhard 1990).
Dilemma 3: The Impact of an Observer’s Presence
Whatever their purpose, an observer is, at best, an invited guest and, at worst,
an intimidating presence in a classroom. A teacher has to find a way of explaining
the appearance of the observer to his/her students and this may be a delicate
matter, especially if the lesson is to be assessed. No observed lesson is ever
completely normal, and the visible presence of a ‘foreign body’ is all too often a
distraction for both teacher and learners. As an invited observer, but also as an
assessor, I have frequently realised that I was witnessing a ‘staged’, one-off lesson,
sometimes even pre-rehearsed, that bore little or no resemblance to what
happens between teacher and class on a day-to-day basis.
In some training and professional development contexts, a code of practice has
been established to provide a basis for classroom observation, supervision and
assessment. This might include the following guidelines, for example:
• The observer should keep a low profile, sitting where she can get a
good view of the lesson with the least possible distraction to teacher
and learners;
• The observer may be introduced to the class as a guest who is interested
in the teaching and learning that goes on in the classroom.
(These first two guidelines will help the learners’ curiosity to be satisfied
and may also lead to a quicker resumption of ‘normal’ classroom
behaviour.);
• The observer should let the teacher know if she wishes to take notes,
stressing that the teacher should not jump to any conclusions when
she notices the observer writing – this is merely a way of capturing
data which may be helpful to the teacher in the feedback session;
• The teacher should make the lesson as ‘normal’ as possible, and explain
to the observer where it fits into a longer sequence of teaching; and
• The observer should not play any active part, or intervene in any way,
in a lesson unless invited to do so by the teacher. (In one observation in
an Eastern European country, I was accompanied by a head of
department whose patience snapped half way through the lesson,
resulting in her standing up and taking the class over from the poor
teacher, causing what I can only imagine was a painful blow to her self-
esteem.)

In a procedure based on good practice in person-centred counselling, Quirke
(1996) suggests that it is sometimes better for an ‘observer’ not to be present
during a lesson but to give feedback based on a teacher’s account of the event.
Though this at least deals with the whole issue of the impact of an observer’s
presence, it is more likely to be effective in an atmosphere of trust within an
institution, possibly between peers working together on a reciprocal basis, rather
than in a more power-based relationship such as between a teacher and a
manager or a trainee and a supervisor-assessor.
Dilemma 4: The Delicate Matter of Assessment
The difficulty in attaining even a modest level of objectivity in assessing teachers
was alluded to in the discussion of the first dilemma above. This is compounded
by the stresses and strains involved for all parties when a teacher is assessed.
For a pre-service trainee it can be a ‘make or break’, life-changing moment; for
an experienced teacher self-esteem and self-worth are at stake. Here are just
some of the conundrums an assessor faces:
• On pre-service training courses, observers are often saddled with the
dual role of trainer and assessor. After a series of developmental
observations followed by formative feedback sessions, they are suddenly
required to carry out a summative assessment which may ultimately
decide whether a trainee is a suitable entrant to the teaching profession.
It can be difficult to reconcile these two roles;
• Is it better to use a checklist and to look for discrete points of
competence in a lesson or should assessment be conducted on a more
holistic basis, encompassing a view of a lesson as a complete event?
• Are the criteria the assessor will use clear and transparent to the teacher
being assessed? Are the criteria ‘global’ in nature or do they take account
of the teacher’s individual characteristics and the context in which he/
she operates? In short, does the teacher know what the assessor is
looking for?
• How can an assessor know that his/her assessment practices are in
line with those of others charged with the same responsibility?
• Is the teacher capable of self-assessment? If so, to what extent can an
‘external’ assessor take account of this when coming to a verdict on a
lesson?
The question of fairness and principled objectivity in assessing teachers and
teaching comes up again and again and is the cause of upset and grievances
when the views of the assessor and the assessed teachers cannot be reconciled.
Here are some examples of good practice in this vexed area of assessment:
• Criteria need to be clear, accessible and transparent. No teacher should
be surprised by the criteria used in their assessment;

• Box-ticking on a checklist of competences can distract an assessor
from the ‘big picture’ of a lesson. Checklists can be useful to refer to as
evidence for a grade allocated initially on the basis of a holistic
impression, but attempts to use them as criteria during an observation
can result in a skewed focus on micro-issues at the expense of a more
complete view of the lesson; and
• Assessors need regular training and updating, using video-recorded
lessons for benchmarking. These often involve useful discussions about
the meaning of criteria and the balance of factors contributing to a
decision about a grade. Exercises like this help assessors to be more
aware of their own preconceived ideas about ‘good teaching’.
Conclusion
In recent years there has been a welcome increase in attention to the training of
teacher trainers and educators. The practicum in pre-service training and
developmental observation for serving teachers are acknowledged as crucial
planks in maintaining and improving standards of teaching, and yet the trainer’s
or educator’s role as an observer, supervisor or assessor remains largely
underexplored, susceptible to subjectivity in its practices and cloaked in silence
and handed-down traditions rather than opened up in public debate. In this
article I have tried to open up some of the issues that need to be talked about
and resolved if this situation is to be improved.
References
Duff, T. (Ed) (1988): Explorations in Teacher Training. Harlow: Longman
Freeman, D. (1982): ‘Observing teachers: three approaches to in-service training
and development’ in TESOL Quarterly 16/1
Gaies, S. & Bowers, R. (1990): ‘Clinical supervision of language teaching: the supervisor
as trainer and educator’ in Richards, J.C. & D. Nunan (Eds) (1990): Second Language
Teacher Education. Cambridge: Cambridge University Press
Gebhard, J (1990): ‘Models of supervision: choices’ in Richards, J.C. & Nunan, D.
(Eds) (1990): Second Language Teacher Education. Cambridge: Cambridge University
Press
Maingay, P. (1988): ‘Observation for training, development or assessment’ in Duff, T.
(Ed) (1988): Explorations in Teacher Training. Harlow: Longman
Quirke, P. (1996): ‘Using unseen observations for an in-service teacher development
programme’ in The Teacher Trainer 10/1
Randall, M. with Thornton, B. (2001): Advising and Supporting Teachers Cambridge:
CUP
Richards, J.C. & Nunan, D. (Eds) (1990): Second Language Teacher Education.
Cambridge: Cambridge University Press

Peer Observation and Feedback in ELT teacher training
programmes: A Constructive Model
Dr Bose Vasudevan, Associate Professor in ELT, Institute of Language
Teaching, Jamnagar, Gujarat
The main objective of peer observation is to give feedback to the trainee on how
well she/he has performed in teaching a particular lesson in a given class. Very
often this main objective is not achieved as the peer group is not guided well
about how and what they have to observe. This paper looks into the possibilities
of using a task-based structured format which offers specific objectives, specified
areas of observation which includes teacher motivation, classroom management,
teacher’s language proficiency and pedagogical competence which includes
implementation of a particular method /approach/technique, interaction patterns,
appropriate use of teaching aids and materials, and strategies for dealing with
the errors.
Background
Peer observation and feedback has been a part of the B.Ed English programme
(a two semester ELT teacher training programme at the Institute of Language
Teaching, Jamnagar). Initially it was carried out without any prior guidance. The
trainees were divided into groups and each group had to teach a segment of
lesson (micro teaching) while another group observed them. The peer group
was unaware of what to observe and upon what to reflect. The outcome of such
observations was often counterproductive as most of the observers came out
with negative comments on their friends’ performance. There were even heated
arguments over the negative comments made by some of the peers on others’
performance. Some even demanded observation and feedback by the trainer
only. Let us look into samples of unstructured feedback given by the peer group
which observed the other group.
Phase I Unstructured Observation and Feedback
Positive feedback
Good/ average/ very good/ excellent/ well done/ good effort/ good command
over language/ good personality/good English/ lesson taught as per plan/
everything fine/ great/fairly confident etc…
Negative feedback
Not so good/below average/poor English/feeble voice/chalkboard use not
appealing/spelling errors on chalkboard/no movement in the class/no teaching
aids used/ everything went wrong/ no confidence/no eye-contact at all etc…..

It was evident from the above “feedback” that the observers tend to concentrate
on elements of personal interest and was not so helpful to the trainees in
developing their professional skills in teaching language. The feedback given
here was vague and over-emphasised the negative aspects of the teacher
observed. The “positive feedback” given was often partial. Ultimately the result
of such an unstructured observation and feedback caused much confusion and
chaos among the trainees and the training climate often became less than
encouraging.
Phase II Structured Observation and Feedback
An alternative scheme was designed to overcome the limitations of the
unstructured observation and feedback by the peers, which was more
quantitative in approach. A sample observation schedule is presented below:
Sr. Parameters Scope for Average Effective Very
No. improvement effective
Presentation 1 2 3 4
a. Details on class board,
lesson title, learning
outcomes, key words
are prominently
displayed
b. Uses relevant vocabulary
c. Exhibits fluency of
language
d. Uses suitable tone of
speech
e. Oral and written
instructions are clear
f. Teaching aids and learning
materials are appropriate,
well-organised, resourceful
and stimulating
The outcome of this type of observation was more acceptable than the first one.
However, it was found that often the observers were preoccupied with the
completion of the schedule and in some cases this proved to be less beneficial
for the person observing. In other words, it benefited mainly the teacher who is
observed. Hence the need arose to design a structured task-based observation

scheme which would benefit both the observer and the observed in a language
teaching context.
Phase III Structured Task-based Observation and Feedback: Objectives
In B.Ed English the trainees have to teach six types of lessons, all focusing on
developing language skills. Each type of lesson is divided into four stages,
typically introduction, presentation, practice/response, and communicative
practice/activity. Each trainee has to take one segment of the lesson and two or
three peers have to observe the lesson. Observation tasks are designed to help
the trainees observe a few aspects of classroom teaching at a time which include
specific objectives, teacher motivation, classroom management, teacher’s meta-
language, teacher’s language proficiency, pedagogical competence (which
includes implementation of a particular method /approach/technique),
interaction patterns, appropriate use of teaching aids and materials, and
strategies of dealing with the errors. These data are later analysed under the
supervision of the trainer and both the observer and the observed benefit from
the discussion. Using an observation task provides a convenient means of
collecting data that frees the observer from forming an opinion or making an
on-the-spot evaluation during the lesson (Wajnryb 1992).
Tryout
The tasks designed were tried out with B.Ed English trainees in their teaching
programme. Two or three peers were guided to observe one teacher-trainee at
a time and their seating arrangement was decided in advance to ensure they
would observe independently without sharing their findings with others at the
time of observation. The outcome of the experimentation of this model shows
that the confidence of the observed and the observers increased as they were
able to identify the benefits of sharing and mutually helping to develop specific
professional skills related to language teaching and learning. The language used
in solving tasks was original and reflected their critical thinking skills. A few
sample tasks which were designed for observation are presented here:
Sample Tasks:
Task 1.
What are the specific objectives of the Presentation stage?
a.
b.
c.

Task 2
a. How does the teacher arrange the class before the session?
b. How does the teacher motivate the class?
Task 3
What are the different classroom interaction patterns that the teacher has
achieved?
a.
b.
c.
d.
Task 4
Grammar
What new language item What techniques does the teacher use to
does the teacher present? present the new language item?
What types of tasks does the teacher give
the class to use the new language item?
Task 5
What types of errors does the How does the teacher deal with the
class make? errors?
Task 6 Teaching and learning roles
Lesson Stage Teacher’s role Learner’s role Comments
a.
b.
c.

Feedback
Observation tasks for each segment of a lesson could be designed so that the
observers would be able to give constructive feedback to the teacher trainee.
An observation task guides what the trainee has to observe so that there is no
confusion and minimal tension. The feedback given based on this scheme is
constructive and supportive, focused on what the teacher achieved in a given
segment. Observations of different segments of a lesson help to contribute to a
more holistic overall picture of a lesson, and brought under the guidance of the
experienced trainer so that all observers and observees can sit together and
discuss their findings to benefit each other.
Implementation
A briefing is necessary before assigning the tasks to the observers as these
tasks are more qualitative than quantitative and thereby they get a variety of
options, based partly on their understanding of teaching-learning theories, and
the model demonstrations given by the trainers. A completed sample task (see
task 7) could be discussed at the initial stage and later the trainees themselves
could carry on the observation and compare their scripts with those of other
peers’. The observers can at a later stage design their own observation tasks
under the supervision of the trainer. These tasks should be used systematically
at least in the initial stage of the training programme. Inclusion of such a
supportive and constructive peer observation and feedback component in ELT
teacher training programmes will certainly help teacher-trainees improve their
teaching style and classroom practice.
Conclusion
In a teacher training programme it is necessary to introduce a task-based peer
observation scheme as the teacher-trainees are inexperienced in observing and
making inferences. It also decreases the psychological pressure which they might
have when they are asked to observe a lesson without any guidance. A task-
Task 7. The teacher’s meta-language
What does the What is the What is the How might this
teacher say? communicative immediate be said to a
purpose? context? native speaker?
“Look at the Giving The teacher is Can you see
picture. Can you instructions setting up a task where the boys
see the boys with a visual aid are sitting (in
sitting under the Directing (pictures/charts) the picture)?
tree?
Adapted from: Ruth Wajnryb (1992)

based observation and feedback scheme will help provide a conducive climate
in which both the givers and receivers of feedback can deepen their
understanding of effective language teaching practices that promote quality
learning.
References
Allwright, R. & Bailey, K. (1991): Focus on the language Classroom: an introduction to
research for language teachers. Cambridge University Press
Poumellec, H.H., Parrish, P.& Garson, J.. (1992): ‘Peer Observation and Feedback in
Teacher Training Development’ in The Journal of TESOL France XII.1: 129-140
Richards, J. C. & Nunan, D. (1990): Second Language Teacher Education. Cambridge
University Press
Wallace, M. (1991): Training Foreign Language teachers: a reflective approach.
Cambridge University Press

How to Integrate the Planning, Implementation and
Evaluation of Trainer Training Courses
Cholpon Musaeva is a PhD research scholar in English Language
Education, and a teaching assistant at The English and Foreign
Languages University (EFLU) in Hyderabad, India
There is a considerable difference between a teacher and a teacher trainer.
Consequently, aims, design, materials, methodology and evaluation of teacher
training and trainer training programmes should be approached methodically
based on profound understanding of the nature and scope of each. Failure in
visualising, operationalising and distinguishing between the two can lead to
ineffective outcomes. This paper examines a one-month trainer training course,
arguing that its purpose (as a trainer training programme) is compromised due
to its resemblance to a teacher training programme. It provides suggestions for
remedying this situation. First, it sets the scene. Then, it lists the programme’s
gaps, and provides ways of bridging them.
Introduction
A teacher trainer (teacher educator) has considerable responsibilities in training
the next generation of teachers. It is a demanding role. The trainer’s job is
multidimensional, involving the mastering of aspects of the subject matter,
methodology and materials development and understanding how this adds to
the knowledge, skills and practices of a good teacher educator. But this is not
sufficient in itself.
A teacher trainer also has to be able to motivate, guide and facilitate participants’
learning; observe classes and give constructive feedback to enable participants
to grow by reflecting upon themselves and their classrooms – in other words,
the trainer needs to understand the human dimension of their work and deal
with trainees’ affective factors and belief systems to enable them to find their
own unique ways of teaching.
Teacher educators should also possess theoretical knowledge of different
teaching and learning principles, concepts and processes; as well as training
models; and be familiar with the academic literature that underpins their work.
This linking of theory and practice is essential to producing well-prepared trainee
teachers and informing their thought and practice about teaching.
This paper examines the trainer training programme for teachers from Central
Asian and South Asian countries jointly sponsored by The EFLU and The RELO,
and run in India. It argues that the course lacks the features of a true trainer

training programme due to a number of gaps identified through a qualitative
investigation into the programme. The study provides suggestions on what the
course components and processes should be; and how trainer training course
organisers, designers, material producers, implementers and evaluators should
collaborate to run appropriate trainer training courses.
Course Outline
The course is divided into three phases:
• The first phase consists of sessions on effective teaching and
introduction to Communicative Language Teaching (CLT); skills and sub-
skills of listening, speaking, reading and writing; grammar; vocabulary;
classroom management, educational technology and assessment. The
sessions are conducted daily from 10 am to 5 pm. After the sessions,
participants work in groups (each group consisting of four or five
participants) and plan and prepare lessons for peer/team teaching with
the help of a supervisor.
• The second phase is peer teaching, where trainees teach sections of
their prepared lessons to their fellow students who act as students. The
lessons are observed by a trainer or guest observer; and are followed
by a feedback session.
• The third phase is ‘real teaching’ where participants teach a lesson with
a group of proficiency course learners, and the lessons are observed
by a trainer and followed by a feedback session.
Participants
In the case of the course under examination, twenty-four participants from South
and Central Asian countries attended a one-month trainer training course run in
India. The participants came from Kyrgyzstan, Kazakhstan, Uzbekistan, Tajikistan
and Turkmenistan, India, Nepal, Bangladesh, Sri Lanka, and Afghanistan.
The participants were unaware of most of the current trends in English Language
Teaching (ELT), and were heterogeneous in terms of their teaching experience.
A few of them had participated in in-service teacher training courses in their
countries, and knew quite a few classroom activities, but they were largely
unaware of more advanced concepts and principles of teaching and learning
methodology, materials development, course design, lesson/course planning,
and monitoring and evaluation principles for example. The participants had
varying levels of competency in English, due in part to levels of regular exposure
to English other than in a formal learning environment. The majority had
experienced (and learned through) grammar translation methods of teaching.
Match and Mismatch
The course components listed above suggest that it was a short-term awareness-

raising in-service teacher training programme which was suitable for majority
of this group. The content of the course was appropriate in that sense, since
probably about 80 per cent of the course participants required teacher training,
whereas only 20 per cent- were ready for a trainer training course (at the
beginning of the course).
This methodology course was clearly an effective teacher training exercise where
the participants carried out a variety of tasks as students, then discussed them
as teachers analysing different frameworks for teaching skills and sub-skills,
lesson planning, classroom management techniques and forms of assessment
for learning and teaching. However, there seemed to be a mismatch between
the title of the course and its content, which leads me to suggest that it should
not be called a trainer training programme for the following reasons:
• In terms of competence:
o It did not mention the importance of awareness and levels of
competence (Luft & Ingram 1969, Underhill, 1992). As Knezedvic
(2001:10) has pointed out, “We cannot develop unless we are aware
who we are and what we do”. According to him developing
awareness is a process of “reducing discrepancy” between what
we do and what we think we do (cited in Bailey 2006: 39). I believe
this knowledge is necessary for a trainer who deals with change
and development.
o Its focus was on teaching knowledge and skills, with only a few
attempts to help participants reflect by asking, ‘What can you take
with you to your classroom?’
o Participants’ beliefs were not examined explicitly nor, was the
importance of beliefs and assumptions in training highlighted and
discussed in detail. This is essential in order to enable new trainer
trainers to plan, teach and evaluate their lessons or courses
independently based on prudent consideration when they go back
to their classrooms.
o The importance of ‘reflection’ in a teacher’s or trainer’s growth
required much more interrogation, to enable them to work
autonomously through conscious and deliberate thought, search,
action, and reflection using different reflective strategies/ practices.
One of the goals of a training programme is change in some aspect
or other (assumptions, knowledge, skills or practices) and profound
change can only happen through reflection.

• Based on professional development:
o The ‘human side’ of the training process also needs to be addressed.
Managerial and psychological knowledge and skills in motivating,
guiding and facilitating learning; observing classes and giving
constructive feedback; managing group dynamics and conflict;
interpersonal skills (the ability to communicate, listen, see, and read
non-verbal signals) are all essential skills that the teacher trainer
requires to guide new teachers and enable them to facilitate
learning effectively.
o An appropriate focus for a trainer training programme would be to
distinguish clearly between the theoretical concepts of teaching
and learning processes that apply to teaching and to training, and
also where they overlap. This needs to be underpinned by a
thorough examination of educational theories; basic models of
supervision, counselling, coaching and mentoring, and the
principles of adult learning. This could include the Vygotskian Zone
of Proximal Development (ZPD) (1978); Bloom’s taxonomy (1956 in
Anderson & Krathwohl 2001); and principles of experiential and
reflective learning.
o A focus on instructional design models and principles would also
have been appropriate so that participants would have some insight
into planning and delivering programmes of professional
development for new or inexperienced teachers in their home
situation. This would require them to develop knowledge and skill
in materials development and the construction of tasks; the nature
and effect of various types of inputs; the purposes of tasks at
different stages of a training course, and so on.
Including all of these above in a short course would be neither realistic nor
feasible. However, I would argue they form the fundamental knowledge base to
function effectively as a teacher educator and as such they are subtly different
from the content of a teacher training course. Therefore, awareness of these
areas will help stakeholders to collaborate to plan, run and evaluate the trainer
training courses.
Factors to take into consideration in trainer training courses
Based on the experience of observing the training course in question, the
following would seem to be important considerations to factor into the planning
and design of future training courses.
• Eligibility criteria for entering ‘trainer training’ programme should be
established, including:

o teaching experience of not less than 4-5 years
o previous participation in awareness-raising in-service teacher
training courses
• The design of teacher educator programmes should be cognizant of
the experience of the trainees and build on it in the following way by
factoring in:
o Proficiency building components to raise awareness and encourage
reflection on the participants’ knowledge and experience. This
would enable important discussions to take place: on the differences
between teacher training and trainer training; how good training
techniques can be modeled in the way the sessions are delivered;
how to design tasks to stimulate reflection and discussion of
teaching and learning issues; as well as how best to address issues
in the teaching of vocabulary or grammar, for example.
o Professional competence/development components dealing with
training the trainer, the human or affective elements of the
educator’s role; understanding and utilizing learning and teaching
theory; how to use frameworks such as the ADDIE instructional
design model (see below and Figure 1); and analyzing elements of
both the training and teaching practicum, amongst other topics.
Analysing the effectiveness of a training programme using the ADDIE instructional
model (Analysis of the target group’s needs, Design/blueprint of the course,
Development of course materials, Implementation/delivery of the course, and
Evaluation (both formative and summative) of the programme) enables us to
appreciate the areas that might require attention to sharpen the focus of future
courses including management issues and a lack of collaboration in the design
and delivery of each phase, and enhanced formative and summative evaluation
of the course.
Ways to Bridge The Gaps between Planning, Implementation and
Evaluation
• In cases when a course is run in a third country, all the stakeholders
responsible for selection of participants from different countries need
to cooperate before, during, and after the course is over. This will help
to assess whether the course has been effective or not. Such
collaboration can be achieved by using the ADDIE model as the basis
for shared planning and understanding going forward. It means that
needs analysts, designers, materials developers, trainers and evaluators
(trainers and country representatives) work together in order to run an
effective trainer training programme.

ADDIE Model (diagram adapted from Steven J. McGriff, Instructional Systems,
College of Education, Penn State University)
The ADDIE model’s focus on formative evaluation of the course will help to build
in immediate remedies during the course. However, to understand the longer-
term actual impact of the training, Kirkpatrick’s model for evaluating training
(which consists of 4 levels: 1. React, 2. Learn, 3. Change, and 4. Impact) can be
adapted. It implies that the concerned stakeholders and the course participants
should continue to work together after the course is over. One could argue,
using Kirkpatrick’s model, that any given training course can be assessed for
effectiveness by examining the extent to which all four of the levels are present
and well-integrated in the design, delivery and monitoring and evaluation of the
training
Conclusion
Understanding the nature of teacher- and trainer- training is crucial for designing
cost-effective programmes which address the target groups’ needs. Establishing
a clear understanding of the role of the trainer and how they deploy the skills of
facilitation of learning is not straightforward. It needs to build on the basics of
teacher training methodology but at the same time go a stage further so that
the experienced teacher wishing to become a trainer reflects on their own
teaching experience and practices to make sense of what is required to train a
new cohort of teachers. In the trainer training room, the teacher has to put him/
herself once again in the position of student to understand how to model
concepts and practices of trainer training, reflect on what they think they already

know, appreciate and anticipate the kinds of problems that new teachers face
and put in front of trainees a range of possible solutions to experiment with.
References
Anderson, L. W. & Krathwohl, D. A. (2001): A Taxonomy for Learning, Teaching, and
Assessing: A revision of Bloom’s Taxonomy of educational objectives. London: Longman
Bailey, K. M. (20060: Language Teacher Supervision. A case-based approach.
Cambridge: Cambridge University Press
Cullen, R. (1994): ‘Incorporating a Language Component in Teacher Training
Programmes’ in ELT Journal 48/2: 162-172
Instructional Design Central. http://www.instructionaldesigncentral.com/htm/
IDC_instructionaldesignmodels.htm (last accessed 1 December 2012)
Kirkpatrick, D. L. and Kirkpatrick, J. D. (2006): Evaluating Training Programs: The
Four Levels. San Francisco: Berrett-Koelher
Kolb, D. A. (1984): Experiential Learning. Experience as the source of learning and
development. New Jersey: Prentice-Hall
Vygotsky, L.S. (1978): Mind and society: The development of higher mental processes.
Cambridge, MA: Harvard University Press
Woodward, T. (1991): Models and Metaphors in Language Teacher Training: Loop Input
and Other Strategies. Cambridge: Cambridge University Press
Wright, T. & R. Bolitho. (1993): ‘Language Awareness: A missing link in language
teacher education?’ in ELT Journal 47/4: 292-304
Wright, T. & R. Bolitho. (2007). Trainer Development (ID 554846 www.lulu.com)

Ripeness is all: a trainer’s narrative of lessons learnt
K. Padmini Shankar, Associate Professor, Dept. of ESL Studies, The
English and Foreign Languages University, Hyderabad, India
“That’s the best part of teaching – the learning.”
(An experienced teacher serving as a Peace Corps volunteer I Kiribati)
This paper narrates the story of a teacher trainer. The data are drawn from a
teacher training course titled Training for Language Teaching offered to semester
III students of the M.A. TESL Programme at the English and Foreign Languages
University, Hyderabad. The discussion spans the peer teaching and practice
teaching phases of supervision from three perspectives: a) the student teacher’s
b) the peer observer’s and c) the supervisor’s. The intention is to delineate
vignettes that provoke reflection and subsequent action regarding issues such
as the role of feedback in language teacher supervision; factors affecting
supervisory discourse; macro-analyses of post-observation conference, and
supervisor’s non-verbal behaviour during post-observation conference.
Aim and background
The aim of this paper is to share lessons learnt from and about language teacher
supervision. A 5 credit core course, titled Training for Language Teaching, has
been offered to semester III M.A. TESL students for the past four years and the
experiences of the trainer are documented in this paper. The participants are
mostly pre-service teachers. The course spans a period of four months. In the
first two months theoretical inputs are offered and in the next two months the
practicum is organised, consisting of peer and practice teaching.
Theoretical support
This narrative is based on the following theoretical propositions:
• A practicum aims to help trainees to enter the real world of the
classroom; however, this can sometimes be “an uneasy transition” laden
with pressing concerns and frustrations (Bailey 2006:239)
• Teacher educators in the practicum setting can empower trainees by
enabling them to understand and explore their teaching. Such
understanding helps trainees to “adapt their teaching to the specific
settings they work in, as well as be able to continuously construct and
reconstruct their teaching and teacher identities” (Gebhard, 2009:255).
• “Trainer development” is a vital aspect of language teacher education
because of teacher educator’s central role in defining and disseminating
ideas about pedagogy. The term captures the developmental process

of constantly “becoming” a language teacher educator. Learning to
become a teacher educator is as multifaceted as the role itself, and a
pedagogy of trainer development will reflect this reality (Wright
2009:102,109).
• A trainer is not only a researcher but also a learner – someone interested
in learning about process from their own and other people’s practice,
from trainee initiatives and reactions, and from reading and thinking.
The practising trainer is experiencing two complements to their role:
as both learner and researcher (Woodward 1991: 231).
The sources (of the narrative)
The data are gathered from:
• Trainee teachers’ profiles filled in at the beginning of the course
• Supervisor’s notes and reflective journal maintained throughout the
course
• Trainee teachers’ feedback forms collected at the end of the course
(after grades were declared so they did not have to say things to please
the supervisor).
The structure (of the narrative)
This narrative presents the perspectives of the trainee teachers, the peer
observers and the supervisor on the experience of the practicum (both peer
teaching as well as practice teaching).
The substance (of the narrative): peer and practice teaching
This narrative consists of the perspectives of the trainees, the peer observers
and the supervisor on the practicum (peer and practice teaching) and the lessons

that the trainer learnt, which are discussed in the later part of the paper. The
three perspectives are presented below.
The trainee teachers’ perspective
Individual trainee’s perceptions on the practicum have been collated and a
representative selection is outlined below:
• I was not very comfortable with teaching my own classmates as though
they were children. But the process helped me in the classroom teaching
session.
• The idea of collaborative teaching is good if all team-mates gel well; it
provides a platform for interaction and learning from more experienced
peers; however, there is the risk of unequal distribution of work.
• Peer teaching was very helpful, although a bit artificial sometimes.
• I realised working in a team is tough but the end results are always
good. I also learned to respect and appreciate other people.
• I cannot say that we became best friends, but we were supporting each
other where we could.
• Peer teaching taught me how to work and coordinate in a group; it helped
me improve not only teaching but taught me to be a ‘team-player’.
The peer observers’ perspective
Individual trainee’s perceptions on the practicum have been collated and a
representative selection is outlined below:
• We learnt how useful a tool lesson observation is “to become more self-
aware”, apart from being an aid to learn to teach and learn to observe.
• We could understand and rationalise ‘the planning and interactive’
decisions teachers made.
• Peer observation included comments on issues such as organisation of
the lesson, teacher’s time management, students’ performance on tasks,
teacher’s explanations, teacher’s action zone, classroom interaction etc
and none of these topics requires the observer “to evaluate the teacher’s
lesson”.
• Being observed by others is always a threatening experience because
the teacher is “on show.”
• Peer observation helped us become more aware of the issues we
confront in the classroom and how these can be resolved; it helped narrow
the gap between our imagined view of teaching and what actually
occurred in the classroom.

The supervisor’s perspective
Throughout the course, I was trying to juggle the twin demands of counsellor
(advising, nurturing, encouraging, guiding, supporting) and critic (analysing,
assessing, pointing out inadequacies). I was trying to be supportive, informative,
prescriptive, confronting, catalytic and cathartic, and the inspiration for these
roles was drawn from Heron’s (1990) framework:
Figure 1: Heron’s six categories of intervention
Heron (1990, cited in Randall and Thornton 2001:78-79) uses these terms
descriptively rather than prescriptively. All the categories are value-free and
are primarily supportive of the value of the individual. They aim to enhance the
client’s ability for self-direction.
So, what was my aim as the supervisor/trainer? My goal was to seek out
“additional opportunities to learn and develop” and these are the lessons that I
learnt.

Lessons learnt
The lessons that I learnt are discussed in terms of the following vignettes:
Vignette 1: Trainee teacher’s views about teaching
• I want to be able to teach in such a way that they learn that learning is
fun, interesting, helpful and life-changing, and I want to make my own bit
of life-changing for them.
• Teaching is fun, pleasure.
• Teaching is constant ‘learning’ to be able to teach better.
• Teaching is encouraging students to find out the best in themselves and
grow to their maximum potential.
What have I learnt?
It is important to explore trainees’ views about teaching because what we teach
and how we teach is greatly influenced by what we perceive teaching to be.
Further, the responses reveal two important things about the trainees: their
readiness and willingness to learn and, more importantly, the aspects of teaching
in which they wished to be trained.
Vignette 2: Trainee teachers’ goals and expectations
• I want to be a good teacher when I return to my country.
• I’m hopeful that this course will help me become a better teacher as
well as provide me with the essential skills for a successful teacher trainer.
• I want to do MA TESL and this is a core course and hence it is important
for me.
• I think this course will give me a concrete guideline for being a teacher
and the practical sessions will be a platform for me before I go to teach
my students.
• I hope this course will give me the confidence to handle teaching children.
• I would like hands-on experience on how to do classroom teaching.
• I have recently started working and that is when I’ve realised that it is
not easy to be a teacher and hence I wanted to get the benefit of the
training.
• I think this course will be informative, practical and we will learn by doing
things; lots of analysis and discussion will take place.
• I wish to learn to pitch my knowledge at appropriate levels, to be
interactive in the true sense of the term, select suitable materials.

• To get a basic idea of the methods of classroom teaching – planning,
implementation, and evaluation.
• I expect to learn how to teach and how to elicit or handle positive
criticism/feedback.
• I heard that the trainer is friendly, supportive and strict at the same time
and presents her subject very well.
What have I learnt?
Learners almost always do have a range of expectations – very general to very
specific – about a training course. It is as important to be aware of these as it is
a challenge to fulfill all of them.
Vignette 3: Role of peer/ tutor feedback in learning to teach
• Post-observation conferences were very detailed and useful; peers were
friendly while giving comments.
• Peer feedback was helpful, but sometimes, peers were a bit harsh with
selection of words.
• Most of the peers were honest with their views and voiced what they
thought.
• Peer feedback was accurate and constructive.
• Peer feedback was detailed and spirited.
• I really didn’t see how important it was when we were doing it because I
thought it was a simulated situation but I saw how important it was when
I actually taught at the school because that definitely helped me a lot.
• Tutor feedback was insightful; there was enough individual attention.
• The tutor needs to be more authoritative in some situations.
• The tutor was supportive but sounded too critical at times.
• Tutor feedback was rational and helped to limit future mistakes.
• Even if there were kicks the tutor made them such that no one got hurt.
• Expert opinion is good and helpful.
Vignette 4: Macro-analyses of post-observation conference
As the essence of providing help is to create a climate of trust, the challenge for
the supervisor is to sustain a supportive and valuing atmosphere whilst raising
uncomfortable and critical comments about the lesson. Sometimes both the
trainee and the supervisor will experience raised levels of anxiety the effects of

which are summarised in the following diagram (Randall and Thornton 2001:84,
adapted from Heron 1990: 46):
Figure 2: Effect of supervisor’s and trainee’s anxiety in the post-observation
conference
The supervisor/advisor’s anxiety about offering negative feedback results in
two possible scenarios: ‘pussyfooting’, wherein negative comments are either
evaded or sugar-coated to avoid confrontation thereby rendering the post-
observation feedback session ineffective; and ‘clobbering’, wherein the advisor
and the teacher are locked in battle, the teacher behind defensive walls and the
advisor ‘clobbering’ with even more aggressive comment leading to a further
cycle of defensive reactions from the teacher without the central issue being
openly and calmly addressed.
What have I learnt?
The tutor should be honest and truthful in giving feedback, but this feedback
should be phrased and conveyed in such a way that trainees understand that it

is not the teacher who is being examined but the way that the lesson was taught.
A ‘problem’ in the lesson is not to be considered a ‘fault’ in the teacher and
trainees should be encouraged to develop such a view of their own and others’
teaching, and how one achieves this is a million dollar question.
Vignette 5: What have my trainees learnt?
The trainees were asked to respond to the following prompt: “after doing this
course, teaching for you is – “ and their responses are reported below:
Teaching for me is…
• Not as easy as I thought it to be, but can be made easy with practice.
• A lot of planning! Didn’t know that planning should be done so much in
detail, but I also see the advantage of it.
• All about developing my personality and my English proficiency.
• Something that needs to be well planned and prepared.
• A never-ending process of exploration and growth.
• Very easy and enjoyable.
• A responsibility that should not be taken lightly.
• I now believe that I can do it and I will do it.
• I realised I love and enjoy teaching.
What have I learnt from what they say they have learnt?
That training, like teaching, is a huge responsibility.
And, a successful trainer is someone who ignites and sustains a passion for
teaching in the trainees while all along playing the role of a signpost directing
but not dominating, suggesting but not imposing, assessing but not judging.
And the list goes on!!
Over the years I have learnt several things. I have unlearnt numerous things. I
have re-learnt innumerable things. And …the journey towards ripeness continues!
References
Bailey, K, M. (2006): Language Teacher Supervision: A Case-Based Approach.
Cambridge: CUP
Gebhard, J. G. (1999): “Seeing teaching differently through observation.” in Gebhard,
J.G. & Oprandy, R. Language Teaching Awareness: A Guide to Exploring Beliefs and
Practices, pp. 35-58. Cambridge: CUP

Gebhard, J.G. (2009): ‘The Practicum’ in Burns, A. & Richards, J. C. (Eds.) The
Cambridge Guide to Second Language Teacher Education, pp. 250-258. Cambridge:
CUP
Hayes, D. (Ed). (2004): Trainer development: Principles and practice for language
teacher training. Melbourne: Language Australia
Loughran, J. (20060: Developing a pedagogy of teacher education: Understanding
teaching and learning about teaching. London: Routledge
McGrath, I. (Ed) (1997): Learning to train: Perspectives on the development of language
teacher trainers. Hemel Hempstead, UK: Prentice Hall
Randall, M. & Thornton, B. (2001): Advising and Supporting Teachers. Cambridge:
CUP
Wright, T. (2009): ‘Trainer Development: Professional Development for Language
Teacher Educators’ in Burns, A. & Richards, J. C. (Eds). The Cambridge Guide to
Second Language Teacher Education, pp. 102-112. Cambridge: CUP
Woodward, T. (1991): Models and Metaphors in Language Teacher Training: Loop input
and other strategies. Cambridge: CUP

Open-book examinations: The Need of the Hour in
Teacher Education
Geetha Durairajan, Professor, Department of Testing and Evaluation,
English and Foreign Languages University, Hyderabad
Most teacher education programmes use only closed book, memory based
examinations for their summative (certification oriented) evaluation. Such a choice
is probably governed largely by practical constraints rather than theoretical
justifications. In real life all teaching practitioners have reference material
available. But case study based evaluations/problem solving questions (which
ought to be the norm) are still the exception. Teacher education programmes
that aim to help the teacher to think, grow and reflect need to be accompanied
by enabling evaluative practices such as open book examinations. In this paper a
theoretical justification and sample tasks will be presented.
Nature of Teacher Education in the 21st Century
In the 21st century, education ought to be inclusive and accessible to all sectors
of society. The National Curriculum Framework (NCF, 2005) makes a clear
distinction between knowledge that is fluid and that which is reproduced. The
former can never be a given and must be constructed by the child. For this to
happen, learning can no longer be the memorisation and transmission of facts;
the purpose of teaching is to enhance children’s desire to learn, help them
express themselves and engage with society (NCF, 2005). Evaluation, as part of
inclusive education also has to play a different role; it cannot be perceived either
as a disciplinary tool (Foucault, 1975 [trans 1995]) or as one that eliminates/
selects. There is a need to shift the focus from the assessment of learning to the
assessment for learning; evaluation will then become an integral part of the
teaching-learning loop.
In order to cater to these demands of inclusive education and evaluation, the
nature of teacher education has to change; it has to help teachers acknowledge
diversity of learning spaces and curriculum styles and also help them build the
capacity in the child to construct knowledge. Teacher education programmes
must, therefore, include reflective practice for unless the teacher is able to reflect
and become critical, he/she cannot enable such thinking in the mind of the
child. The National Curriculum Framework for Teacher Education (NCFTE) clearly
mandates that teacher education must be comprehensive, and evaluate
conceptual and pedagogical aspects, along with attitudes, values and dispositions
(NCFTE, 2009).

Evaluation practices in teacher education courses
Most teacher education programmes in India (pre- or in-service) use only closed-
book memory- based examinations as part of summative assessment. These
examinations are certification- oriented but lack predictive validity. The certificate
is supposed to guarantee that its holder can teach well, can access knowledge
using multiple resources either to answer student queries or update their own
knowledge. But the examination questions are factual, demand reproduction or
recall of facts and therefore provide training, (if it can be called that) in judicious
cramming.
The need of the hour in teacher education is tests and examinations that
• evaluate higher order thinking skills;
• move from knowledge and comprehension to analysis, application and
synthesis;
• distinguish between core and back up knowledge;
• trigger deep rather than surface learning;
• make teachers situate their knowledge in suitable contexts, and thereby
• enable reflective thinking.
Alternative assessment practices (assignments, term papers, projects) in teacher
education courses do focus on these aspects and are excellent evaluation tools,
but there is no guarantee that they will serve as true indicators of individual
performance and by inference, capability. One cannot prevent students from
discussing assignments and term papers with more experienced colleagues or
more enabled peers. Scaffolding is not a necessary evil to be lived with and
should be encouraged to enable learning to happen. However ‘scaffolded
performance’ is not desirable where individuated certification is involved. In
this context, open book examinations need to fulfil the requirements of summative
evaluation and also ensure that knowledge is not merely reproduced but applied,
synthesised and evaluated.
Definitions and Types of Open Book Examinations
An open book examination (OBE) is a valid method of testing how much a student
has learnt and internalised from a course; it is an examination that does not test
knowledge of theory but instead focuses on practical applications of theory, in
new/authentic contexts. The questions/tasks are usually context specific and
may also carry the possibility of open ended responses. It is an examination
where students can have access to their prescribed textbooks and even other
resource material. Since such referring is possible, meaningful and deep learning
can be fostered.

There are many kinds of OBEs, ranging from allowing access to restricted
resources, access to unrestricted resources, as well as open resource
examinations where access to an entire library is permitted. In a restricted OBE
printed documents such as logarithmic tables or dictionaries are permitted. A
few primary sources may also be accessed, but only with the prior approval of
the course instructor/examiner. An unrestricted OBE is one where lecture notes,
worksheets, handouts and even books can be brought in. The choice of such
material is left completely to the student. The open resource/open library OBE
is like a take home examination; but in such contexts the questions/tasks set are
likely to be individualised and the evaluation stringent to avoid mass copying or
plagiarism.
Nature of Questions in Open Book Examinations
There are many kinds of questions (objective and subjective) that can be set in
OBEs. With a little care even multiple choice questions can be asked (see
Appendix, Set A). When needed, these can even be in the true false format, but
are likely to demand a justification for the stance taken. Labelling and
classification questions are of a similar nature (see Appendix, Sets B and C).
Questions that demand a subjective response could either ask for short answers
(see Appendix, Set D) or focused and specific responses (see Appendix, Set E)
or even a free response (see Appendix, Set F). In some contexts it is also possible
to visualise the creation of tasks/materials (see Appendix, Set G).
Such a shift, however, from questions that require reproduction of information
(regurgitation of knowledge) to the processing or even creation of knowledge,
is not without its disadvantages both for the paper setter and the test taker.
Disadvantages of Open Book Examinations
OBEs take a lot of time to create. A three hour paper may demand 10-12 hours
by the paper setter to conceptualise the tasks and then fine tune them. The
responses to Open Book tasks also demand a large amount of evaluation/
correction time. At the end of the day such effort pays off, for the responses will
provide glimpses into student minds, but the time required will have to be factored
in.
OBEs demand a different orientation not only from the test creator but also
from the test taker both in terms of preparation and in terms of planning for the
actual examination. As one of my students succinctly put it: “I spent most of my
time when I took my first few open book examinations searching in books for
answers rather than thinking.” Another added: “I did not know how to make notes
for an exam where I could carry my books inside or how to approach application
based and evaluation type questions and use reasons/evidences/examples to
substantiate my claim/opinion.”

There is no scope for memorisation in OBEs. In contexts where such examinations
are the exception rather than the rule this could be perceived as a big problem.
But if OBEs become the norm, the advantages would outweigh the disadvantages.
Advantages of Open Book Examinations
OBE questions, particularly of the problem solving kind mirror real life tasks; in
order to answer them the test taker has to access and use resources that are
usually available in academic and professional contexts. The education system
in India focuses on reproduction. As such, teachers are not always able to transfer
the essence of a problem to a new area/context; the OBE enables and facilitates
logical ways of thinking.
Such questions, therefore, influence the nature of learning. Students have to
read and study across the entire semester to understand and apply concepts.
The nature of ‘studying’ or preparation for an end of course/end semester
examination also changes. These examinations are not memory based and no
cramming is required. This actually reduces the cognitive (and also the emotional
stress) load on the student. As one of my students put it, “It hardly feels like an
‘exam’ in the conventional sense and does not carry the anxiety associated with
routine exams.” He went on to add: “It makes studying for an exam redundant,
which is how I think exams should be; (I only need to revise my materials and flag
what I may need).” Another student reiterated this by affirming that in an open
book examination he is creating and composing text rather than reproducing
memorised knowledge/text.
The absence of so-called ‘discipline’ issues is another major advantage of the
OBE. Students have no time to ask their friends for answers and copying is
practically impossible. From a teacher educator’s perspective, feedback on OBE
responses can be individualised, constructive and therefore genuinely
pedagogical in nature.
The advantages of OBEs outweigh the disadvantages, but they are not very
popular in India, whether in general or teacher education.
Unpopularity of Open Book Examinations
A general lack of awareness of possibilities could be one major cause for this
unpopularity. Another reason is that although the cognitive load on the student
while taking the examination is low the demands made on both the test taker
and the paper creator are high. The student needs to prepare differently and
diligently and the teacher has to spend a lot of time thinking about problem
areas and solutions. Question papers cannot be set overnight. However, OBEs
are the need of the hour in teacher education programmes in India and elsewhere.

Why are Open Book Exams the need of the hour?
A major problem with Indian teacher education is that it has no incubation period.
A teacher gets a degree and if he/she gets a job, then, without any hands-on
training or experience he/she is expected to start teaching. This is in direct
contrast to other professions like medicine, law, engineering or architecture
where there is an induction period between the time someone enters the
profession and begins working autonomously (Padwad, 2011).This implies that
there is very little professional assistance available on the job. OPEs can stimulate
this kind of ‘apprenticeship’ through its problem solving tasks. The conditions in
which they can learn for themselves can be provided. If this happens, then every
examination can become a learning experience, albeit in the context of testing
maximum performance rather than typical workday conditions.

Appendix: Sample Tasks
Set A. (multiple choice)
1. English is Greg’s native language. When he is four years old, his family
moves to another country, where Spanish is spoken. He learns Spanish,
becoming fluent. What phenomenon is Greg exhibiting?
a. simultaneous bilingualism
b. successive bilingualism
c. partial bilingualism
d. subtractive bilingualism
2. Bilingualism does not enhance the following metalinguistic ability:
a. early word-referent distinction
b. phoneme substitution
c. analysis of ungrammatical sentences.
d. counting of words in a sentence
Set B. (state whether true or false with justification)
P. Bilingualism as a phenomenon originates in individuals and is then
reflected in society.
Q. Societal bilingualism implies that all members in that society are bilingual.
R. A strong system of bilingual education, in a society with horizontal
bilingualism, will lead to additive bilingualism.
(Sets A and B were used in a course titled Bilingualism and Bilingual Education)
Set C. (labelling/classifying, with justification)
1. In the discussion of BICS and CALP, it has been stated that the difference
between the two lies in the interaction between two characteristics –
context reduced/context embedded and cognitively demanding/
undemanding. Given below are three activities. Using these
characteristics explain whether they fall into the category of BICS or
CALP. Give reasons for your answer.

a. receive a telephone call and take down the message for someone.
b. write an instructional manual for an electronic equipment
c. phonetically transcribe commonly mispronounced words.
2. Given below are 3 statements. State whether each one belongs to the
Hallidayan [H] or to the Chomskian [C] paradigm in linguistics; you may
use the terms formal/functional if you prefer. Justify your classification
in not more than 2 lines. Point form justifications are fine.
a. Language is a communicative tool that is reshaped by its users
according to their needs.
b. An individual’s actual language use may be error ridden, but this is
never a reflection of his/her language incapability.
c. Speakers modify their language to suit the context and ability of
their interlocutors.
(Set C was used in a course titled Linguistics for Language Education)
Set D. (Fill in the blanks and short answer questions)
(part of an open book quiz to check understanding)
Text Extract
Title: Errors and strategies in child second language acquisition
Author: Heidi C. Dulay and Marina K. Burt
Reference: TESOL Quarterly, 8(2), 1974
Abstract: The study attempts to determine whether the syntactic errors
children make while learning a second language are due to native language
interference or to developmental cognitive strategies as has been found in
L1 acquisition. 513 utterances containing errors were extracted from the
natural speech of 179 children, 5-8 years old, learning English as a second
language. Only those errors which could be classified without question as
‘interference’ or developmental’ were included in the sample. The results
indicate that first language interference accounts for only 4.7% of the
children’s errors, while developmental strategies account for 87% of the
errors. These findings suggest that less explicit teaching of ESL syntax to
children may produce better learning.
On the basis of your reading the study ‘Errors and strategies in child second
language acquisition’ by Dulay and Burt (1974) answer the following questions.

[I have reproduced only the abstract so question 3 may not be answerable
based on the information given above]
1. Which area of language acquisition does this study look at?
2. List the names of errors that the study looks at.
3. Who had coined the term ‘creative construction’? According to the study
what does this concept mean? Why is the concept referred to in the
present study?
(Set D was used in a course titled: Applied Linguistics for Language Learning)
Set E. (Focused, specific written response)
Write out the necessary instructions (that a student of language would need) to
complete the items given below. Use the cue given in brackets to help decide
what the task focus should be.
1. (spelling)
a. The men were seen on the parc bench. ( . . . . . . . . . . . . . . . . .)
b. We all went picnicing yesterday. ( . . . . . . . . . . . . . . . . .)
c. Hawe you sent the letter yet. ( . . . . . . . . . . . . . . . . .)
d. These children are very inpolite. ( . . . . . . . . . . . . . . . . .)
2. (right or wrong)
Venkat, Sundar and Savitri had planned to go for a movie on Saturday.
They met at Sundar’s house. But just as they were leaving for the cinema
hall, Sundar’s uncle came to visit him. Venkat and Savitri were
disappointed, but decided to go and see the movie together.
a. the three of them saw the movie. ( )
b. Venkat and Sundar saw the movie. ( )
c. Savitri and Venkat saw the movie. ( )
d. Sundar did not see the movie. ( )

B. (extract modified from the TKT sample Paper)
For questions 1-5, match the underlined mistakes in the learner’s composition
with the types of mistakes listed. Then create an exercise each for any 3 of
these errors that will help the student overcome his/her ‘errors’.
[1] Last summer we went on a lovely holiday. All family went
[2] together to a house by the sea. My family is horribly enormous.
[3] There were twenty of us so we rented a really big house that was having
[4] enough room for all of us, There were my brothers and sisters,
[5] my parents my cosins and their parents. It was great!
Type of Mistake
A. wrong vocabulary
B. wrong tense
C. wrong word order
D. wrong spelling.
E. wrong punctuation
F. word missing
Set F. (Free Response Tasks: Need to Apply and Synthesise Information)
Given below are descriptions of 3 tests that have been perceived as problematic
by the students who took them. Read the descriptions carefully and decide
whether the problem is one of validity or reliability or both. State this, and then,
write down what can be done to make each test as valid and reliable as possible.
You need not answer in complete sentences; point form will do.
P. This is an ELT course (teaching methods, maybe?) and there are students
who have taken ELT courses earlier; others have not. For some students
therefore, this course is the first orientation to ELT. The course does
not explicitly state that the students have some familiarity with ELT
concepts, but the questions in the final examination paper assume that
such knowledge is available.
Q. Interviews are being conducted for a job as a marketing manager in a
national company. Each candidate is asked 8-10 out of a fixed set of 20

questions. A total of 15 candidates are interviewed; all of them are asked
to sit in an ante room, and asked to go back and wait in that room until
all interviews are completed, for the interviewing is to be followed up
with a group discussion.
R. A village school conducts all its examinations, under a banyan tree,
which is where all classes are also held. Some parents feel that there
has been some discrepancy and protest to the public education ministry.
(Part A of Set E, and Set F were used in a basic Language Testing Course)
Set G. (Task/Materials Creation)
Along with questions on methodology, (closed book) candidates are asked to
pick a paper from a set of slips on the table (which has level and nature of lesson
stated), use the library for an hour to collect resources, and then create a
complete lesson plan along with texts, tasks etc. The resources used must be
cited, but need not be reproduced. (Adapted from a teacher training examination
in Poland)
References
Foucault, M. (1975): Discipline and Punish: The Birth of the Prison. New York: Vintage.
Trans. Alan Sheridan (1995 edition)
National Curriculum Framework. (2005): www. http://www.scribd.com/doc/
43903533/National-Curriculum-Framework-2005-NCERT-India (accessed 17
December 2012)
National Curriculum Framework for Teacher Education: Towards Preparing
Professional and Human Teachers. (2009): New Delhi. NCTE
Padwad, A. (2011): ‘Professional Development – Why Bother?’ in Gunashekar, P.,
Padwad, A. and Pawelec, D. (Eds) Starting, Stimulating and Sustaining English Language
Teacher Education and Development: A Selection of Papers presented at the
International Conference in Hyderabad in January 2011. British Council

Evolving a process for Continuing Professional
Development of Teachers in Remote areas through
SMS
Nivedita V Bedadur is working as Specialist, Academics and Pedagogy
(English) in the University Resource Centre of Azim Premji University,
Bangalore
This study engages with the question of how the SMS feature of mobile phones
can be used for continuous professional development in English language where
the context does not provide opportunities of engagement. Mobile phones are a
popular medium of communication and language is primarily communication and
engagement. We are trying to understand the process of using a viable alternative
tool/ process for engaging with English as a Second language for teachers who
have no source of exposure to English. The study explores the hypothesis that an
everyday contextual task-based exposure to language will trigger the motivation
and confidence of the teacher and create conditions for moving towards
communicative engagement with the language in learning networks. SMS is a
tool promoting group culture and cooperative learning as it can be easily
forwarded and shared. How can this be explored as a learning tool? We look at it
as a process of continuous learning. What is the social, personal, communicative
and cognitive process by which the engagement with English Language learning
is activated in rural remote areas with teachers? How can teachers form learning
groups while engaging in texting in English? What is the process by which they do
so? These are the questions that we will explore through this paper to arrive at a
model of m-learning (i.e. learning with handheld devices 9(mobile phones) using
the SMS feature of mobile phones.
Why M-Learning?
The knowledge commission report on English states: ‘The nearly four million
school teachers all over the country, regardless of their subject expertise,
especially teachers at the primary level, should be trained to improve their
proficiency in English…. Most teacher training programs are not based on a real
assessment of needs of teachers…. Language learning opportunities should also
be created outside the classroom. (Knowledge Commission Report 2006: 8).
This objective is far from being achieved today, six years after the publication of
the report. It is a well-known fact that regional medium school teachers and
rural private school teachers lack proficiency in English, even today. Sixty percent
of the teachers mentioned in the knowledge commission report are working in
State government, regional medium schools which are geographically dispersed,
socially disadvantaged and mainly in rural areas. To these teachers mobile phones

are the only accessible and affordable window to the outside urbanised world
of knowledge and skill building: opportunities and advantages.
Additionally, more than one-third of the world’s adult population – most living in
the developing world – has no access to printed knowledge, new skills, and
technologies that could improve the quality of their lives (Dhanarajan 2009: 46).
Inequalities in access to education continue to pose major barriers in the
developing world, and the delivery of cost-effective and quality education remains
a persistent problem.
All over the world research in mobile learning has generated an interest amongst
scholars and researchers towards the concept, pedagogy and practice of mobile
learning. This paper makes a case for maximising the potential of m-learning for
teacher capacity development by developing a model for m learning based on
the authors small experiments conducted in rural and remote areas.
Why M-Learning? — my context
I work with 30 state board school teachers in the areas of Narayanpur, Kembhavi
and Surpur in the Surpur district of Karnataka for their English language capacity
development. These are disadvantaged districts in the context of education.
The occupations of the people are mainly agriculture and small business. Many
of the students and teachers are first generation learners. The schools are
situated in remote rural areas in the midst of farms and fields. There is no evidence
of written material in the form of pamphlets, sign boards, hoardings,
advertisements in the areas where the schools are situated, no hoardings, no
newspapers, no access to informal acquisition of literacy in the school language
and none in English. The thirty teachers I work with are interested in learning
English but have no access to the language beyond the classroom. Their school
life is ridden with problems. This creates a vacuum in the teacher’s life due to
lack of appreciation and motivation for self-development. The teacher is an adult
learner who is afraid of exposing herself to ridicule in face to face learning
situations. She hesitates to come forward and take advantage of the workshops
conducted.
Language learning needs continuous engagement and face to face modes of
learning have to be supplemented. Moreover, dependence on face to face
learning and lack of awareness regarding continuous development avenues
creates a situation of unequal access and teachers need to be weaned from top
down engagement. Faced with the problems outlined above I discovered that
the teachers I work with had only one means of daily engagement with the
language and that was a mobile phone. This led to the thought of using SMS for
daily language support with the dream of creating language learning networks.

My experiment with mobile learning
The experiment with mobile learning was conducted with ten teachers randomly
selected from the thirty teachers I was working with. These experiments have
been described in detail in a paper published in a book brought out by British
Council titled ‘Continuous Professional Development- Lessons from India’.
The experiment went through three phases which began with using SMS for
English language learning. In the next phase small networks were formed and
teachers began to experiment with pedagogy. As this was a mobile device I
believed that it could be used to create communities of learners. I tested the
concept through a (first) pilot project conducted with 10 teachers of two schools
for children of migrant labourers run by Azim Premji Foundation in Bangalore.
The second pilot on SMS-based interventions was conducted with a group of
teachers from State Board Schools. I decided to send an SMS every day to 10
participants who were randomly selected for the second pilot. The objective of
this pilot was to test whether the engagement level of the teachers from these
remote and rural areas could be sustained if the project was run by one teacher
pushing the content to other teachers (Bedadur 2012: 88,91).
Towards conceptualising a model of m-learning
This paper outlines the process of m-learning that was observed during these
experiments. We then conceptualise a model of learning through mobile phones
in rural areas in India as a result of the above mentioned experiments. In my
experiments of learning through SMS, teachers did not sustain their interest in
answering the SMS messages every day. However, after one month they formed
networks of SMS senders and one teacher took up the responsibility of sending
messages to other teachers. This formed small networks. The same teacher
began an SMS programme with his students. He started sending messages to
his students and the students replied. This became a practice which he shared
with other teachers. Other teachers showed keen interest. Soon they became a
learning community. They explored other means of using a mobile and sharing
learning.

Pedagogy
Pedagogy / Pedagogy How did it How did it Ease and
content support support Frequency
learning? creation of of use
learning
networks
I sent a Vocabulary They learnt They learnt Needs time to
message inputs new together and explore and
containing an vocabulary in exchanged creates interest
input context news and but does not
introducing a views regarding sustain
vocabulary their learning. engagement
item with a They consulted
question which each other for
required answers.
response every
day for one
month
One of the Vocabulary They learnt They learnt by Self-driven and
participants inputs vocabulary forming a therefore
began to send chosen by group and creates longer
a message their learnt through engagement
every day to colleagues sending as
their which was well as
colleagues for more relevant responding
one month to their context
The participant Vocabulary Creating a They learnt Creates lots of
began sending inputs of message – through excitement and
messages to student level reading a sharing and sharing among
his students message – collaboration young learners
responding to as responding
student queries to the
and messages message was
a challenge
The Shared Teachers learnt Teachers Spontaneous
participants of classroom best/ new shared use
the networks videos practices from innovations
began each other and
experimenting innovative
with other ideas
mobile devices
like the mobile
phone camera,
digital camera

Figure 1 below is a model of creating learning networks through SMS messaging
or m-learning:
My network of ten teachers led to an outer circle of ten more teachers and
when one of the teachers started sending SMS messages to his students they
developed the potential of an expanding outer circle of students and student
networks.
As the teachers started sharing SMS messages new ideas emerged and teachers
started experimenting with technology and learning. A network of learning
community was in the making.
Conceptualising a process for M-Learning in rural areas: Key elements
and expanding horizons
Given below is a model of m-learning that emerged through the process that my
learners went through. In this model SMS acted as a trigger to create interest in
learning. This interest was not sustained for long in the group. When teachers
created their own network the interest was reborn. Teachers shared their learning
in the process of sharing a message. This sharing created the awareness that
they could also drive their own learning and create their own network. Teachers
dared to experiment with new ideas using SMS to try it for students’ learning.
Recognition and sharing of this practice led to expanding networks and innovative
ideas being explored.

Figure 2
Reflection
Learning through mobile devices happens when expert directed synchronous
learning becomes learner directed asynchronous learning. M-learning has the
potential to create learning networks as it is by itself a networking tool and
lends itself easily to sharing and forwarding. Researches in this area substantiate
these findings as quoted below.
Given that social interaction is central to effective learning, as indicated by
theories of new learning, mobile phones should also impact educational
outcomes by facilitating communication. Mobiles permit collaborative learning
and continued conversation despite a lack of physical proximity and thus advance
the process of coming to know, which occurs through conversations across
contexts and among various people. Via mobile technology, learners engage in
conversation whereby they resolve differences, understand the experiences of
others, and create common interpretations and shared understanding of the
world (Nyiri 2002; Sharples et al. 2007: 225-26 as quoted in Valk, Rashid and
Elder 2010).

Pedagogical Model of Mobile learning with SMS for continuing
professional development
Figure 3 above illustrates pedagogical model of m-learning using SMS involves
a demonstration which would act as a trigger. The ‘teacher’ would then step
back and allow for ownership to emerge through exploration. This would lead to
acceptance of the model leading to new experiments in learning through
networking and sharing.
Reflection and Concerns
My experiments with m-learning have led to designing a model of learning where
the learner moves through a top down teacher managed programme to creation
of learning networks leading to an autonomous self-created learning environment
which will lead to continuous engagement with learning. It is, however, yet to be
seen whether m-learning networks sustain their engagement over long periods
and the learners become autonomous or whether they lose interest in learning.
It is also a matter of concern that the content may come from sources which
may not be authentic when learners create their own mobile networks as they
direct how they learn as well as what they learn. We cannot monitor the variety
of English the teachers are learning if they are learning on their own. It is also

difficult to record whether learning is happening or not, and to assess learning
- language learning is related as much to culture and social usage in such a
context. This creates limitations for learning through mobiles.
References
Bedadur, N. (2012): ‘A Design Based Study Using M Learning for CPD.’ Continuing
Professional Development Lessons from India. New Delhi: British Council: 88,91
Dhanarajan, T. (2009): ‘Education for all in the digital age’ in Akhtar, S. & Arinto, P.
(Eds.), Digital Review of Asia Pacific 2009-2010, pp. 45-49. New Delhi: Sage
publications
Government of India. (2006-2009): The National Knowledge Commission Report to
the Nation. Available at http://knowledge commission.govt.in/reports/default.asp
(accessed 4 December 2012)
Valk, J-H., Rashid, A T. and Elder, L. (2010): ‘Using Mobile Phones to Improve Educational
Outcomes : An Analysis of Evidence from Asia Pan Asia Networking’, paper published
in IRRODL Vol 11, No 1, Canada (www.irrodl.org/index.php/irrodl/article/view/794/
1487) last accessed 4th December 2012

Using classroom presentations as a tool for self-
assessment
Padmini Boruah, Associate Professor and Head of the Department of
ELT, Gauhati University, India
In our MA ELT programme, learners are encouraged to develop self- and peer-
assessment tools for Powerpoint presentations on syllabus components. The
assessment criteria, created by the students themselves, are based on three
aspects: content, slide show and language of presentation. Collated grading and
constructive feedback follow. This strategy has many benefits: learners develop
self-confidence; the visual presentation makes learning points easy to remember;
and learners are empowered to take responsibility for their own assessment.
Such strategies help in making teaching and testing converge in a shared space,
and demonstrate learner autonomy in assessment practices.
The background issues: traditional models of assessment
In many university contexts such as the one I work in, post-graduate programmes
follow traditional modes of assessment, the criteria for which are set by the
university, with little room for improvisation. In 2001, our university moved from
the annual to the semester system of study, which introduced the concept of
ongoing assessment through an ‘internal’ component in the testing system. In
2010, the university introduced Choice Based Credit and Grading, which replaced
the traditional system of assigning individual marks to students with relative
grading across a batch. These modifications were based on the belief that judging
learners’ abilities on the basis of one summative, ‘external’, end of term
examination is both an unhealthy and impractical practice, because it does not
reflect students’ learning development over the academic term.
Even with these new policies, however, our assessment system has not been
able to get away from a top down model, one in which decisions about what and
how to assess are taken centrally. The tools of assessment include the end of
term summative ‘external’ examination that carries 60 to 80 per cent weight,
and the ‘internal’ component comprising sessional examinations, class tests,
home assignments, class attendance and participation in class discussion that
makes up the rest of the grade. It is assumed that these tools suitably gauge
learners’ understanding of content knowledge and their ability to express this in
a written mode. In all these traditional models, the assessment criteria are set
with little or no learner involvement. Consequently, the expectation that these
modifications in the assessment system will lessen or remove exam fear, and
reduce the accompanying psychological burden on exam takers, remains a
distant dream.

In this system of testing, the only area that seems to have some space for
experimentation is that component loosely termed ‘participation in class
discussion’. In the past few years, I have tried to bring in learner participation
into the assessment system by experimenting with an alternate mode of testing
in this component, in the third semester of the M.A ELT programme. Since teaching
in a six month semester is always a tight schedule, I linked this experiment to a
syllabus component, and used the model to develop criteria for self and peer
assessment. The structure of the M.A. programme, detailed below, has allowed
this kind of experimentation within the internal assessment component,
contributing significantly to learner autonomy in both learning and evaluation.
Course structure
The M.A. ELT course of Gauhati University is part of a unique post-graduate
programme. Named M.A. (LELT), this is an integrated four semester programme
incorporating elements of Linguistics and English Language Teaching, and is
offered jointly by the Departments of Linguistics and ELT of Gauhati University.
The programme is open to graduates of all disciplines, and admission is based
on a compulsory English proficiency test. There is no undergraduate ELT or
Linguistics programme in colleges affiliated to Gauhati University, or any other
university in the region, so students aspiring to study these subjects at the post
graduate level do not usually have any background disciplinary knowledge. To
help them make a more informed choice, students are admitted into the joint
M.A. (LELT) programme, in which they study basic aspects of English linguistics
in the first two semesters. The programme then bifurcates, and based on their
area of interest, students can decide whether they want to work towards the
M.A. (Linguistics) degree or the M.A. (English Language and ELT) degree in the
next two semesters. Students opting for the M.A. in English Language and ELT
have to take courses on foundational aspects of language acquisition and
language teaching such as methodology of teaching English, materials design,
testing and evaluation, English for Specific Purposes (ESP) lesson planning,
projects in ELT and practice teaching, all of which are assessed through the end
of term ‘external’ exam and the ongoing ‘internal’ exam.
Modes & tools of assessment in the M.A. LELT programme
In the M.A. ELT course under the M.A. LELT programme, the main mode of testing
is the summative, end of term, written ‘external’ examination. Like any other
university exam, this kind of evaluation involves strict anonymity at all levels,
from paper setting to assessment, ostensibly to avoid subjective marking, undue
favours to select students and unfair assessment. There is no scope to provide
learners with feedback on their exam performance, since the answer scripts
constitute university property and are taken through a series of rigorous
processes – marking, scrutiny, calculation of grading formulae, tabulation – before
the marks / grades are printed on report cards. In addition to the theoretical
components that are tested through a written exam, the M.A. programme includes

a practical paper called Practice Teaching, but this is also graded by an ‘external’
examiner.
The ‘internal’ evaluation procedure, which is meant to be part of formative
assessment, comprises sessional tests, home assignments, class tasks,
discussions and learners’ classroom behaviour. The first three are tested through
written exams; these are graded by teachers who also provide feedback on
learners’ performance. Classroom behaviour and participation in discussions
are difficult to quantify, so the marking is subjective, based on the extent to
which the teacher is satisfied with learners’ responses in the class. For all of
these, the criteria for grading are set by the individual teacher.
The ‘other’ perspective: an experiment on assessment as a self-learning
tool
In a restrictive evaluation system like this, with pre-determined grading criteria,
there is little scope to make learners accountable for their own learning
development, or to help them map their learning progress. Within the ‘internal’
examination component of the M.A. ELT programme, however, I realised that
classroom discussions on a syllabus topic might present opportunities a little
experimentation.
For several years, I had been giving students a presentation assignment, to be
done individually or in pairs, on the different approaches and methods in language
teaching as described in Richards & Rodgers (2001). For my experiment, I decided
to ask students to make the presentation on Powerpoint. Presentations, especially
those given through Powerpoint (ppt), give learners practice in both oral and
written mediums of communication, and offer a space for immediate feedback.
Since I wanted student presentations to serve as self- and peer-assessment
tools, and yet taken seriously by learners, I announced that the grades they
gave to their peers would be included in the internal assessment for Third
Semester.
I explained that this would serve as a good learning experience, as it would help
learners study the content carefully, become familiar with multimedia
presentations, and help develop oral proficiency. They would learn seminar skills,
such as asking for and responding to clarifications, giving feedback, and other
discussion skills, all of which would help them develop self-confidence. I
announced that we would develop the criteria for assessment together, so that
they could take it as an exercise in self- and peer-assessment. My aim was to
find out the efficacy of assessment as a self-learning tool used by students,
rather than on students. The response of the students was encouraging; they
were enthusiastic about the challenge of making Powerpoint presentations (for
most of them this would be the first time), and about being consulted in the
assessment process.

Since this was the first time students were doing an assignment like this, we
decided they would prepare for it and present it in pairs. We also decided to
have just one presentation a day - a thirty minute presentation followed by thirty
minutes of discussion in a one hour class period. Taking feedback from peers
and teacher is integral to the learning process, so we decided to spread the
preparation over a couple of months, as this would give the class enough time
to prepare for the assignment thoroughly. Self-learning was encouraged, and I
asked them to use the internet to surf for learning tips on both the content and
style of the presentation. We set up the criteria for assessment before the
presentations began, so that the students could use these as guiding principles.
The criteria were based on three aspects: the content, the visual impact of the
slides and the oral support (verbal presentation), with ten marks for each criterion.
The assessment would include peer grading and then collation of grades with
the tutor’s. Another ground rule was that each student would have to read up
the topic of the presentation to be made by their peers, so that the feedback
would be constructive, and could be used to modify the presenter’s slides. This
was important, as I felt students who did not do too well could have an option of
presenting again, and the feedback they received could be incorporated for the
next (optional) round. I wanted the students to take this experiment as a learning
experience that could help improve both their content knowledge and
presentation skills.
Learner feedback on engaging with assessment
This experiment has been working successfully for the last three years, and the
learners have claimed that the experience has helped them develop both their
language proficiency and understanding of the content. As their tutor, it has
helped me step back and let learners take responsibility for their own learning
development. The experience has also helped students in subsequent academic
activities – facing a group of listeners has made them more confident in their
practice teaching classes in a local school for their fourth semester course
requirement. Alumni who have made seminar presentations in national and
international conferences have received encouraging feedback from senior
faculty from other universities, and current students who attended these
conferences later reported how motivated they were to see their seniors receive
such positive comments.
Here are some of the comments made by students in video-recorded discussion
sessions on their experience of Powerpoint presentations:
• “We remember the contents better because we see them on the slides”
• “We learn not to crowd our slides with too much text”
• “We learn to ask questions…and also answer them”

• “We become more confident”
• “We enjoy the presentations”
• “We think it is better than taking an exam…”
Impact of self and peer assessment on learners and learning
The experience of making visual presentations on a curricular component has
made a positive difference in learners’ attitude to assessment. Since the criteria
for assessment were set in consultation with them, and the test was administered
democratically in a non- threatening atmosphere, learners reacted positively to
the experiment. This participatory approach to learning accountability had added
benefits: the learners knew their performance would reflect in semester grades,
hence they took it seriously. Several learning points emerged from the
experience, such as the value of visual impact, which made learning points easy
to understand and remember; students learnt both presentation and critiquing
skills, and Powerpoint ‘smart tips’ in context. More importantly, learners were
empowered to take responsibility for own learning as well as that of their peers.
With their newly acquired skills, they could now also replicate it for other
assignments.
Self and peer assessment as a learning tool/strategy
When considered from the ‘other’ perspective – from the point of view of learners
- assessment can prove to be a valuable learning strategy. Self- and peer-
assessment are reflective activities that nurture language and content
development. In a teacher education course like ours, this also demonstrates
effectively the benefits of formative assessment based on constructive pedagogy
and experiential learning. If the criteria for evaluation are formulated by and
agreed upon by students, they are likely to be based on students’ perceived
needs, and what they feel is important for them to learn. Such a procedure
follows a bottom-up approach: it allows the important questions of why, how and
what to assess to converge meaningfully, while assigning priority to how the
process will feed into learners’ learning graphs. It also brings in a sense of fairness
and transparency, being grounded in principles of inclusive pedagogy. As a result,
evaluation becomes for learners an enjoyable rather than an overwhelming
experience.
References
Richards, J.C. & Rodgers, T.S. (2001): Approaches and Methods in Language Teaching
(Second edition). Cambridge: Cambridge University Press

Assessing the Critical Language Awareness of ESL
School Teachers
Deepesh Chandrasekharan is a PhD (ELE) scholar at The English and
Foreign Languages University, Hyderabad
Schools aim to help learners make sense of the complexities in society and prepare
them for life’s challenges. Language is both a tool to grasp the world and a site of
social conflict. In plurilingual India, where language is a complex identity-marker
and politics complicates all language issues, and where English is perceived either
as a threat or as a passport to success, raising learners’ Critical Language
Awareness (CLA) can be empowering. Language teachers’ CLA is then crucial,
but do they have a critical understanding of the issues at hand?
The languages situation in India
Language is not a politically neutral, merely communicative tool. Even in largely
monolingual contexts, speakers use different varieties of language and questions
of power and identity arise. These questions become more important in elite
multilingual countries like Canada. The questions of language, power and identity
are so complexly enmeshed in grassroots multilingual societies like India, that it
is impossible to ignore them in any language education programme.
The language scene in India is complicated by several individual, societal and
policy-based realities. Language has been an important marker of identity since
the end of the 19th century (Mitchell, 2009) Languages are given an exalted
status (Telugu deified as ‘talli’ or mother-goddess) and several individuals have
even committed suicide on issues concerning language (Potti Sriramulu’s fast-
unto-death for the formation of a Telugu-speaking state).
At the societal level, India has seen grassroots multilingualism and people use
different languages to perform different functions in society. So, it is quite
common, for example, to find an individual in Hyderabad speaking Marwari at
home, Telugu or Urdu with vendors in the market and English in the school or at
the office. Languages share a complementary relationship in a fluid, non-
competing way.
At the policy level, the constitution’s VIII schedule is a list of the languages
considered “official” by the Government of India. Though several deliberations
have happened over the issue, no language is called the ‘national language’ in
India. Though the constitution provided for Hindi to become the official language
of India in 1965 (15 years after the constitution was adopted), anti-Hindi agitations
led to the continuation of English being considered the ‘associate official’

language and presently, all languages in the VIII schedule (presently 22
languages) enjoy official status, state support and ample resources for the
promotion of language use, maintenance and propagation. The Census of India
is the richest source of language data for the country. The 2001 census had
6661 “raw returns of mother tongues”, “rationalised” into 1635 mother tongues
using “thorough linguistic scrutiny, edit and rationalisation” and grouped together
as 122 ‘languages’, using “the usual linguistic methods for rational grouping
based on available linguistic information” (Census of India 2001). The arbitrariness
of the terms ‘mother tongue’, ‘language’ and ‘scheduled language’ in the census
and the VIII Schedule is amply clear from the fact that Maithili was only a ‘mother
tongue’ grouped under the ‘language’ Hindi up to the 1991 census and became
a ‘language’ by itself in the 2001 census, after it was included in the VIII schedule
of the constitution through an amendment in 2003. Sanskrit spoken by 14,135
speakers is included in the VIII schedule but Bhili with 9.58 million speakers and
seven other languages with more than a million speakers are not (Constitution
of India 2007: 330).
To further complicate the landscape, there is the question of the status of English
in school education in India. English is the ‘associate official language’ of India
but is not listed in the VIII Schedule. The knowledge of English is increasingly
perceived as a passport to success and the idea that direct instruction in English
from the earliest years of schooling is best has gained currency, even in states
like West Bengal and Gujarat which have had governments politically opposed
to the early introduction of English-medium instruction in schools.
Even the question of which language variety to use in the classroom is a significant
one. On the question of ‘Standard English’, several scholars have pointed out
that the notion of a standard is often more ideological than real, and that teachers’
insistence on standards, which is based on their bias against certain varieties,
can negatively affect the chances of minority language speakers to succeed in
educational settings (Lippi-Green, 1997). Some point out that adherence to
standards is even used to assess people’s intelligence and moral worth (Clark &
Ivanic, The Politics of Writing, 1997, p. 189) and ask for a critical scrutiny of
standardisation tendencies and non-conformation to the dominant conventions
(Clark & Ivanic, The Politics of Writing, 1997, p. 191).
The Study
Is the grassroots teacher of English aware of this complicated reality?
This study is an attempt to capture English teachers’ critical understanding of
the issues concerning language, power and attitudes towards the statuses of
the different languages found in the school’s immediate context. It focussed on
the notions of mother tongue, national language, multilingualism, Standard
English and medium of instruction in schools. It was conducted at a privately-

run secondary school which uses the curriculum recommended by the Andhra
Pradesh state government. Twenty-two teachers teaching English at the primary,
middle or secondary school level took part in the study by responding to a
questionnaire which had several open-ended questions.
Teachers’ Responses
A close look at the teachers’ responses gives us a clear understanding that they
have a very simplistic view of the languages situation in India and are unable to
grasp the complexities involved. A sample of the kinds of responses given by
the teachers follows.
The teachers give esoteric definitions of a mother tongue like the language
“heard in the mother’s womb” (R-13) or as dependent on the “tradition” (R-19)
they were born into. One teacher calls a language her mother tongue, because
it teaches her to do well in life (R-2).
On the question of a National Language, teachers expressed some inconsistency
when those who called for Hindi or English to be the National Language gave
similar reasons for their choice: widespread reach and easiness to learn. They
often exhibited recourse to emotion. For example, one teacher referred to Hindi
as rashtrabhasha (‘national language’, R-10) and another justified the use of
English as National Language because “the world is running behind (sic) it” (R-
9). Some teachers called Hindi a National Language elsewhere, whereas official
policy documents refer to Hindi only as official language.
A few teachers actually saw multilingualism as a disadvantage by asserting that
knowledge of English is enough or by blaming multilingual education policies
for some learners’ inability to be proficient enough in English. They thought that
time that could be used for other school subjects is wasted because learners
have to concentrate on many languages.
When asked which languages are important for children in their area, the teachers
ranked English, Hindi and Telugu in that order as most important. They chose
economic, religious and emotional reasons (and not linguistic, historical or
political reasons) as most important factors determining the importance of the
languages. Those who ranked English as most important did it because it is a
‘global’ language. One teacher wrote that to learn other languages is “useless in
the future” (R-15).
Seventeen of the twenty-two teachers recommended English as the medium of
instruction in schools and said this is because English is the “world’s most
important language” (R-19). One teacher highlighted the perception that English
is the language of higher tastes and culture when she justified English-medium
education by saying that there is need to “culture the upcoming generation” (R-

9). Some of the teachers also mentioned the fact that we live in a “computerised
world”(R-13) and that “parents are impressed” (R-12) if the school is an English-
medium one. Most such teachers wanted English as a medium of instruction to
begin either at kindergarten or class one level.
When asked how important it is for school students to use ‘Standard English’,
most teachers felt it is very important. Some of the reasons they gave were that
the use of Standard English “can secure a good future” (R-14), lead to a “good
status”(R-14) in life, help learners do well “in competitive exams”(R-7, R-10) and
to “make parents proud” (R-22) of their children.
Analysis
Why is it that the teachers, for whom the complex languages situation in India is
a reality that they live and not a community they imagine, seem incapable of
understanding the complexities involved?
Teachers treat each language as a separate compartment, as a system of rules
different from other systems of rules or merely as a politically neutral tool of
communication. This is not because they are incapable of thinking about the
complexities of language, power and identity found around them, but because
teacher education fails to look at the political aspects of actual language use.
There is a gap between what is taught in the teacher education programmes
and the reality found in Indian society.
Critical Language Awareness as the solution
The solution readily available to bridge this gap is Critical Language Awareness
(CLA). CLA is a critical understanding of the ways in which language represents
the world, and reflects and constructs power relations (Clark & Ivanic, 1999).
Such an understanding, developed and given shape in the formal setting of the
classroom, can help learners gain fresh perspectives on the issues, and not be
blindly prejudiced or impulsively judgmental.
Even at the micro-level of language use, language is hardly neutral and the use
of particular expressions creates particular shades of meaning. CLA helps
language users become sensitive to the variety of ways of expressing meaning
that are available and to choose expressions, being fully aware of the political
load of meanings they carry and to also take responsibility for the choices they
make. A simple example is the use of “he or she” or “she or he” instead of “he”
(or him and his) as a generic pronoun for all humans.
For this sensitivity to be raised among learners of languages, it is obvious that
teachers need to have a higher level of awareness of the complexities of the
languages situation in India apart from having a deeper sensitivity to the hidden
meanings that the use of certain language expressions generates. Since the

discourse in the classroom is also largely regulated or controlled by the teacher,
what the teacher thinks and believes is extremely important.
Teacher practice is shaped by their theoretical understanding of the notions of
language, culture, teaching and learning. This understanding is shaped and
informed both by ‘official’ theoretical input from pre-service teacher education,
professional development activities etc and also from their own experiential
knowledge, both as teachers of language and also as language learners
themselves. Teachers do have the power to do their bit in raising awareness
among learners even with curricula which don’t take such activities into
consideration. As Suresh Canagarajah argues, “at the micro-social level of the
classroom... teachers and students enjoy some agency to question, negotiate,
and resist power...” (Canagarajah, 1999, p. 211).
In the light of the teachers’ responses to the issues raised in the questionnaire,
we can conclude that there is need for a greater critical understanding of the
language issues. Teachers of English need to have a higher critical awareness
of the issues in order to look at them dispassionately and critically and also to
help learners develop a critical understanding of them. It is also imperative that
the teachers empathise with the situation of the speakers from minority languages
and non-standard varieties in order to help them involve in the learning processes
more fully and with confidence.
Recommendation
There is a strong case for the inclusion of CLA components in teacher education
programmes in order to help prospective teachers develop an informed
understanding of the politics of language use in society. This is imperative in a
country like India, where in addition to the complexities mentioned above,
minority groups are made to feel marginalised adding to their existing anxieties.
However, there is a need for caution. In the enthusiasm to introduce CLA, teacher
educators must ensure that the teachers’ opinions are not imposed on the
learners in an uncritical top-down approach. Instead, as in all programmes
including CLA elements, teachers need to create room for thorough discussions
that look at issues from many well-researched perspectives and allow the learners
to arrive at their own conclusions after much thought and consideration.
It is only when such awareness is developed among learners that true democracy
can be achieved in the languages arena and individual students from all linguistic
groups can be truly empowered.
References
Canagarajah, A. S. (1999): ‘On EFL Teachers, Awareness and Agency’ in ELT Journal,
53/3, 207-14

Census of India. (2001): Available at http://censusindia.gov.in/Census_Data_2001/
Census_Data_Online/Language/gen_note.htm (accessed 15 November 2012)
Clark, R., and Ivanic, R. (1999) ‘Raising Critical Awareness of Language: A Curriculum
Aim for the New Millennium’in Language Awareness , 8/2, 63-70
Clark, R., and Ivanic, R. (1997): The Politics of Writing. London: Routledge
Constitution of India. (2007): Available at http://lawmin.nic.in/coi/coiason29july08.pdf
(accessed 15 November 2012)
Lippi-Green, R. (1997): English with an Accent: Language, ideology, and discrimination
in the United States. London: Routledge
Mitchell, L. (2009): Language, Emotion, and Politics in South India: The Making of a
Mother Tongue. Bloomington and Indianapolis: Indiana University Press

Evaluation of the Class X Course Books by Classroom
Teachers in Bangladesh
Mian Md. Naushaad Kabir is a doctoral researcher in the School of
English Language Education at The English and Foreign Languages
University, Hyderabad
The National Curriculum and Textbook Board (NCTB) introduced CLT in teaching
English at the secondary level in Bangladesh in 1996. The impact of such an
educational innovation was implemented through teacher training, materials
development, and testing practices. However, the introduction of CLT and the
absence of explicit grammar teaching were criticised by lay people including
many stakeholders of education. Hence, NCTB introduced a new book for teaching
grammar though the earlier book was retained. The two books of the same set of
course materials followed two different approaches—communicative and
structural. This modification continues to create inconveniences,
misinterpretations and scope for improper implementation of the curriculum to
the teachers and the learners. This paper attempts to analyse and evaluate the
set of course books to find out how far it offers ‘support’ and ‘constraint’ for
practising teachers. Data were collected from 9 rural English teachers of
Bangladesh who responded to specific questions and used a checklist for
evaluating the course books. Most of them were found to have appreciated the
book on grammar. The paper discusses the implications of such findings and
suggests more in-depth training for teachers to enhance reflective and
professional practices such as evaluation of course books under development.
Hence, more practically oriented activities in the teacher training programmes
are recommended, though with some caution.
Introduction
As per the recommendations of the Curriculum Report (NCTB 1995), CLT
(communicative language teaching) replaced GTM and the Structural Approach
at the secondary level in class VI in 1996. To implement the innovation, English
for Today (a new set of course books for learners from class IV to X, hereafter
Book I) was designed based on CLT principles and attempts were made to test
learners’ communicative competence in both the formative and summative tests.
However, people criticised the absence of explicit grammar teaching and testing
forcing NCTB (National Curriculum and Textbook Board) to offer English Grammar
and Composition (a new grammar book, henceforth Book II), later used as a
reference book for the second part of the English course and the examination.
The composition, teaching and testing of Book II were based on structural
approaches. To justify the sudden innovation in the course book, the 2009 NCTB

syllabus was modified. Thus the two course books for class X learners followed
two different principles.
Construct of CLT
An attempt will be made here to show how the construct of CLT has been
ambiguously presented in various educational documents:
The curriculum delineates the target level of the terminal competencies to be
attained by the class X learners in terms of listening, speaking, reading and
writing. Specifications on the skill-wise target terminal competencies are detailed
in terms of sub-skills appropriate to each of the skills. For example, while providing
specifications on reading, it is stated that students should be able to understand
different written texts; read extensively with appropriate speed; skim and scan;
infer meanings from contexts; distinguish facts from opinions; draw appropriate
conclusions; recognise the significance of cohesive devices; and recognise the
functions of different punctuation and graphological devices (Kabir 2011: 33).
Besides these, the curriculum emphasises using ‘culture specific texts’, dialogues,
poetry and drama with integration of structures, topics/themes, functions,
situations, and vocabulary in communicative contexts. The explicit teaching of
grammar is discouraged. Topics are chosen based on context-suitability, age-
specificity, and learner-familiarity. Learner-centredness is emphasised. As for
vocabulary, high frequency words are chosen. While teaching them, connotation
is prioritised over denotation and presenting an L1 equivalent is discouraged.
While providing instructions for testing, suggestions are made to integrate
grammar and vocabulary with reading and writing in the summative test where
listening and speaking are not included. However, continuous assessment of all
skills including grammar and vocabulary is recommended. Scope for rote learning
and testing of explicit knowledge of grammar and vocabulary are discouraged.
Assessment should focus on the use of grammar within meaningful contexts.
The analysis of the question paper of the SSC examination, 2009 (Kabir, op. cit.:
35) reveals that the question papers (particularly for Book II) closely conform to
the modified syllabus (NCTB 2009) and the new grammar book.
Mismatches in documents and practices
The above discussion substantiates the view that the construct of CLT is ill-
defined because:
• the holistic perception and presentation of language learning in
developing the construct of CLT in the curriculum is not reflected in the
examination-oriented presentation of the learning items in the modified
syllabus; and

• the juxtaposition of the two course books following different principles
without any proper pedagogical or educational rationale creates
confusion in teachers’ perceptions of CLT.
Data Description, Analysis and Interpretation
In the light of such background, the author conducted a survey using specific
questions and analysed the course books using a checklist with 9 randomly
chosen semi-urban teachers of English from Bangladesh. Since all the teachers
did not respond to all the questions all the time, the number of the participants
who actually responded to specific questions is given in parentheses. The data
collected through them are presented in the following section.
Description and analysis of questionnaire
Question 1 (Q1) on training background and expectations
While asked about their in-service training experience, it was found that three
teachers (out of 8) had not received any in-service training, four teachers had
in-service training of 5-15 days and only one teacher had had a 6-week long in-
service training experience. One of them received training from the USA. When
asked about their expectations from the next training programme, everyone
wanted training on English language, on CLT and on continuous professional
development strategies. Even the teacher trained in the USA for 6 weeks wanted
training on teaching English communicatively. Data clearly show that the training
experiences the teachers had, were not adequate for effective teaching.
Q2 on perception of the communicative curriculum
We found 50% of the teachers (out of 8) were not familiar with the curriculum
objectives (COs). Those who read the COs could mention some of them, for
example, enabling learner to communicate in different situations, motivating
them to participate in various language activities, etc. However, one teacher
found the English first paper unsuitable for the learners though it was completely
based on CLT principles.
Q3 on finding and bridging the gaps between the curriculum and the course
materials
The teachers reading the COs were asked to find relevant gaps between the
curriculum and the textbooks. One teacher said, “Translations should be included
in the curriculum” while another teacher said that changes in the curriculum
happened because of changes in politics.
A supplementary question was asked to find out how to bridge the gaps. Some
of the relevant suggestions offered are presented below:
“Though speaking is one of the most important skills, it is not included for the

academic exams. So translations as well as speaking should be included in the
public exams.”
“The gaps can be bridged by introducing more grammatical terms and
grammatical explanation with the text books [sic].”
Q4 on suitability, strength and weakness of the course materials
When asked about the suitability of the course books for the learners, almost all
the teachers found them suitable as they were well designed, well printed, nicely
written, user friendly, and offered different types of texts. For two teachers, the
course books were loaded with too many lessons and units to teach within the
available time.
When asked about the suitability of the course books for the teachers, most of
them, except two teachers, replied positively stating they were suitable only for
well-trained teachers. Of the two, one teacher mentioned that learner’s beliefs
about rote-learning constituted a major obstacle against the successful
implementation of such materials. Another teacher said, “Most of the teachers
are in traditional teaching method and have no clear conception about the
textbook. So it is not suitable for the teachers.”
While talking about the strengths of the textbook, all of the nine teachers
appreciated it for creating scope for practice of all skills, selection of topic,
clarity of print, presentation of real life pictures and incidents though one teacher
said, “More structural exercise should be increased [sic].”
All of them spoke highly about Book II because it has rules and exercises. They
even commented that rules of grammar make the lesson easy, enjoyable, and
pleasant. They found the rules suitable for the learners. One teacher remarked,
“The book selected for the students is not self-sufficient for teaching grammar.
I would like to say that more rules should be included in it.”
About the weaknesses, all of them criticised the excessive number of lessons,
fewer grammatical and structural exercises, unclear pictures. Some of them
referred to lessons that are irrelevant for learners and unsuitable for their age.
One teacher said, “Most of the lessons are teachers- or reference book-
dependent. The reference books are not available in our country. It is very difficult
to present the lessons using aids as it is costly and teacher’s salary does not
support the fact”.
Q5 on teachers’ choice on teaching method and their rationale
The following options related to the teaching of prepositions were given to the
teachers to understand which option they tended to follow:
A. Describe the rules of prepositions and then make the learners practice.

B. Show the learners different pictures having different relationships
between the objects and ask them to explain them orally. Invite others’
feedback on one’s answer. Ask them to talk about different objects of
the classroom. Finally, ask them to guess the rules and give feedback.
C. A mixture of both: how and why?
D. Others, and why?
We found two teachers out of eight opted for A (25%); two for D (25%); and
three for C (38%). Option B was not chosen though it completely reflected
communicative principles. One teacher (12%) did not reply. Interestingly, when
some of the responses for C and D were analysed, it was found they actually fall
under option A, as follows:
Responses for option C:
“First I describe the rules and then say that the rules are put into the
note book. After completing note, they memorise and practice [sic].”
“Preposition always is very important parts of speech and without
preposition the meaning of the sentence is not possible clear to express
the sentence. So the conception of the rules of preposition should give
grossly to the students [sic].”
Responses for option D:
“At first it has to be acknowledged that the group of preposition is very
important so the students should be taught about the phrasal preposition,
verbal preposition, appropriate preposition [sic]”.
It is evident that teachers’ choice towards teaching prepositions is expressed in
terms of structural approaches and memory-based learning. They were also
asked what they thought their learners would learn from this. Their responses
are as follows:
“A learner would learn from this exercise that more and more practice…
they (need to) memorise it …[sic]”
“Yes. The learners would learn easily because they have known the details
discussion [sic].”
“Yes I think they will learn because I follow the communicative
approaches”
“Learners should practise this exercise through which they would be
able to learn much.”

“Lecture, practice, and taking feedback make a learner clearly
understood regarding the topic.”
“Practice makes a man prefect. So after learning some rules, Ss start to
practise.”
Checklist
To get hands-on knowledge about the clarity of the pictures used in Book I
(pages from 100 to 126, the central part), the author asked the teachers to use
the checklist to identify people belonging to learners’ age, people from urban
and rural areas, their sex, religion, and class. The data are presented below:
Maximum number Minimum number
identified identified
Men 45 10
Women 88 23
Urban 102 10
Rural 34 1
People belonging to
learners’ age 56 11
Religion Islam
Social class Lower and middle class
None of the teachers could come up with the same response for any of the
items except the issue of religion. Their counting for each of the other items
differed. This suggests that the pictures are not clear for proper perception
though their responses to the questionnaire give the opposite picture. However,
religious bias was found.
Eight important aspects related to the course book were presented to them and
they were individually asked to sequence them according to their own priority.
When the responses were statistically collated, it was found that reading and
explaining the textbook ranked first; writing, second; actual use of English, third;
speaking, fourth; grammatical rules and translation, fifth; and finally listening,
sixth. Their cumulative preferences reflect their practices. Since reading and
writing are given more emphasis and speaking and listening are given less
emphasis in the textbook, teachers tend to prioritise reading and writing.
Interpretation
When the teachers were asked direct questions on their perception of the

communicative curriculum, their responses somehow matched with the COs (see
Q2) but when they were asked to find gaps and suggest measures to bridge the
gaps, their suggestions showed their preferences for grammar and translation
(see Q3). Furthermore, their appraisal or criticism of the materials is found to be
based on their structural approach orientation (see Q4). Their choice of the
teaching method clearly shows their preference, comfort or familiarity with the
structural approach (see Q5).
Through the checklist evaluation, we found that the poor printing quality definitely
fails to provide the teachers and the learners with visual stimulus, often required
for teaching a language communicatively. Their grading of some of the aspects
of the course books evidently shows their bias towards structural aspects, though
arbitrarily. It was more evident when they highly appreciated Book II and
suggested including more grammatical rules.
However, a critical reading, deeper understanding and triangulation of data with
other responses reveal what is happening in the practical field of English language
teaching.
Conclusion and Implications
The above discussion shows there is a random mixture of CLT and GTM and the
structural approach at the level of theories and practices in Bangladesh. The
situation is worsening due to the ill-planned presentation of CLT in the policy
documents along with the superficial conceptual orientations of the teachers.
The data were anonymously shown to the teachers in the post survey session.
They agreed that there is more to know about such innovation. They do not
know the rationale for such innovation. Some of them expressed positive views
towards GTM. Recalling their training experiences, they said that training sessions
were full of theoretical discussions and practical activities related to those
theories only. They unanimously agreed that they needed more in-depth
extensive training that would cater to the mismatches and misunderstanding
happening in reality. Another teacher said that the course materials sometimes
helped them and sometimes confused them.
Undoubtedly, materials, in this context, work more as a constraint than a support.
Prabhu (1988) once argued, how materials can offer ‘’support’ to teachers and
work as ‘constraint’ for them in their teaching practices, depending on teaching
contexts. One teacher proposed that they needed to engage in professional
activities such as evaluating course books, discussing academic issues for
professional development, etc. the way they did here. However, the only
precaution could be to avoid any situation where a blind man touches one part
of an elephant, and describes it to another blind man who disagrees with the
description of the elephant because he touches another part.

References
Kabir, M.M.N. (2011): ‘Class X curriculum of Bangladesh: inconsistencies in didactics
and praxis’. Sanshodhan (Special issue on international conference Prabuddhan in
December 2011): pp.32-6. Available at http://www.sanshodhan.net/
sanshodhan%20isue%205.pdf (accessed 29 November 2012)
National Curriculum and Textbook Board (NCTB) (1995): Curriculum and Syllabus
(Secondary level). Classes IX and X Report: 2nd part. Dhaka
National Curriculum and Textbook Board (NCTB) (2001): English for Today (for classes
IX and X). First paper. Dhaka
National Curriculum and Textbook Board (NCTB) (2005): English Grammar and
Composition (Second paper for classes IX and X). Dhaka
National Curriculum and Textbook Board (NCTB) (2009): Syllabus and marks
distribution of secondary school certificate. Dhaka
Prabhu, N.S. (1988): ‘Materials as support: materials as constraint’. Paper presented
at the RELC Seminar, Singapore

Training Needs Assessment of Secondary School
English Teachers in Aligarh
Raashid Nehal, Associate Professor, Department of English, Aligarh
Muslim University
The paper reports on perceptions of training needs of English teachers as part of
the study conducted under UGC Major Research Project (2010-12) on ‘Training
Needs Analysis of Secondary School English Teachers at IX and X level in Aligarh:
Implications for English Language Teacher Education’. The data collected were
based on a literature survey, schedules and interviews with teachers in a focus
group meeting in a teacher training programme. Ninety-nine teachers and 90
principals from 56 government secondary schools of Aligarh district blocks,
affiliated to state Board of Uttar Pradesh, participated. The primary data generated
identify training needs against the backdrop of lack of teacher training
opportunities in rural areas.
Objectives
Little research appears to have been carried out in rural areas and resource-
poor contexts concerning complex issues related to teachers’ perceptions and
their concern with ‘quality curriculum’ and effective teaching-learning
methodology’(NFG,1996). Keeping in view the mismatch that often exists between
the teaching contexts and the curriculum (NCF, 2005), the questions this paper
focuses on are:
1. What are the perceptions of teachers about their training needs?
2. How do local contexts and the administrative structure influence
teachers’ perceptions about their training needs?
3. What are the issues related to teachers’ assessment of restructured
curriculum and materials design in rural settings?
Methodology
Sample
Aligarh is in the northern Indian state of Uttar Pradesh. The city is located about
90 miles (140 km) southeast of New Delhi inhabited by a population of half a
million Urdu and Hindi speakers.
The sample shown in Table 1 belong to different blocks of Aligarh i.e.Lodha,
Bijauli, Iglaas, Gonda, Akrabad, Tappal, Gangiri, jawa,Khair and Chandausi.

Table 1
Teacher Principal No. of Blocks
Respondents Respondents schools surveyed
99 90 56 10
Different sets of schedules were administered separately with teachers and
principals. The teachers’ schedule was divided into the following sections.
• background Information;
• teacher’s workload;
• teaching language skills;
• domain-specific language use;
• language proficiency of the teachers;
• difficulty in evaluation of language skills, grammar and pronunciation;
• pedagogical and administrative challenges faced by the respondents;
• attitude towards the teaching profession, and
• teacher training and support.
The principals’ schedule examined the following areas;
• teacher attendance/ accountability;
• training sessions;
• types and duration of training programmes;
• special provision for SC/ST, minorities and the physically challenged;;
• resources required for teacher education; and
• infrastructural facilities.
Interviews were conducted with 99 teacher respondents in focus group meetings
as part of the teacher training workshop organised as part of the ‘Professional
Development of Secondary School English Teachers in Aligarh’ in association
with DIOS, Aligarh on 7-8 July, 2011 (UGC Project op.cit.).
Literacy-related problems
Only 10-12 per cent of students in a class are functionally literate to pass their
board exams under UP Secondary Education Council. Some teachers report
that 50 per cent of students participate in reading but only when the text is of
interest. Only 10-15 per cent of teachers report that their students are good at
writing. Some teachers report that only half of the class can write by copying
from the blackboard but they cannot read English textbooks. Teachers also report

that at the secondary level students need attention because they lack basic
reading, writing and logical skills. The issue of learning gaps is critical to the
connection between the primary and secondary level.
English Teaching in Rural areas
Teachers report the serious lack of an English environment in rural areas due to
a huge shortage of staff and resources. Teachers have low levels of language
proficiency and motivation to teach. Ganguli (2011:2) reports that ‘there are
districts in UP where only 2 to 5 per cent of teachers can actually teach English’.
Students in rural area schools are below average in basic English language skills.
Some teachers report that they have to admit all the students without assessing
their eligibility test. Students in rural areas are often ‘charmed’ by the government
policy of providing a mid-day meal and many parents visit the school only to
collect the scholarship cheques. Teachers are assigned duties related to
scholarship disbursement, fee collection, population and animal surveys, in
addition to helping to administer election and other non-teaching duties.
Some teachers report that the students’ attendance rate in the classrooms drops
from 75-80 per cent to 25-30 per cent during the harvest season. Teacher
absenteeism, high drop out levels and student attendance issues all contribute
to making a dysfunctional context, and yet there are popular aspirations for
learning English. Teachers report that they are under pressure from the
authorities to pass all the students. Girls, compared to boys, are more
conscientious about their studies and attend school more regularly.
Use of English/Hindi in Classroom
Table 2
high moderate low
Use of English in Classroom 21% 47% 9%
Use of Hindi in classroom 42% 15% 10%
In the area in question students do not have the advantage of an English
environment at both primary and secondary school level which can facilitate
their learning of the language. Consequently, their exposure to English is minimal.
Teachers generally use Hindi as a medium of instruction to teach English
language. Table 2 shows that 42 per cent respondents use Hindi and this figure
correlates with 47 per cent use of English in moderate degree. There is a general
perception among teachers that if they use only English in the classroom,
students may not understand anything. A majority of students lack the motivation

and confidence to speak in English. This further affects their participation in
classroom activities. Teachers feel that students are not learning the
fundamentals of English language, manifested by very low levels of competency
in basic English structures and vocabulary. Teachers reported that more than
70 per cent students are weak in listening skills. Almost 80-90 per cent students
do not speak in English. Table 1 shows the use of English in classroom is 21 per
cent which directly correlates with 42 per cent use of Hindi in English classroom.
The examination system is entirely based on written exercises, which creates a
negative effect on classroom practice, opportunities for communicative practice,
and the motivation of students to develop their spoken English.
Teaching methodology
Many teachers tend to enter their classroom without any lesson plan. Teachers
regularly complain that 45 minutes is too short to explain a topic to the whole
class, especially when the time has to be shared with checking students’
comprehension based on written work. Most of the students studying in class IX
can hardly read or write. Teachers also report that some of their students are
below average in grammar.
Teachers rely heavily on the textbook as the principal teaching aid and source
in the classroom, and lessons tend to be very teacher-centred as a result of this.
Many teachers report that they are unaware of alternative language teaching
approaches and expressed their desire for training to develop their awareness
and skill. However, there is little incentive in the system to experiment with
teaching approaches as the inspection process in schools is focused on coverage
of the syllabus rather than teaching methodology.
Institutional and Job-related Challenges faced by the respondents
Table 3
Column 1 Column 2 Column 3
Extremely Challenging Not Challenging
Challenging at all
in covering the course
on time 32% 40% 28%
in scheduling administrative
responsibilitiesso as to
meet the desired learning
outcomes 22% 51% 27%

in making commitment to
see the desired learning
outcomes 34% 42% 24%
in making commitment to
the job as per the
institutional requirements 25% 47% 28%
Table 4
Challenges in evaluation Serious Moderate Slight
of Language Skills difficulties difficulties difficulty
Listening 3% 46% 51%
Speaking 29% 39% 32%
Reading 6% 53% 41%
Writing 10% 43% 47%
Grammar 20% 46% 34%
Challenges in evaluation of Language Skills
The issue of what training is required for these teachers is a direct corollary of
what challenges they face as illustrated in Tables 3 and 4. The merged figures of
column 1 and 2 in Table 2 indicate that the main challenges dominating are
course coverage and scheduling administrative responsibilities so as to meet
the desired learning outcomes. In Table 3, the challenges in evaluation of
language skills are high with nearly half of the respondents indicating particular
difficulties with respect to listening skills, speaking skills and grammar, followed
by reading skills and writing skills.

Principals’ Data
Figure 1: Number of Training Sessions and Types of Training Programmes
Attended.
Need for attending various training programmes
As evident in Figure 1 more than half of all teachers have not attended training
in the last five years. Only a very small proportion of those interviewed have
attended more than one training course in any one year over the five years
under examination. Chart 1 reports the need for regular, short-term training
programmes (as indicated by principals) in the following areas:
• refresher courses in English teaching;
• classroom observation and feedback training;
• curriculum and learning outcomes; and
• effective student assessment.

Chart 1:
However, 98 per cent of principals report that there is no budget for teacher
training programmes. Chart 2 outlines the main resources required as indicated
by the same group of principals.
Chart 2:

Key areas of Training Needs
In addition to the type and duration of training programmes required for teachers
as specified by the principals, other key areas of training needs were identified
by the teachers surveyed for this study:
• how to access training facilities and opportunities for training and
teacher development programmes;
• dealing with drop-out and student absenteeism;
• adapting appropriate materials for classroom teaching;
• teaching low-ability and weak /mixed ability learners;
• teaching large often multi-level classes;
• how to manage the transition from primary to secondary school
level;
• learner-centred teaching methodology;
• maximising classroom time for spoken English;
• bilingual modes of teaching to facilitate learning.
Conclusion
Policy Intervention
English language is seen as the language of opportunities and instruction and
yet opportunities for teacher training to develop this important skill in this part
of India are minimal. Teachers report that teaching is well paid for government
job holders But for those who are employed as private teachers, salaries are
low and workloads are heavy. There is no financial or job security.
These are areas where teachers themselves need help (Azavedo, 2011),
particularly when delivery of the content is the norm rather than the exception.
In reality, English and Hindi go together in the English classroom; teachers need
training to understand how best to exploit the use of both languages to aid
understanding and facilitate learning.
The State Government of UP has initiated some steps to make English compulsory
in primary education (The Times of India, 2010) and to maintain adequate
numbers of trained teachers. However, the recruitment of trained and qualified
teachers, together with investment in human resources in the school system
has to be addressed, and this means enhanced and regular opportunities for
the kind of training outlined above. Kumar (2010:8) states that behind this
shortage (of qualified teachers) lies ‘a long history of neglect of teacher training
and the poor social status of the elementary school teacher’.

More than offsite training there is an urgent need to conduct training needs
assessment at onsite locations. In resource-poor environments pooling training
opportunities in clusters of schools with similar profiles and needs can be an
effective way of raising standards and supporting the on-going professional
development of teachers in the fundamental aspects of teaching English
identified in this paper. But this needs to be based on a clear examination of
needs so that scant recourses are not wasted or misdirected.
References
Azavedo, F.M.J. (2011): ‘A Teacher Orientation Programme’ in Gunashekhar, P., Padwad,
A. and Pawelec, D. (Eds.). Starting, Stimulating and Sustaining English Language Teacher
Education. British Council.www.teaching English.org.uk
Bharadwaj, V. (2011): ’Extra teachers to go to high strength schools’. The Hindustan
Times, New Delhi, July 3
Ganguli, A. (2011): ‘Innovate and Inspire’. The Hindustan Times.New Delhi, Tuesday,
December 27
Kumar, K. (2010): ‘India’s children have a precarious right’. The Hindu. April 3, Editorial
National Curriculum Framework (NCF) (2005): Available at http://www.ncert.nic.in/
rightside/links/pdf/framework/prelims.pdf (accessed 27 December. 2012)
National Focus Group (NFG) (2006): Position Paper: Curriculum, Syllabus and
Textbooks. NCERT. Available at http://www.ncert.nic.in/new_ncert/ncert/rightside/
links/pdf/focus_group/cst_final.pdf (accessed 27 December 2012)
The Sunday Times of India. (2010): ‘Dalits see English as social leveller’, January 17
The Times of India. (2011): ‘UP Board introduces Changes in Exam Pattern’, October
7
UGC Major Research Project. (2012): Training Needs Analysis (TNA) of Secondary
School English Teachers in Aligarh: Implications for English Language Teacher
Education. (University Grants Commission, New Delhi)

Portfolio Assessment: Do Teachers and Experts Speak
the Same Language?
Ravinarayan Chakrakodi, Faculty, Regional Institute of English South
India, Bangalore
This research study was carried out to develop indigenous criteria for assessing
writing portfolios in a teacher education context. The study looks at (i) teacher
trainees’ views on the criteria to be used for the assessment of portfolios, and (ii)
the criteria used by expert raters to assess portfolios. The research was conducted
in two phases. In the first phase, questionnaires were administered to 19 teacher
trainees seeking their views on how the portfolios should be assessed. In the
second phase, four expert raters were asked to assess an authentic portfolio
using their own criteria. The data gathered from the teacher trainees revealed
that while assessing portfolios, the drafts as well as the completed products should
be taken into account. Also, some criteria appeared to be more important than
the others to teacher trainees as well as expert raters in assessing portfolios.
However, it was not clear whether to score the individual pieces in the portfolio
and then add up the individual scores or to score the portfolio as a whole. The
study will help in developing a new scale that will be used to assess portfolios in
the specific teacher training context.
Introduction
A portfolio may be broadly defined as ‘a purposeful collection of student work
that exhibits to the student (and/or others) the student’s efforts, progress, or
achievement in (a) given area(s)’ (Northwest Evaluation Association, 1991: 4 cited
in Weigle, 2002: 198). The assessment of the portfolio that consists of samples
of work produced by students over a period of time may be called portfolio
assessment.
As many research studies indicate, portfolios have several advantages. However,
the assessment of portfolios still seems to be a fuzzy area. There seems to be
some confusion among teachers and teacher educators as to how to assess
portfolios (Burroughs, 2001; Tillema and Smith, 2007). Also, as Burns (1999)
and Tillema and Smith (2007: 443) report, there are few research studies carried
out on the appraisal of portfolios and the judgemental processes involved on
part of the portfolio raters.
This research study seeks to answer the following questions:
1. What are the teacher trainees’ views on how portfolios should be
assessed?
2. What criteria are applied by expert raters to assess portfolios?

Background to the study
The context for the study was an in-service teacher training programme. The
teachers involved in the study belonged to one cohort (67 in number) and they
assembled portfolios in their writing classes during a two-month primary-level
in-service teacher development programme at the Regional Institute of English
South India, Bangalore. The writing programme was based on the portfolio theory
and the pedagogical practice related to this theory.
In teaching writing, the process-oriented approach was followed. Each week,
one writing task was administered to the teachers. The task was completed
using various stages such as free writing, prompted discussion, brainstorming,
peer assessment, revising, editing, etc. Teachers produced the final piece using
process strategies. The process data such as initial drafts, comments from the
peers and feedback from the trainers were kept in individual portfolios along
with the final products. Teachers carried out self-assessment regularly using
the self-assessment checklists provided to them. After the completion of all the
tasks, teachers recorded reflections on the processes of writing they followed
and on their own development in writing over a period of time.
Goals and contents of portfolios
The purposes of the portfolio in the in-service teacher training programme were
manifold as listed below:
• to develop the writing abilities of the teachers;
• to train teachers in the pedagogical aspects of portfolio assessment;
• to use portfolios for assessment and certification purposes.
Hence, the portfolio was used as a tool for both teacher development and teacher
assessment.
The portfolio consisted of four core elements:
• writing tasks with multiple drafts of each task;
• comments from peers;
• feedback from trainers, and
• self-assessment and reflection.
The portfolio covered a wide range of tasks. The curriculum was built around
these tasks. Some tasks had an element of choice within them whereas others
were common to the whole class.
The portfolio included texts in a variety of forms or genres, written for a variety
of audiences and for a variety of different purposes. The variety and the range

of texts included in the portfolio would make it difficult to identify consistent
features of writing at different levels. This is pointed out by Hamp-Lyons and
Condon (2000: 54) as well as Weir (2005: 165). They indicate that if portfolios
varied widely in what they contained, as well as in the quality of the work, it
would add a degree of difficulty to readers. Also, where a selection of topics is
provided, it is very difficult to compare performances. Hence, it was decided to
have two sections in each portfolio: an optional and a compulsory section. The
optional section consisted of the following tasks:
• A diary entry/ a curriculum vitae;
• A review of a book/movie;
• An essay on a given topic: descriptive/argumentative/narrative, and
• Designing activities to teach vocabulary/grammatical structures.
On the other hand, the tasks included in the compulsory section were as follows:
• A brief biography;
• A report on an event;
• Writing a message, and
• A letter to the editor of a newspaper.
For the final assessment, the four compulsory tasks and the reflective responses
were taken into account. The selection of four tasks would make it easier to
compare the portfolios as there would be greater resemblance from one teacher
to the next. Also, it would be easier for readers/assessors to agree about scores
and grades.
However, the portfolios were not graded for summative assessment because of
two reasons: firstly, it was for the first time portfolios had been introduced into
the training programme and secondly, there were no established criteria for
assessing the portfolios. In order to use portfolios for high-stakes purposes, it is
essential to develop a set of assessment criteria.
In this study, I made an attempt to involve teacher trainees and expert raters to
establish a set of criteria for assessing portfolios. The section that follows presents
and describes the methodology adopted and the instrument used for the
purpose.
Teacher trainees’ views on assessing portfolios
A questionnaire was used to gather teacher trainees’ views on how portfolios
should be assessed. The questionnaire had different parts which functioned in
different ways. There were questions that elicited background data from the

teacher trainees such as the years of teaching experience, the knowledge of
portfolios, etc. The other parts in the questionnaire elicited teacher trainees’
views on how portfolios should be assessed (see Appendix 1 for details). There
were three sections: Section A, Section B and Section C. The questions in Section
A and Section B required Likert scale answers and the questions in Section C
required rank-ordering of items.
Altogether, 67 teacher trainees attended the training programme and assembled
the portfolios. However, for the study I selected a smaller group of 30 teacher
trainees. I gained responses from 19 teachers from among whom 4 teachers
had teaching experience ranging from 1-5 years, 5 had teaching experiences of
5-10 years, 3 had 10-15 years of teaching experiences and 7 of them had more
than 15 years’ experience in teaching. This subgroup/sample is representative
of the larger population.
I will first present the teacher trainees’ responses to the questions in Section A
of the questionnaire. The majority of teachers, 89.4% (17 of 19) strongly agreed
or agreed that A3 (the development of the topic from first draft to final draft)
should be considered while assessing portfolios. A similarly large proportion of
respondents, 84.2% (16 of 19), strongly disagreed or disagreed with the
statement that only the best pieces of work should be looked at (A7). However,
they were divided in their opinions about the overall approach to assessing
portfolios; they were not sure whether the individual pieces in the portfolio should
be rated, and then individual scores should be added up (A1), or whether the
rater should make a single judgment of the portfolio as a whole (A2).
Let us now look at Section B in the questionnaire. The majority of the teachers
strongly agreed or agreed with all the statements. None of them expressed strong
disagreement with any of the statements. All the dimensions seemed to be
important for them in assessing portfolios: the dimensions of writing ability, as
well as those related to the features of the portfolio.
Finally, in Section C, teachers were asked to rank order the dimensions according
to the degree of importance they attached to each dimension. Each of the
teachers ranked the dimensions in different orders. Some of them, however, did
not rank all the dimensions. For example, Teacher 6 did not rank B8, B9 and B10
and Teacher 19 did not rank any of the dimensions.
The statistics clearly suggest that some dimensions were ranked as highly
important by the teacher trainees, some were ranked as being of medium
importance and one dimension, in particular, was ranked as of very little
importance.
Dimensions such as ‘adequacy and relevance of content’, ‘reflective thinking’,

‘organisation of ideas’ and ‘growth, development and engagement as a writer’
were ranked as highly important by the teacher trainees. Other dimensions such
as ‘accuracy and appropriacy of syntax and lexis’; ‘ability to write for different
audiences and purposes’; and ‘ability to use processes and appropriate strategies
for different pieces’ were ranked as being of medium importance.
Discussion and Conclusion
Teachers seemed to suggest that the following dimensions were important in
assessing portfolios:
• adequacy and relevance of content;
• growth, development and engagement as a writer;
• reflective thinking;
• control of grammar and mechanics / accuracy and appropriacy of syntax
and lexis; and
• coherence and cohesion/ organisation of ideas.
Of the five dimensions listed above, some are related to features of writing and
some to the features of portfolio. However, the presentation of the portfolio
itself did not appear as an important dimension for most of the teachers. This is
a useful data because in some contexts where studies on portfolios have been
carried out, presentation of portfolio work - labelling the drafts, stapling the
final draft on top of the others, etc. – has been of greater importance in their
evaluation.
Teachers also seem to suggest that rather than focusing on the final drafts, we
should take the processes and strategies used to arrive at the final products
into account when assessing portfolios. This goes against the idea of grading
only papers designated as final drafts in summative assessment. Teachers’ views
do not support the argument of several researchers such as Ford and Larkin
(1978 cited in Sommers, 2003: 380), Burnham (1986 cited in Sommers, 2003:
380), and Elbow and Belanoff (1986 cited in Sommers, 2003: 380) for whom a
portfolio is ‘a sampling of finished products selected by the student for evaluation’.
Furthermore, the idea of having two portfolios, one containing a sample of ‘best
work’ for summative assessment purposes, and another containing broad and
various samples of work for formative purposes, as is done in some contexts,
does not appear to be attractive to teachers. However, this needs further
investigation, as the sample is too small to generalise the findings.
Expert raters’ criteria
The main aim of this section is to examine the different criteria used by the
expert raters.

Method
Four expert raters were involved in this study. An authentic portfolio was given
to them for assessment. Each of the raters was requested to (i) come up with a
set of criteria for assessing the given portfolio; and (ii) provide a score out of 50
to the portfolio, indicating how the 50 marks would be distributed among the
different criteria. Four expert raters who had postgraduate ESL qualifications
with experience ranging in length from 12 years to more than 20 years were
selected for this study. All of them were experienced in assessing ESL writing
either at the university level or in teacher education contexts. All the expert
raters were non-native speakers of English. They had detailed knowledge of the
training programme where the study was conducted and had knowledge about
the background of teachers who assembled the portfolios. The curriculum and
expectations for performance at the training institute were familiar to them.
Results
All the four expert raters came up with their own criteria for assessing the
portfolio and scored the portfolio out of 50 marks. The criteria for assessing the
portfolio varied from expert to expert as shown in Appendix 2. On the whole, 13
criteria/dimensions were pooled out from the four expert raters as follows:
• overall scope and form / organisation of the text;
• relevance, adequacy, audience awareness of content;
• control over clause level grammar and vocabulary / accuracy of
language, vocabulary and tone;
• cohesion / organisation;
• format and style;
• quality of the final product;
• response to feedback and improvement across revisions;
• teacher’s own comments on her growth / depth of reflections;
• development of the teacher;
• learner effort;
• presentation and organisation of portfolio;
• range of writing tasks given; and
• trainer’s evaluation and comments.
All the expert raters used analytic scales. Only one of them (Rater 1) used distinct
levels in the scale. Three of the expert raters assessed the portfolio as a whole

whereas one of them scored individual texts in the portfolio and added up the
scores to arrive at an aggregated score. As shown in Appendix 1, Expert Rater
1 awarded score to each of the criteria by examining individual tasks in the
portfolio. On the other hand, Expert Rater 2 awarded scores to each criterion
by examining the portfolio as a whole. Furthermore, the weight given to each
criterion varied from rater to rater but the aggregated scores produced by the
raters were sufficiently consistent.
Discussion
It appeared, from the criteria used by the four expert raters, that the following
attributes were important to all of them:
• relevance and adequacy of content;
• organisation of ideas;
• accuracy of language; and
• improvement across writing.
However, a close examination of the individual rater’s criteria revealed that they
were valuing different aspects of writing. For example, Raters 1 and 2 seemed
to focus mainly on the features of writing (e. g. ‘cohesion’, ‘relevance, adequacy
and audience awareness of content’) at the expense of the features of the
portfolio such as reflective thinking.
Further, Rater 3 seemed to take the position of an external examiner and
therefore she appeared to examine the portfolio from an outsider’s perspective.
For Rater 3, it was not only the features of writing (e.g. ‘Accuracy of language,
vocabulary and tone’) and features of the portfolio (e.g. ‘Trainee’s own comments
on her growth’) that were important but the trainer’s ability to design tasks (‘Range
of writing tasks given’) and give critical feedback (‘Teacher’s evaluation and
comments’) was equally important. Perhaps, Rater 3 was not clear about the
purpose of the assessment. She carried out a programme evaluation of portfolios
instead of using the portfolio for summative assessment.
Above all, the scales used by the raters did not show clear distinction between
the levels in the performance. This may be explained by the fact that they have
had the opportunity to examine only a single portfolio. Additionally, an informal
talk with some raters revealed that much of their focus during the assessment
was on the quality of the final draft. Furthermore, there still seemed to be some
confusion regarding the overall approach to assessing the portfolio: whether to
score the individual samples and add up the individual scores or to evaluate the
portfolio as a single entity. This is clear from the two contrasting approaches
Rater 1 and Rater 2 adopted.

As discussed in the previous chapter, teachers were also not certain about the
overall approach to assessing the portfolio. The questionnaire responses of
teachers in the first phase of the study indicated that they were not sure whether
to score the individual pieces in the portfolio and then add up the individual
scores or to score the portfolio as a whole.
Overall discussion and conclusions
On the whole, it seems that there is a need to construct a common scale to rate
portfolios taking into account the data gathered by the teachers, the expert
raters’ and the criteria used in other contexts where studies on portfolios are
conducted.
The data gathered will, thus, help in developing a draft common scale in the
next phase. Interviews will be conducted with the expert raters to seek their
views on the draft common scale. The draft common scale will be used to train
raters and to examine the validity and reliability measures in future. The new
scale will finally be used to assess portfolios in the specific teacher training
context.
The advantages of adopting a procedure such as this, where teachers and expert
raters are involved in the assessment process, are many. The data gathered will
help the faculty members to develop indigenous criteria for assessing teacher
trainees’ portfolios. In addition, the involvement of teacher trainees in the study
will enable them to gain knowledge and expertise in the development of
assessment criteria. It will also help them to develop criteria when they assess
their students’ portfolios.
References
Burnham, C. (19860: ‘Portfolio Evaluation: Room to Breathe and Grow’ in C. Bridges
(Ed.), Training the New Teacher of College Composition. Urbana. IL: NCTE
Burns, C. W. (1999): ‘Teaching portfolio and the evaluation of teaching in higher
education: Confident claims, questionable research support.’ in Studies in Educational
Evaluation, 25, 131-141
Burroughs, R. (2001): ‘Composing Standards and Composing Teachers: The Problem
of National Board Certification’ in Journal of Teacher Education, 52 (3), 223-232
Ford, James E., and Larkin, G. (1978): The Portfolio system: An End to Backsliding
Writing Standards. College English, 39, 950-55.
Hamp-Lyons, L. and Condon, W. (2000): Assessing the Portfolio: Principles for Practice,
Theory and Research. New Jersey: Hampton Press, Inc.
Northwest Evaluation Association. (1991): Portfolios. Portfolio News, 2(3), 4
Sommers, J. (2003): ‘Bringing Practice in Line with Theory: Using Portfolio Grading
in the Composition Classroom’ in Tavers, J.K. and Moore, C. (Eds.), Teaching in Progress:
Theories, Practices and Scenarios (3rd edn). New York: Longman

Tillema, H., and Smith, K. (2007): ‘Portfolio appraisal: In search of criteria’ in Teaching
and Teacher Education, 23 (4), 442-456
Weigle, S. (2002): Assessing Writing. Cambridge: Cambridge University Press
Weir, C. J. (20050: Language Testing and Validation: An Evidence-Based Approach.
Hampshire: Palgrave Macmillan

Appendix 1: Questionnaire
This questionnaire is about assessing the writing portfolios of in-service teacher
trainees. The trainees, who attend two-month primary-level in-service teacher
development programmes at the Regional Institute of English South India,
Bangalore, assemble portfolios in their writing classes. This research study
attempts to examine the trainees’ views on assessing such writing portfolios.
Hence, your views on the criteria to be applied for portfolio assessment are
important for this study. Please provide the information as required.
Name (optional):
Years of teaching experience: Nil/1-5/ 5-10/10-15/more than 15 years
How do you rate your knowledge of portfolios? adequate/ limited/ inadequate
Have you assessed teacher portfolios before participating in this project?
Yes/No
A. If I were to assess a portfolio, I would… (underline one of the
following):
A1. look at individual pieces, rate them and add up the individual scores
Strongly Agree Agree Not sure Disagree Strongly Disagree
A2. make a single judgment of the portfolio as a whole
A3. consider the development of topic from first draft to final draft
A4. mainly focus on the final drafts
A5. take into account the processes and strategies used for writing

A6. look at drafts as well as completed products
A7. look at only the best pieces of work
A8. Any other (please specify)
B. If I had to assess a portfolio, I would use the following dimensions
for assessment:
B1. Adequacy and relevance of content
B2. Accuracy and appropriacy of syntax and lexis
B3. Control of grammar and mechanics
B4. Organisation of ideas
B5. Coherence and cohesion
B6. Ability to write for different audiences and purposes
B7. Ability to use processes and appropriate strategies for different
pieces of writing
Strongly agree Agree Not sure Disagree Strongly Disagree

B8. Growth, development and engagement as a writer
B9. Reflective thinking
B10. Presentation of portfolio work
B11. Any other (please specify)
C. Which of the above dimensions are most important in assessing a
portfolio? Rank them in order of importance and distribute 50
points among these dimensions.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Thank you for your time.

Appendix 2: expert raters’ criteria
Expert Rater 1
Task 1 Task 2 Task 3 Task 4
Criteria Weight Marks Marks Marks Marks
allotted allotted allotted allotted
Content (clarity, purpose
achieved) 10 6.5 6 5.5 7
Organisation (sequencing of
ideas, use of cohesive devices,
introduction, development
and conclusion) 10 6.5 7.5 7 7
Accuracy (sentence structure,
syntax, spelling and
punctuation) 10 5 6 6 5
Format and style (originality,
creativity, strategies used to
capture reader’s attention) 10 6 6.5 6.5 6
Learner effort 10 6.5 5.5 6 5
Total 50 30.5 31.5 31 30
Average score – 30.5 + 31.5 +31 + 30 = 123/4 = 30.75
PORTFOLIO SCORE: 31
Expert Rater 2
Dimensions of writing (written text) considered:
Overall scope and form/organisation of the text
Relevance, adequacy, audience awareness of content
Control over clause level grammar and vocabulary
Cohesion
Response to feedback and improvement across revisions

Scale for each dimension
A A+ : 41 - 50 points
B B+ : 31 - 40
C C+ : 21 - 30
D : 11- 20
E : 1 - 10
Summary Award
Dimension Grade / Points
1) B+ 37
2) B 35
3) C 30
4) B 31
5) A 40
173 / 5 = 34.6
This may be taken as equivalent to 35 marks out of 50, with the caveat that
30 (60%) is not considered to represent ‘First Class’.
Expert Rater 3
Criteria Weight Marks
Range of writing tasks given 10 8
Trainee’s own comments on her growth 10 8
Accuracy of language, vocabulary and tone 10 5
Trainer’s evaluation and comments 10 4
Development of the Trainee 10 6
Total 50 31

Expert Rater 4
Sl No. Criteria Components Weighting Marks
1 Content Inclusion of the
required entries,
Purpose of the text 30 11/15
2 Efforts of the The strategies
learner adopted to improvise 20 8/10
3 Quality of the Vocabulary, style,
final product Accuracy and
Organisation 30 10/15
4 Depth of
reflections 12 4/6
5 Presentation and Lay out, Design,
organisation of Format and
portfolio Paragraphing 8 3/4
Total 100 36/50

Locating Continuous Comprehensive Evaluation in a
Pedagogic Frame
Jacob Tharu, retired Professor and Head, Department of Testing and
Evaluation, Central Institute of English and Foreign Languages (now EFL
University), Hyderabad
One of the most common terms featuring in discussions relating to the educational
process today is Continuous Comprehensive Evaluation (CCE). The opinions of
students, parents, and teachers are often quoted in newspapers and magazine
articles. A number of papers presented at the TEC 2012 Conference included
CCE in the title, and others had it as a sub-theme. There is considerable activity
implementing and institutionalising CCE in directorates of education, boards of
studies, and so on. Presentations relating to assessment made by the states at
various fora in 2012 often mentioned the trialling, piloting and preparations for
up-scaling. There is much evidence of various interpretations of the basic notion
and a range of practices on the ground (trial or final versions) in the school
systems of the nation. The National Council of Educational Research and Training
(NCERT) took the initiative to work with the states (beginning with a national
seminar in October 2012) to evolve a coherent and workable scheme. While a
plurality of means and processes in the curriculum is undoubtedly welcome, it is
important that the essential and worthy educational purposes of CCE are well
served. Developing a pedagogically workable scheme for CCE with a high level
of clarity relating to core operations at the school and classroom level is a matter
of considerable urgency now. The landmark Right to Education Act (RTE) of 2009
makes specific mention of CCE as the approved mode of pupil assessment for
the major phase of school education.
It is with these current and heightened concerns relating to new approaches to
the assessment of students’ progress that this paper attempts to clarify the key
concept(s) and educational goals that ground the notion of CCE (treating it as a
goal) hoping to discover what makes it so appealing in current discourse. There
is no effort to develop a supposedly improved practical model. On the contrary,
the focus is on what educational purposes are served and what values promoted
through this approach, without dealing directly with the details of implementation.
The discussion will consider CCE in two different frames: first as a further phase
or continuation of general examination reform measures, then as a potential
resource for enriching pedagogy. The latter theme will take up the greater part
of this paper.
Examination reform: a quick historical survey
A concern for the reform of examinations has been present from the very early

decades of modern public education in the 19th century. The necessity and
desirability of ‘examinations to monitor students’ progress’ has never been
denied. But various negative effects on the educational process have surfaced
and persisted and driven successive reform agendas. Curiously, it is difficult to
find any pattern of increasing scope or complexity in the practical measures
suggested over this long period. It is surely a surprise to find in the report of the
Hunter Commission (1882) the observation that “...it is beyond doubt that the
greatest evil from which the system of Indian university education suffers is that
teaching is subordinated to examination and not examination to teaching.” This
distortion that examinations can cause cannot thus be attributed too casually
to pressures arising from mass education introduced in the post-Independence
era. Formal examinations by their very nature seem to generate collateral
damage, and any serious analytical discussion of vision, values, structure of
formal education has to address this issue.
Virtually all discussion of ‘examinations’ has been marked by a reform stance,
following the lead in the Higher Education (Radhakrishnan) Commission Report
(1949) which famously declared that if a ‘single most important’ recommendation
had to be identified, it would be examination reform’. At the school level active
concern about flaws in examinations and efforts to spell out concrete measures
towards improvement are found from the early post-Independence years. The
Secondary Education (Mudaliar) Commission (1952-53) made thoughtful
recommendations for improving the quality and educational value of
examinations such as reducing the number of external examinations; minimizing
subjectivity by introducing objective testing; assessing the pupil’s all-round
progress through a proper system of school records, and so on.
Over the next half-century, commissions and task forces have identified several
measures that could improve the validity, reliability, practicality of ‘home’
examinations and the more formidable external/public examinations, and a fair
number have been implemented in various locations. A detailed list of these
changes is not relevant here, but a few are mentioned to capture the thrust of
practical steps considered necessary. The quality of the syllabus-governed
‘question paper’ has received the most attention—through precisely stated
learning outcomes (with Bloom’s Taxonomy serving as a major guide after the
mid 1950’s), the use of a range of item types (importantly to reduce the
dominance of the essay question), blueprints to provide balanced coverage of
objectives, clear wording of questions, and scoring guidelines to reduce inter-
rater variations. At the tertiary stage, such changes and certain others were
considered and taken up somewhat haphazardly following the UGC’s (1973)
initiative ‘Examination Reform - A Plan of Action’. Notable among these are: the
introduction of the semester system, the use of grades rather than marks,
granting weight to (college level) internal evaluation, institutionalizing the facility
of make-up (supplementary) examinations, and the development and use of

question banks to upgrade the quality of question paper setting. Higher education
does not have any parallel of the regular class tests and unit tests that punctuate
curriculum transaction at the school stage. The well-established pattern of
instruction in the colleges was and largely is the phase of lectures followed by a
phase of study leave which serves as preparation for a totally external
examination. The focus of reform in higher education thus remains on this (single)
examination.
It is in the National Policy on Education (NPE) of 1986 that the notion of CCE first
appears as a proposed mode of evaluation relevant to the school stage. The
Programme of Action (POA 1986/92) based on the Policy statement includes
clauses such as:
Along with external examinations, continuous institutional evaluation of
scholastic and non-scholastic aspects of education will be introduced, and
separate certificates issued; and
Integration of evaluation with the process of teaching and learning.
The National Curriculum Framework (NCF) 2000, the next major statement of
policy, reiterates that:
Evaluation needs to be integrated with the process of teaching and learning.
The greater the integration the better the outcomes of learning, and
Evaluation can be a powerful means of influencing the quality of teaching
and learning, and for the development of both cognitive and non-cognitive
capacities.
From such statements it can be seen that the authoritative discussion of CCE is
more the signalling of policy direction than the identification of specific practices
at the level of classroom level. No coherent measurement model or pedagogic
strategy is invoked. Institutional evaluation that is wide in scope, spread over
time, and potentially in the hands of the teacher is the central theme. There is
not much clarity about what is supposed to happen in classrooms as teaching
‘with built-in assessment’ proceeds. Yet the idea of CCE is certainly appealing
since it endorses long-standing concerns about the dominance of the single
final examination, and to broaden the scope of what should be assessed in the
name of students’ development. It is worth noting that the phrase CCE does not
feature as a specific term in the register of educational evaluation outside India.
More concrete guidance for the teacher becomes available from 2000. The
Central Board of Secondary Education (CBSE) introduced a scheme for School
Based Assessment in 2005, linked to an Achievement Record card reflecting

attainments in scholastic and co-scholastic domains. Declaring pass/fail on the
basis of a single year end examination was discouraged. NCERT prepared the
Continuous and Comprehensive Evaluation - Teachers Handbook for the Primary
Stage in 2003. It built on the idea of school based assessment, and listed various
elements under the major co-scholastic components, the psychomotor and the
affective domains. The means to assess these include conventional tests,
observation schedules, checklists, anecdotal records, peer assessment, self-
assessment. A set of elaborate tables and forms that would make up the individual
pupil’s multi-dimensional (and multi-page) record along with indications of
appropriate periodicity for recording ‘data’ under these different heads was
prepared. The CBSE specifications for CCE (2009 with some modifications in
2011) is probably the most complete scheme developed with more components
in the co-scholastic category and detailed indicators of levels/progress and the
criteria for awarding grades. The schemes developed or being developed in the
states echo the models of NCERT and CBSE.
This use of these recording and reporting formats is the visible indication of the
implementation of CCE in our schools today. There is ample evidence from
teachers on the frontline that they find it cumbersome and unrealistic. Students
have expressed confusion and anxiety. There is a need for debate and a
commitment to learn from experience and slowly bring about meaningful and
appropriate improvements of CCE. Activities under CCE, like all orders to be
implemented in our top-down system, are subject to inspection for compliance
and the space for dialogue has to be created. This issue is taken up in the
concluding section.
How far have we moved on the examination reform path?
The analysis of progress is based on a distinction between the competent
implementation of concrete measures contained in the recommendations for
action in examination reform discourse on the one hand, and signs of mitigation
of the negative impact of the evaluation system on the wider educational process
on the other. The latter is in fact the driver of efforts towards examination reform.
Prof. Krishna Kumar (former Director of NCERT and acknowledged guiding force
behind the National Curriculum Framework 2005) in a piece titled ‘The Long
Road to Examination Reform’ (The Hindu, 21 November 2009) highlights the
power of the final Board examination in an echo of the Hunter Commission of
1882:
“Children begin to feel its power as soon as they enter Class I in the primary
school or even earlier, in the nursery. By the time they come to the higher
secondary level, the students themselves become convinced that marks,
and marks alone, matter. Colleges and universities do not consider it
necessary to apply their mind to assess the student’s potential. They go by
the student’s Board marks.”

Over decades, starting with the Radhakrishnan Commission, the role of
examinations in promoting memorization and rote learning has been a focal point
of criticism. Reiterating the need for reform, the position paper on Examination
Reform (NCF 2005) declares yet again that “question papers are of low
quality…[and] usually call for rote learning and fail to test higher order skills like
reasoning and analysis.” Even technical improvements have been minimal. A
third and perhaps most widely deplored aspect of the system of evaluation is
the tension and pressure it brings to bear on students.
Understanding the stability of the existing system
If the impact of numerous practical measures to reform the system is so low, an
attempt to understand what the reason(s) are should come before acting on a
valiant resolve to try yet again. For this the history of education framed by
examinations in India needs to be revisited, and features bearing on this
resistance to change identified. The system has certain unique characteristics:
Firstly, the model for the evaluation of pupils’ learning is founded on the principle
of external control: secondary education boards control evaluation in schools,
and the university likewise controls evaluation in affiliated colleges. A critical
factor is that the pattern of all local ‘home’ tests strictly follows that of the public.
An objective and credible verification of the progress made by students towards
declared and accepted learning objectives is a positive thing: a resource for the
individual and for society. However, public examinations in India have always
played another powerful — non-educational — role: that of selection. Post-
secondary education in India was developed primarily to prepare a (small) body
of individuals who would comprise the manpower handling the lower
administrative levels of the colonial government. A government job then was (as
it remains now) a sought-after prize. Secondary education started with a narrow
academic bias and naturally served as a filter that limited the numbers proceeding
further. The key to understanding the stability of the system lies in this filtering
function.
External examinations can spur students’ efforts to do well. There is no basis for
arguing that students should not strive to excel in examinations. However, just
below the surface of the apparently healthy challenge they represent, lies the
canker of competition: only the best in rank order actually succeed. The relentless
spread and increased virulence of the competitive ‘disease’ is closely linked to
the expansion of the educational system.
Competing with others in teams and as individuals is a normal and undoubtedly
healthy aspect of social life in nearly all societies. However, there is a potentially
problematic factor lurking in our system. For example, in high-stakes competitions
the competition is intense and the potential gains or losses are enormous. This
scarcity of sought-after ‘seats’ is a fact of life. Individuals who aspire to gain

access to them have to find ways of coping with the stress and tension of
competition. It is matter of choice on their part and not an imposition of society.
At the same time society does have an important responsibility, which is the
protection of equal rights. In a legalistic frame this means equal treatment:
everyone must be examined in the same standardized manner, with no variation
that can be interpreted as unfair. However, there is a serious aberration in the
Indian system. The distinction between the formally declared competitive
examination and the routine achievement test to monitor individuals’ progress
has been lost. All tests are seen in a competitive frame. This brings to light the
fundamental problem with evaluation as we experience it. There is the demand
that ALL assessment be subject to this law of uniform treatment to ensure fairness
for all. Rightly or wrongly, this pressure comes forcefully from students and
parents who are the primary stakeholders of school education. Their voice in
large measure controls the discourse around assessment – perceptions,
purposes, values.
It is this culture of use dominated by the inter-individual competition principle
that reform measures have run into. Changing the nature of tests is not a relevant
way of addressing this issue. It generates a strong pressure to keep patterns
familiar and stable to minimize real or imagined impediments to high scores.
Specific steps to improve the technical qualities of tests as measurement tools
have been absorbed and domesticated. Rigorous examination reform should
deal with such issues, not incremental technical improvements.
A glimmer of hope: the promise of a rediscovered CCE
Two themes representing far-reaching changes in the larger system have
appeared in the discussions revisited above. One is internal assessment, which
in our context means giving the power to handle pupil assessment to the teacher
and the school. The second is formative evaluation, using the information
generated by assessment to enhance learning. It is important to note that that
these aspects linked to agency and purposes in the cultural sphere are totally
independent of test structure. Any type of assessment can be handed internally
or externally; likewise the output (information) from any type of assessment can
be used for formative or summative purposes. CCE rests on both these
conceptual foundations. However, the whole discourse rather naively assumes
that these elements are realized in the actual system (and infuse its culture)
simply because their desirability is asserted. The teacher remains little more
than the administrator-supervisor of pre-constructed tests sent to her/him. This
is a familiar means of quality control, but far from a form of internal assessment.
This limits the teacher’s agency; the role is similar to that of a field assistant
employed to collect data using standardized tools for a research project.
When ‘syllabus’ effectively means fixed portions of knowledge to cover in a fixed
time frame, there is no room for the teacher to treat learning requirements in a

flexible manner with a degree of collaboration and negotiation. The syllabus has
to be covered in the way the textbook and test models indicate, and endorsed
by training inputs by all teachers. But what CCE envisages and calls for is exactly
this flexible and dynamic learner-centred and collaborative transaction of the
curriculum. The necessary pre-condition for CCE to make sense – a new
perspective on knowledge – is only just emerging and spreading.
A sign of hope lies in the discourse now epitomized by the label NCF 2005. This
vision of education, stresses teaching-learning that goes beyond textbook-
centred narrowness partly by relating school knowledge to students’ life outside
school. Students are recognized as proactive participants in learning who
contribute to the construction of the knowledge they gain. Thus what is learnt
(and indeed is to be learnt) is personal in a significant sense. There is also a
commitment to accepting and valuing diversity among learners on many
dimensions which makes different styles, rates, and paths of learning being
perfectly natural and valid, and makes the single correct answer ‘according to
the book’ virtually irrelevant.
Even a vague sense of this new approach is enough to show that the practices
represented by CCE are called for and not something different that has to be
accommodated. The new pedagogy views knowledge as emerging though an
open-ended process of extending and transforming the base given in the
textbook. This process needs to be monitored – internally – to try and keep
progress along intended lines to a reasonable degree. Such monitoring to
generate feedback has to be a dynamic ‘real time’ process, and the response to
feedback also needs to be timely and ongoing. This is exactly what continuous
assessment means – monitoring activity by the teacher which is integrated with
teaching, not tests given only after instruction has been completed. The
commitment to diversity in education, means recognising the many qualities of
children that can develop through participation in the curriculum in its richest
sense and this needs comprehensive monitoring for formative support.
The essential argument at the core of this entire discussion is that effective
pedagogy in keeping with the still emerging curricular framework needs formative
support in the form of continuous and comprehensive assessment, and only the
classroom teacher can handle this.
Supporting the teacher to manage CCE
The presence of CCE at the level of the school/classroom is represented by the
large report card that needs to be completed, some entries based on repeated
rounds of raw data collection. The massive challenge for the teacher from CCE
lies in the need to build observation of many aspects of students’ progress into
lessons and to capture its various strands in meaningful descriptions. They need

to be helped to gain understanding, practical knowledge, and appreciation with
regard to these new elements of day-to-day practice.
A serious study of what adequate teacher support would involve has sobering
lessons for all of us located outside the class who wear the mantle of service
providers. The basic fact is that virtually all that is significant in CCE lies in the
hands of the teacher in his/her unique setting – often multi-grade and diverse in
unpredictable ways. Consequently, all that is provided from the outside can only
be guidelines to follow and other resources to draw on as needed. Tightly packed
training modules of 5 or 10 days can at best be priming devices. Ongoing support
in a dialogic-collaborative mode has to be provided over a long period. Both
training inputs and ongoing support presuppose the availability of relevant
‘expertise’. It is hard to support the proposition that expertise in paper setting
(the keystone of external summative evaluation) readily extends to the strikingly
different sub-area of evaluation now opening up.
We can do better. And that is to learn, importantly by being present as genuine
observers (and perhaps as low-key collaborators) in classrooms. This would allow
the gradual articulation of a meta-knowledge relating to the practical knowledge
and skill applied when assessment-merged-with-teaching plays out in real time.
Such meta-knowledge emerging from practice is the critical base on which
training material can be developed. Existing theory about learning, pedagogy
and assessment goes into general training. Some ideas from this knowledge
base serving as a supplement to CCE-focused discussion might be useful. The
collaborative mode through which training and support material is developed
could serve as a safeguard to prevent teacher training and support for CCE
lapsing into the well-established tradition of talking down to teachers.
To sum up, two central themes in the discussion need to be restated. Firstly,
CCE is better treated as a constitutive component of the new pedagogy
associated with NCF 2005 than as an extension of examination reform grounded
in an earlier curricular model. Secondly, the challenge of the new pedagogy
includes developing appropriate resources for teacher training and support.
The newness of the field requires learning; and this provides an opportunity for
teacher educators to enhance their professional expertise.
References
Ministry of Education (1952): Report of the Secondary Education Commission: Mudaliar
Commission Report. Retrieved December 31, 2012, from http://www.teindia.nic.in/
Files/Reports/CCR/Secondary_Education_Commission_Report.pdf
Ministry of Education (2009, August 27): The Right of Children to Free and Compulsory
Education Act, 2009. Retrieved December 31, 2012, from http://mhrd.gov.in/sites/
upload_files/mhrd/files/rte.pdf
Ministry of Education (1949): University Education Commission (1948-49). New Delhi:
Government of India

Ministry of Human Resource Development (1986): National Policy on Education (1986).
New Delhi: Government of India
Ministry of Human Resource Development (1986): Programme of Action 1986-92.
New Delhi: Government of India
National Council for Educational Research and Training (2000): A Framework for
School Curriculum. New Delhi: NCERT
National Council for Educational Research and Training (2003): Continuous and
Comprehensive Evaluation, Teachers Handbook for Primary Stage. New Delhi: NCERT
National Council for Educational Research and Training (2005): National Curriculum
Framework. New Delhi: NCERT
National Council for Educational Research and Training (2006): Position Paper:
National Focus Group on Examination Reforms. New Delhi: NCERT

Assessing and evaluating

More Related Content

What's hot (20)

Similar to Assessing and evaluating (20)

More from AWEY MULYANA (8)

Recently uploaded (20)

Assessing and evaluating