Assessment in SEBD schools

Editor's Notes

• #2: The audience for the presentation is made up of the whole school staff and the lead governor responsible for assessment and monitoring. The SMT had asked me to look into this because of my interest in nurture groups and their use of the Boxall profile. I suggested that I felt it would be useful to us as a school that has a very nurture oriented ethos. In view of this I decided to look at what other schools were using. Also this presentation is going to form the basis of reporting back to our National Body - the Social, Emotional and Behavioural Difficulties Association (SEBDA) at the annual conference in May 2014. By which time I will have much more data.
• #3: The merits of this quote were discussed and I mentioned that given his track record of academic prowess it was probably a good starting point. I suggested that it was a very difficult case to argue against and used our own example of using PASS data for the last 5 years which shows whole school progression, though in a less rigorous way than I would like, which is my point of doing the study. I told them I was looking for evidence of something more concrete.
• #4: I have been at the school for 8 years and have over 28 years experience in total. Having helped many of the staff with various projects and been mentor to some I believe I have a certain level of professional respect from my peers. Also I regularly attend association conferences and have presented before Nationally with a project demonstrating the use of opera and classical music to aid development and learning in SEBD pupils.
• #5: Brief explanation of Critical Realism: “theoretical explanation proceeds by description of significant features, retroduction to possible causes, elimination of alternatives and identification of the generative mechanism or causal structure at work” (Bhaskar 1998a: xvii).Some discussion was needed to explain the difference between qualitative andquantitative data for some of our more junior members of small but I was able to relate it to our Individual Education Plans and how they work with numbers and general statements about behaviour.It was a real shame I could not get the interviews done in time for this because that would have generated further qualitative data. However, I would be wary of using only two people for any generalisations based on their testimony.
• #6: In line with my RQ’s I outlined how the questions needed both quantitative and qualitative data to get what I was looking for and how a mixed methods approach (Creswell 2011) fitted in with the Critical Realism paradigm as outlined by Krauss (2005)
• #7: I duly went through the process of applying for ethical approval from the University giving assurances that that there were no particular problems arising from the data collection in terms of vulnerability as the material is not sensitive, relating as it does to generic methods of assessment rather than specific individuals. Also that considerations were made of anonymity.The sample group is one of professionals who were invited to participate because they had some professional interest in the outcomes and were in some way involved in the assessment process in their establishment. Indeed one respondent said “ If you develop something good from your research I would love to see it” (respondent 5)
• #8: When this survey is carried out next time it will be refined to ask questions specifically about behavioural assessment to filter out the responses I am not interested in such as reading, numeracy or similar. The contributors will be much more random as the access to the weblink for the questionnaire will be public though hosted on a couple of specialist websites. Also I have now got access to a full version of SurveyMonkeytmand so have much better potential for online analysis.
• #9: These diagrams were surprisingly readable when put on a white board and nicely illustrated the comparison between Qualitative and Quantitative data analysis with regard to a Mixed-methods approach as suggested by Krauss (2005) . The simple yes / no questions generated the graphs and the others were analysed using AtlasTI by assigning codes to words / phrases. These codes were then grouped into families of similar concepts. Because the relationships between codes can be seen it offers some insight into the links between the text that generated them.
• #10: I was happy that there was a good response from my requests. I was glad to see that 55% of those asked to complete the survey did so as according to University of Texas, Instructional Assessment Resources website, a figure of about 30% is usual (University of Texas 2011) this is further backed up by Watt et al (2002) 32%, Naire et al 31% and Ogier (2005) 30%. Though higher figures such as 43% (Dommeyer et al 2004) and 47% (Ballantyne 2005) are quoted, the figure of 55% stands up well even in a small survey. (in Nulty 2008)However, the data sample is too small even if everybody responded, to draw any conclusions that one might be able to apply to a general population. The next survey will be open to the general population ( or at least a specialised population) via Association websites and this should allow for more impact of the data. This was always to be considered as a pilot and it has been an extremely useful exercise in that respect.The slides that follow are ways of showing the data obtained in a quantitative manner. This was achieved by looking at numbers of code occurrences or themes.
• #11: This is a little surprising that some respondents (20%) did not do any assessment. I think this relates to the notion of testing as outlined in the reflective report and that it is a question of recognising what is and what is not “assessment”. I would argue that any type of judgement about a behavioural function is a form of assessment.
• #12: The only surprise here is that some people use no named assessment. That I was not expecting but realised that if an establishment developed it’s own assessment regime it was likely to use it if it felt it delivered all they needed. This could be due to the idea that people would have more faith/confidence in a system they have collaborated in the drawing up of.
• #13: I pointed out that I was not surprised at the wide range of assessment tools but was surprised at the lack of specific behavioural ones. Once again this would only have real significance in a much larger sample and pointed this out to the audience. I suggested that as many of these are tests in some way it was perhaps more in the psyche of the respondent to easily interpret them as “assessments”. Perhaps there would have been more of they had believed their daily records constituted assessment. Perhaps I could have clarified this but definitely will be looking at data specific to behavioural assessment / measurement / profiling. The wording has yet to be decided.
• #14: Although a little small I was able to zoom in using a remote mouse and highlight those named tools that were of interest. For me this was PASS, McSherry and Boxall. A discussion ensued relating to the compound use of PASS and Boxall. I feel this is the way to go for our establishment. I highlighted the WRAT 4 and PASS as it is one of only two with any commonality.
• #15: I pointed out that the range of reasons in no way showed any kind of pattern. The more arbitrary comments were illuminating and I suggested that the response “we built it” was a comment of someone who believed very strongly that they had the right material for them. I would like to have had more to go on here but it’s diversity was perhaps a commentary of the general idea of assessment?
• #16: There was no surprise here. The actual figure is really 100% in my view, as what appears to be negative was from an E.P. who comments about the whole school monitoring that they deal with rather than monitor individuals.
• #17: This was a very easy to slide to go through though there was some discussions as to what I meant by “directly”. I gave the example of how we use our system to interventions e.g. if a child is consistently late for something, our system will flag it up automatically and then one of the support staff will be put in place to escort a pupil between lessons until they can do it alone and be on time. This is a good example of how monitoring can directly inform intervention.
• #18: Most commonly respondents did not mention how they specifically recorded their data (70%). Is this relevant? Well only in as much as those who record electronically, I believe, will have a much better ability to manipulate the data with ease to look for trends /patterns than poring over piles of paper in long term or multiple interventions.Reviews are statutory and as such material relating to behaviour must be provided so the figure of 5 for frequency is only surprising because it is quite low in my opinion.
• #19: Perhaps the low amount of “end of term reports” reflects the traditional approach of schools in reporting subject attainment rather than focus on behaviour. This is to me an interesting notion and one I believe stems from a perception of the need to create Ofsted and Local Authority friendly data that can demonstrate school efficacy via league tables. Personally I don’t think it is a very healthy route, but understandable nevertheless.
• #20: No specific mention of quality of reporting. This, I believe, is important because a child can have a relatively low number of incidents, say 8, but against a baseline of none or very few, represents a dramatic change in externalising behaviour from those with a consistently high level. One needs to make sure that the recording allows for such patterns to emerge. The idea of Raised achievement / attainment / learning is also easier to demonstrate and the testing of NC levels, reading, spelling etc have been used to demonstrate this. However, nobody has tried to establish a link between the behavioural data helping to ameliorate the learning.
• #21: Comments were made by the audience in agreement with one of the respondents that there was too little time to carry out any more assessments. I pointed out that it was “instead of” not alongside the current practice that I was looking at. There was a consensus of opinion that good assessment was a desirable thing but that the recording of progress is difficult because of the format of the assessment. In that if this data was to be used for Ofsted then there might be a desire to skew the data and this would be easy to achieve as judgements are somewhat simplistic. This broadly concurs with the findings of Elliot et al (2008 p.16) “Ratings of behaviour are evaluative judgments that are affected by the environment and a rater’s standards for behaviour.”