Research and evidence in ELT

After the slightly surprising appearance of Ben Goldacre (Guardian science journalist) on last night’s #ELTchat about classroom research (here’s the transcript), I went to bed musing over research and evidence in ELT. It didn’t keep me awake for too long admittedly, but it seemed worth sharing a few of my thoughts here.

First, to explain a bit of background, Ben Goldacre has just written a report for the UK Department for Education about how some of the methods used in science, and particularly Medicine, could be used to provide a more evidence-based approach to education, including in particular randomized trials to determine best practice. His recent Guardian article sets out his basic arguments, or you can read his full report here. Whilst what he has to say makes interesting reading and seems eminently sensible, it did leave me with several nagging “yes buts”.

Yes, but … we do already use research evidence to inform ELT

As an occasional corpus researcher myself, I’m very aware of the huge amount of corpus research that has gone on and is going on using both native speaker and learner corpora in order to determine what language (both vocabulary and grammar) is most useful to teach and how to prioritize what to teach first. This is perhaps most obviously demonstrated in the published teaching materials that a lot of this research feeds into, but it also permeates the profession in more general ways, such as with Averil Coxhead’s Academic Word List which has spread widely in the world of EAP teaching.

Also as someone involved in EAP, I’m always hearing how important it is for EAP practitioners to be involved in research in order to gain the respect of the wider academy (for those of you not in EAP that translates as staff teaching EAP in universities showing that they’re proper academic lecturers by doing research).  And I know that a lot of EAP folks, especially those with proper university posts, put a lot of effort into research.

I’m less up-to-date with other ELT research, but from what I can think of off the top of my head, I suspect that a lot of ELT research generally  is about what language to teach (the corpus research) and how students learn (second language acquisition), rather than so much about teaching practice – the focus of Goldacre’s report. And I also suspect that what research there has been into the effectiveness of different classroom practices is rather small-scale and not always widely applicable. 

Several people in last night’s #ELTchat brought up Penny Ur’s talk at last year’s IATEFL conference It’s all very well in theory but …  about how teachers don’t read and keep up-to-date with research. It was an interesting talk and one point in particular caught my attention enough to follow it up. She pointed to research that suggested teaching lexical sets (a common practice in ELT) was not an effective way to teach vocabulary. As lexical sets in some form are quite prominent in some of the materials I work on, I was a bit worried so followed this up.  When I read the original paper*, I discovered that firstly, it actually only concluded that the practice was not effective with beginner level students (presumably because you’re throwing a whole new set of vocabulary at them and they have no way of processing it, whereas intermediate+ learners already have existing knowledge to slot it in with; a place to file it). Secondly, it was also a very small-scale study and the two groups of learners used (beginner adults and intermediate children) were not directly comparable. That’s not to dismiss the study out of hand, it does raise some very interesting ideas, but it’s clearly not widely generalizable and it certainly doesn’t fall into the kind of wide-scale, systematic, randomized trial that Goldacre is advocating.

It does, however, bring me to my second nagging doubt …

Yes, but … will it work in ELT?

I can see how the population of mainstream school students in the UK can provide an excellent population to study systematically, because although they clearly exhibit a degree of variability, they also share enough common characteristics to be able to generalize the findings of any research across the system. I can see how you could conduct a randomized trial across a large number of classes at the same level, of roughly the same age, in similar size classes, studying the same subject for a similar number of hours per week and across a whole academic year, say. How often could you do that in ELT?! As if I wasn’t already aware from my own varied teaching background, the discussions on #ELTchat, and even on the more specialized #EAPchat, time and again throw up how many different contexts there are in ELT and how different the issues thrown up in different situations can be. It’s much more difficult to compare a class of Greek kids in a private language school, with a group of mixed nationality teens on a two-week summer course, and a businessman taking one-to-one lessons, who could all feasibly be studying, say, pre-intermediate English. Then when you throw in the practical issues of time (many ELT students don’t provide a full-time captive audience), commercial interests (much ELT teaching goes on in the private sector) , lack of a single overall ‘system’, not to mention cultural differences, it all starts to look incredibly messy.

Does all that mean we shouldn’t be conducting research or trying to feed it into classroom practice? Of course not.  I think the goal of such wide-scale systematic research is a really great one and I completely agree with the title of Goldacre’s Guardian article Teachers need to drive the research agenda. But with any research, you have to start off by establishing the whys, whats and hows first and in an area as diverse and messy as ELT, I think that’s quite a challenge. 

* Papathanasiou, E. (2009). An investigation of two ways of presenting vocabulary. ELT Journal, 63(4), 313-322.

Why mistakes matter

Last week, I clicked on a link on a friend’s Facebook page to read an interview in an online fashion magazine. As I started to read, I found that I had to reread the first few lines a couple of times and still couldn’t quite get the flow of the writing. Thinking that in a slightly trendy, arty publication the writer was trying to achieve some kind of creative effect, I read on. I kept, however, stumbling over sentences, having to go back and reparse again and again. Inevitably, given my day job, I started to analyse what it was that was troubling me about the writer’s style, and I started to pick out grammatical errors; missing subjects, mismatched subjects and verbs, awkward parallel constructions. After a while, the awkwardness of the grammar started to really irritate me and eventually, became so tiring, I gave up reading.  It was only then that I noticed the name of the writer and after a bit of clicking around, realized that the website was based in Spain and quite possibly not written, or I suspect even edited, by native English speakers. This perhaps explained the slightly odd writing style, but my first impressions stuck and I couldn’t summon up the energy or ‘understanding’ to go back and finish the article. I don’t mean to be disparaging about the writer’s attempts to write in English. He was clearly a very proficient English speaker and had been ambitious in his writing style and very nearly pulled it off – the errors were not basic, but generally stemmed from his use of more challenging structures. But it seems to me, that if you’re going to publish for an international audience in English (or any language come to that), then you really have to get your writing edited by a native (or near-native) speaker.

I’ve long been interested in the area of learner errors, especially through my long-standing work with the Cambridge Learner Corpus. When I started doing talks about learner errors and how to help students eliminate them, I often came up against resistance along the lines of; but shouldn’t we be encouraging fluency and confidence, not focusing on errors all the time? And I would find myself explaining that yes, of course fluency and confidence are very important, especially in spoken communication, and no, I wasn’t advocating a focus on error correction “all the time”. I do firmly believe though that if we’re going to teach writing skills, then helping students to identify, correct and eventually perhaps eliminate errors has to be a part of that process. And in some contexts, a very important part.

In my own current area of interest, EAP, we bang on a lot about critical thinking and we encourage students to ask critical questions about the accuracy, reliability and credibility of information. These are all qualities that are highly valued in academia – if you’re going to make a claim, your arguments and evidence have to be clear, unambiguous and precise. If a student hands in a piece of writing to their subject tutor that contains inaccuracies or ambiguities, they will quite likely question the students’ understanding of the topic before they put the deficiencies down to language errors.

It seems to me that we’re selling our students short if we mark a piece of written work littered with language errors as “good”, when it clearly isn’t (a brief nod to Jim Scrivener there). Ever since I’ve been involved in ELT, there seems to have been a general distinction made between errors which hamper meaning (bad and to be marked down) and those which don’t (okay to let slide). Whilst that may be valid where simply conveying a message by whatever means possible is our aim, especially in high stakes writing, I don’t think that’s always enough. In the same way that I got tired and irritated by the awkward grammar of my Spanish fashion journalist, a subject tutor ploughing through a pile of student essays may equally feel the linguistic strain placed on them by the errors of their international students, even where they don’t directly impact on the basic meaning.  Even apparently minor errors in academic writing can undermine the writer’s credibility and the degree to which their reader is persuaded of their argument.

So for me, regular error analysis and correction (in a variety of different forms) and occasional activities on ‘basic’ areas of grammar (articles, prepositions, subject-verb agreement) should always have a place in any EAP teacher’s repertoire, even at the highest levels.