The occasional ramblings of a freelance lexicographer

Thursday, March 21, 2013

Research and evidence in ELT

After the slightly surprising appearance of Ben Goldacre (Guardian science journalist) on last night’s #ELTchat about classroom research (here’s the transcript), I went to bed musing over research and evidence in ELT. It didn’t keep me awake for too long admittedly, but it seemed worth sharing a few of my thoughts here.

First, to explain a bit of background, Ben Goldacre has just written a report for the UK Department for Education about how some of the methods used in science, and particularly Medicine, could be used to provide a more evidence-based approach to education, including in particular randomized trials to determine best practice. His recent Guardian article sets out his basic arguments, or you can read his full report here. Whilst what he has to say makes interesting reading and seems eminently sensible, it did leave me with several nagging “yes buts”.

Yes, but … we do already use research evidence to inform ELT

As an occasional corpus researcher myself, I’m very aware of the huge amount of corpus research that has gone on and is going on using both native speaker and learner corpora in order to determine what language (both vocabulary and grammar) is most useful to teach and how to prioritize what to teach first. This is perhaps most obviously demonstrated in the published teaching materials that a lot of this research feeds into, but it also permeates the profession in more general ways, such as with Averil Coxhead’s Academic Word List which has spread widely in the world of EAP teaching.

Also as someone involved in EAP, I’m always hearing how important it is for EAP practitioners to be involved in research in order to gain the respect of the wider academy (for those of you not in EAP that translates as staff teaching EAP in universities showing that they’re proper academic lecturers by doing research).  And I know that a lot of EAP folks, especially those with proper university posts, put a lot of effort into research.

I’m less up-to-date with other ELT research, but from what I can think of off the top of my head, I suspect that a lot of ELT research generally  is about what language to teach (the corpus research) and how students learn (second language acquisition), rather than so much about teaching practice – the focus of Goldacre’s report. And I also suspect that what research there has been into the effectiveness of different classroom practices is rather small-scale and not always widely applicable. 

Several people in last night’s #ELTchat brought up Penny Ur’s talk at last year’s IATEFL conference It’s all very well in theory but …  about how teachers don’t read and keep up-to-date with research. It was an interesting talk and one point in particular caught my attention enough to follow it up. She pointed to research that suggested teaching lexical sets (a common practice in ELT) was not an effective way to teach vocabulary. As lexical sets in some form are quite prominent in some of the materials I work on, I was a bit worried so followed this up.  When I read the original paper*, I discovered that firstly, it actually only concluded that the practice was not effective with beginner level students (presumably because you’re throwing a whole new set of vocabulary at them and they have no way of processing it, whereas intermediate+ learners already have existing knowledge to slot it in with; a place to file it). Secondly, it was also a very small-scale study and the two groups of learners used (beginner adults and intermediate children) were not directly comparable. That’s not to dismiss the study out of hand, it does raise some very interesting ideas, but it’s clearly not widely generalizable and it certainly doesn’t fall into the kind of wide-scale, systematic, randomized trial that Goldacre is advocating.

It does, however, bring me to my second nagging doubt …

Yes, but … will it work in ELT?

I can see how the population of mainstream school students in the UK can provide an excellent population to study systematically, because although they clearly exhibit a degree of variability, they also share enough common characteristics to be able to generalize the findings of any research across the system. I can see how you could conduct a randomized trial across a large number of classes at the same level, of roughly the same age, in similar size classes, studying the same subject for a similar number of hours per week and across a whole academic year, say. How often could you do that in ELT?! As if I wasn’t already aware from my own varied teaching background, the discussions on #ELTchat, and even on the more specialized #EAPchat, time and again throw up how many different contexts there are in ELT and how different the issues thrown up in different situations can be. It’s much more difficult to compare a class of Greek kids in a private language school, with a group of mixed nationality teens on a two-week summer course, and a businessman taking one-to-one lessons, who could all feasibly be studying, say, pre-intermediate English. Then when you throw in the practical issues of time (many ELT students don’t provide a full-time captive audience), commercial interests (much ELT teaching goes on in the private sector) , lack of a single overall ‘system’, not to mention cultural differences, it all starts to look incredibly messy.

Does all that mean we shouldn’t be conducting research or trying to feed it into classroom practice? Of course not.  I think the goal of such wide-scale systematic research is a really great one and I completely agree with the title of Goldacre’s Guardian article Teachers need to drive the research agenda. But with any research, you have to start off by establishing the whys, whats and hows first and in an area as diverse and messy as ELT, I think that’s quite a challenge. 

* Papathanasiou, E. (2009). An investigation of two ways of presenting vocabulary. ELT Journal, 63(4), 313-322.

Labels: , , , , ,


Anonymous Anonymous said...

Hi Julia and thanks for sharing this post - I thought it ought to generate more discussion but, sadly, it didn't.

One of my dissertations on my applied linguistics MA involved comparing the 'scientific research' paradigm vs the class research paradigm and I do share your doubts about all this 'evidence based' drive which, in my mind, suggests a lack of understanding the difference between adding this chemical or not adding it to some medication and teaching this way or that.

I really don't know of any research on learning and the effects of teaching on learning which I can call conclusive.

I was amused with what you wrote about Mrs Penny Ur's misquoting of the results of the research she mentioned .

Just goes to show you :-)

Thinking of publishing that piece myself - you have inspired me.


5:17 pm  
Blogger The Toblerone Twins said...

Hi Marisa,

Thanks for taking the time to comment.

To be fair, I think Ben Goldacre's comparison between medicine and education was more focused on clinical practice (i.e. dealing with patients, managing conditions, etc.) rather than more measurable drug trials etc. So perhaps the parallels are more valid. Even so, I think ELT in particular is a much more tricky area to look at from that type of research perspective.

Most of my background is in language research, so much more easily researchable and measurable parameters. Over the past year though I've been involved in writing a coursebook and I've started to ask a lot more questions about what actually works in the classroom. I know I ought to read up more on the research that is already out there, but as with everything, it's a matter of fitting it in.

I definitely think it's a topic worth returning to on #ELTchat, although I suspect you may get a similar split between those with a more heavy academic research bias (usually working in universities) and those with a more practical, classroom approach (on-the-ground teachers in everyday teaching contexts). As someone involved in EAP, it's a split I come across quite a bit and rather find myself trying to straddle ;)


5:39 pm  
Blogger Leo said...

Hi Julie
Really liked your post - perhaps because I'm also interested in corpus research.

I didn't hear Penny Ur's talk last year but as her former MA advisee I can imagine the issues she addressed (must be similar to her recent article in the Guardian here: http://gu.com/p/3bvee/

On the issue of lexical sets it's the first time I've seen the article you and she mentioned but there were a couple of studies in the 1990s that showed that teaching new words in semantic sets impedes learning: Tinkham, Waring etc. (can give you full references if you like). You're right the article concerns beginner students but teaching in semantic sets (animals, colours, items of furniture) mainly occurs in beginner courses.

Enjoyed reading this post as I do with your blog in general :)

8:27 am  
Blogger The Toblerone Twins said...

Hi Leo,

Thanks for your comment, glad you enjoyed the post.

Yes, I can see that teaching a whole set of new vocab in a sematic set wouldn't necessarily be effective. I work mainly at higher levels, and more specifically in EAP, so when I'm looking at lexical sets, usually a large proportion (if not all) the vocab will already be familiar to the learners. So the activity will often be about getting them to use a wider range of language for a particular function, say reporting verbs, pulling stuff from their passive to their active vocab; experimenting, stretching themselves a bit more.

The thing that made me smile about Penny Ur's talk (which, by the way, was very good and very thought-provoking) was that she rather fell into the trap that she was telling us to avoid - slightly overgeneralizing a piece of research to make her own point. Always a bit of a minefield ;)


5:51 pm  
Blogger Patrick Andrews said...

Thanks for this interesting posting. I agree with a lot of what you say about the importance of context in what we think about in ELT and I think we are aware of the details that affect teaching and learning in our contexts.

It is an unfortunate fact that most ELT teachers do not get enough time to do a large amount of reading. I do try but there are so many strands of interest for us to keep up with.

3:12 pm  
Blogger The Toblerone Twins said...

Hi Patrick,

Thanks for the comment. I know exactly what you mean about not having the time to read. I feel like I'm reading a bit more since being more connected online (via Facebook, Twitter etc.) - I click through to stuff that looks interesting, but it's a bit random and largely depends on when something pops up and how busy I am. I know I ought to be a bit more systematic and keep a list of interesting links and articles, but it comes some way down a very long 'to-do' list of other work!

See you in the summer? I'm doing the 6-week pre-sessional this year.


4:50 pm  
Blogger Patrick Andrews said...

Not working at Bristol Uni this summer as I have a lot of other things on.

Do you use delicious or some other social bookmarking to keep links organised? I find delicious very useful.

2:12 pm  

Post a Comment

<< Home