Corpus insider #1: Representativeness

As I was putting together my talk for the ELT Freelancers’ Awayday and the follow-up blog post, I realized that over 20 years of using corpora, there are a whole host of factors I’ve learnt to take into account. I touched very briefly on a few of them in my talk, but I thought some might be worth exploring further. So, this is the first in a series of posts about things you might need to bear in mind if you want to use corpus tools to inform your work on ELT materials.

When I explain to people what a corpus is, I usually start off by saying that it’s a large collection of language that we use to represent the way English is used as a whole. It seems a simple premise, but when you dig a bit deeper, it gets more complicated. As with any research, the validity of your results is dependent on the quality of your data. The data your chosen corpus contains will determine exactly what kind of language it can actually be said to represent and so how useful it is for your purpose. To take a simple example, if you were writing for a specifically British English market, using a corpus that contained only American data wouldn’t be very useful. Similarly, if you were working on speaking materials, looking at usage in a corpus of entirely written data wouldn’t really tell you much about how people normally speak. Understanding a bit about the corpus you plan to use, the data it contains, and what that might represent is absolutely essential before you start doing any corpus research.

Corpus types:

There are two main types of corpus, those which contain data drawn from one type of source or genre and those which are said to be ‘balanced’ and contain data from a wide variety of different genres. The first type includes purely spoken corpora (like the Spoken BNC2014), corpora of academic writing (of either published texts, like the academic part of COCA or student writing, like BAWE or MICUSP) and many corpora are composed largely of journalism, because it’s one of the simplest sources of data to collect, especially for those corpora that rely on web-based content (e.g. Monco, NOW, etc.).

Large balanced corpora, containing written and spoken data from a wide range of sources, are much more difficult to put together. For this reason, they’re mainly owned and maintained by large publishers, especially those who produce dictionaries, and aren’t publically available. The British National Corpus (BNC) is a balanced corpus that’s freely available, but it’s relatively small by modern standards and, perhaps more importantly, it’s becoming increasingly out of date (with data from the 1980s and 90s). The Corpus of Contemporary American English (COCA) sits in a mid-ground with data from spoken sources (although all radio transcripts rather than everyday conversation), fiction, popular magazines, newspapers and academic texts. It’s reasonably balanced, although all American as the name suggests.

The trouble with media hype:

Data from newspapers, magazines and blogs is very easy to collect and makes up a large proportion of many corpora. It can provide lots of interesting information about language used to talk about a wide range of topics, but it’s important to remember that journalism as a genre has its own quite marked features that don’t necessarily reflect the way that ordinary people use language day to day. It may seem obvious to say that journalists report news, but that means they’re generally writing about what’s new, surprising, shocking or problematic. They also want to draw their readers in and keep their attention with colourful language choices and hyperbole. For my recent talk, I demonstrated an example of a query about the language of social media and in particular, which verbs collocate with the noun ‘newsfeed’. I used the Monco corpus, because I was interested in up-to-date usage, and came up with the following verbs:

scroll through your newsfeed
pop up on your newsfeed
fill/flood/dominate/clog up your newsfeed

The first two feel like expressions you might use in conversation, the others, however, are clearly journalistic in style; bemoaning the way that a particular trend is overtaking our online lives. Searching a couple of other news-dominated corpora came up with similar results (enTenTen: spam/clutter/bombard/clog your newsfeed; NOW: scroll through/appear on/pop up on/tweak/flood/clog your newsfeed). They’re all interesting collocations, but they’re probably not the first ones you’d choose to teach an intermediate learner who wants to talk about the way they use social media themselves. That’s not to say you shouldn’t use these corpora when you're researching ideas for ELT materials, but knowing a corpus contains only or largely data from journalistic sources means that you can be on the lookout for this type of language and be selective about what you use as appropriate for the learners you’re writing for.

Professional and lay writers:

Unsurprisingly, the majority of written corpus data comes from published sources and, as such, it’s written by people who are professional writers: authors, journalists, copy-writers. As we saw with journalism, above, this can mean the language is more colourful and probably more varied than the average lay person typically tends to use. This came out very clearly in a recent study into academic vocabulary (Durrant, 2016*) which looked at how many of the words on the Academic Vocabulary List (based on a corpus of published academic writing) were actually used regularly by student writers (using a corpus of university-level student writing). It turned out that the student essays contained a vastly narrower range of vocabulary than the published academic texts, written by experienced (and edited!) academics. That’s not to say the student writing was in some way lacking – all the papers had got high marks – it’s just a different genre with different expectations. 

When you’re using a corpus to search for ideas, it’s all too easy to pick out examples and patterns that are elegant or appealing, but I think it’s always important to ask yourself how typical they are of what the average person might say or write. Is it a writerly flourish? Is it helpful as a model for your target learners?

I’m not saying that as ELT writers and editors we should reject all corpus evidence as flawed and unhelpful. Far from it, I think corpus tools can be incredibly helpful in backing up our intuitions and uncovering patterns of usage we might not have thought of, but they are just that, ‘tools’ and should be used with an element of caution. It's all too easy to be drawn in by a corpus that's new or especially large or has a nice interface and nifty tools, but making sure you know what your corpus represents is vital. If a collocation or pattern feels unlikely or overly fancy, then ask yourself why. Don’t just accept the first results that pop up, click through to the examples, scroll down to see where they come from and understand exactly what’s going on.


* There's a good summary of Durrant's study on ELT Research Bites.

Four Favourite Corpora

Recently, I gave a 10-minute talk at the ELT Freelancers’ Awayday in Oxford about “Simple corpus hacks for ELT editors”. I only had time to look at one corpus and a handful of searches, but I promised to share some of my other favourites in a blog post. So here goes…

1 Monco: In my talk, I looked at the Monco corpus. I chose it because it’s a monitor corpus, so it monitors current usage, updating with new data daily and as such, I find it useful for answering language questions that haven’t yet made it into conventional reference sources like dictionaries. For example, in my talk, we looked at how wellbeing (spelled as a single word) may be catching up with more traditional hyphenated form (well-being) that you’ll find in most dictionaries (simply by typing well-being|wellbeing into the search box). The split was 35% – 65% in Monco compared with 17% – 83% in the British National Corpus (with data from the 1980s and 90s). We also turned up some potentially useful verb collocates for newsfeed, including scroll through and pop up, which won’t have yet made it into a collocations dictionary. One of my favourite features of Monco, especially for the corpus novice, is its user-friendly search screen and its nice graphics for results.

On the downside, Monco’s data is drawn from entirely online news sources which means that it’s really only reflective of journalism, rather than language usage in general. And although it includes sources from the UK, US, Canada and Australia, it isn’t balanced, so there’s significantly more data from some sources than others – a factor to bear in mind that can skew the results.

2 Brigham Young University: Not strictly a single corpus, but a collection of different corpora available via the same site and the go-to source for lots of queries. Personally, I tend to use COCA (the Corpus of Contemporary American English) for checking US usage. It’s a large corpus containing a nice variety of contemporary sources (1990 – present), including radio & TV transcripts, fiction, newspapers, magazines and academic data. Through BYU, you can also find host a specialized corpora including a corpus of Wikipedia entries and even, slightly weirdly, the Hansard corpus of British parliamentary proceedings, should that happen to fit your purpose!

My main grumble with BYU is that I find the interface clunky and frustrating to use, especially with its rather distracting colour-coding.

3 BAWE and BASE: The British Academic Written English corpus (BAWE) and the British Academic Spoken English corpus (BASE) are composed of written and spoken data collected from university students at a number of British universities. The written corpus contains essays and other coursework which received a good pass mark and the spoken data includes lectures and seminars. I particular like these corpora because they’re an example of language as it might be used by the peers of the students we’re aiming at, rather than text produced by professional writers, journalists, academics, etc. which doesn’t necessarily provide an appropriate model for the average ELT student. This is obviously university-level language, so is especially relevant for EAP, but I think BAWE could be useful for any advanced students who need to write formal essays (IELTS, CAE, Proficiency). And if you’re looking for US academic equivalents, you could also check out MICUSP and MICASE.

BAWE and BASE are actually available via several sources, but I wanted the excuse to get you to experience Sketch Engine, for me, the gold standard when it comes to corpus tools and the interface used by all the major dictionary publishers for their large corpora.

4 Spoken BNC2014: I admit this is the corpus on my list that I’ve probably used least so far, but I’m including it because it’s one I’m quite excited about finding uses for. Slightly contrary to its name, it was only released in 2017 and is the result of a massive project to collect data about current spoken English used in everyday contexts. If you’re working on speaking materials, looking at evidence from written English is not going to tell you anything terribly useful, because we just don’t speak how we write. So I think this could become the go-to corpus for anyone who wants to know how people actually say things.

Unfortunately, the Spoken BNC2014 doesn’t have the most user-friendly interface and getting access involves a bit of a faffy sign-up process which could be off-putting for the casual user. If spoken language is your thing though, I think it’s worth investing the time and effort to check it out, not least because some of the content is just really funny!

A note about corpora and copyright: It’s important to remember that, in general, the data that appears in a corpus is liable to all the usual copyright restrictions. That means you can’t just pull a big chunk of language from the corpus and use it in your activity, especially not if it’s for commercial publication. Occasionally, of course, you come across very short, ‘vanilla’ examples which could have come from almost anywhere (A young woman opened the door. The traffic was particularly bad.), but to be honest, these are few and far between. Generally, when I search for a particular language item, I’ll scan through the examples and jot down a ‘frame’:

I/you scroll through my/your (Facebook) newsfeed to see/searching for/on the train …

Then I’ll use my notes as the basis for an example that keeps the feel and pattern of the ones I’ve looked at, but fits my teaching purpose … and doesn’t infringe copyright.

There are lots of different corpora out there and corpus fans will have their personal favourites. If you’re new to corpora though, I’d say pick one or two to check out, play around with a few simple searches, use the help to get you started, and see what’s most useful for you. Be warned though, it can be addictive!