Language data and permissions: AI vs corpora

Like everyone else lately, I haven’t been able to avoid hearing about generative AI, including a session at last week’s Freelancers’ AwayDay. One point in the debates around it though has jumped out at me, in particular. The language data collected to train the large language models that power the likes of ChatGPT is scraped from the internet with no attempt to get permission from the original creators of those texts. This has raised alarm bells with writers such as the US Authors Guild who are taking action on the issue.

As a lexicographer, I work with very large collections of language data every day: corpora. Working with publishers’ corpora on materials for publication I’m very aware that permission and copyright are issues we absolutely don’t ignore. The multi-billion-word corpora collected and held by dictionary publishers contain material that has been added with the permission of the copyright holders. This is generally the publishers of the texts rather than the individual writers allowing for the collection of large quantities of data, such as all the newspapers published by a particular media group or all the academic journals from a certain academic publisher.

That permission also comes with restrictions on how the data can be used. Specific agreements vary between corpora, but they usually include limitations about the length of excerpts that can be used, not generally a big problem when you need short dictionary examples. We are also generally required to edited examples drawn from a corpus so that they’re not obviously identifiable. Very ‘vanilla’ examples that could have come from anywhere – She left and closed the door behind her. – can safely be copied as they are, but any references to real people, places, events, or organizations will usually be removed, and often replaced with the minister, a company spokesperson, in the region, etc. Incidentally, this also has the advantage that the examples are less likely to date and will be accessible to a wider audience because they rely less on culturally-specific references.

As lexicographers, we do occasionally turn to other sources to research language, especially when we’re looking at new or niche uses for which we may have scant corpus evidence. In these cases, our editors are even more insistent about the need for caution. Ideally, we’ll refer to online sources to confirm how a word is being used, then try to make use of the few corpus examples we do have, informed by what we’ve seen elsewhere, to come up with appropriate example sentences.

I should note that I’m specifically talking about publishers’ corpora here. There are, of course, plenty of corpora out there, including numerous web corpora, where the issues around permission and copyright are very different. Many of these were originally collected for academic purposes, i.e. principally to research language usage rather than to publish commercial materials. They should also come along with notes about how they can be used – although I wonder how many users actually read the small print.

As writers ourselves, in whatever form, I think we should be especially aware of how the content we create is being used, and potentially abused, with and without our permission, and crucially, in turn, have respect for how we use the intellectual property of others.

#IVACS2018: new corpora and new collaborations

I came away from the IVACS conference in Malta a couple of weeks ago with my head buzzing with ideas which still haven't quite settled themselves down into a coherent order for a blog post, but here goes anyway ...

Lots of corpora:

Almost every speaker seemed to start off by describing the new corpus they'd compiled for their research. To the outsider, this might seem slightly pointless, constantly reinventing the wheel ... why not just use an existing corpus? I guess the answer is that the English language isn't a single homogeneous whole and if you want to study a particular area of usage or group of users, you do need to look at relevant, representative data. A lot of the corpora I heard about were fascinating, but a couple I took note of as being especially useful for ELT - and crucially, available or almost available to use - were ...

EFCAMDAT: This is a corpus of learner writing collected from the language school chain EF’s online learning platform, based on responses to a variety of tasks at a range of levels. I had come across it before and tried to register to use it but never got the confirmation email through. (Tried again this week, but I seem to have got stuck at the same point, sadly). I think it's potentially a useful addition to the CLC because of its wider range of task types. Ute Römer talked about her research into verb patterns using EFCAMDAT and it looked like it was bringing up a nice range of examples. I’ll keep trying to register …

Growth in Grammar: Phil Durrant described his work compiling and analysing a corpus of children's language. The project is aimed at understanding how (native speaker) UK school children's language develops between the ages of 5 and 16 and the corpus consists of writing kids have done in school, not tasks set by the researchers, just their day-to-day schoolwork. The corpus is due to be made publically available in the near future and it struck me as a potentially really useful resource for writers of ELT materials for young learners. A number of times lately, I've been asked about using corpora to inform YL materials and I've drawn a bit of a blank because most existing corpora consist firmly of grown-up data which just doesn't seem a relevant point of comparison. When you're asked to look at the authenticity of the language in a story about a talking octopus, the BNC really doesn't offer very much!! So I'll be keeping an eye open for when Growth in Grammar becomes available for all my YL ELT writer colleagues.

Research into practice:

A number of speakers were talking about the potential implications of their research for teachers and learners. Again, loads of interesting stuff, but I just want to highlight a couple of things that have stuck with me ...

Native/non-native teachers, teacher talk and ELF: Perhaps my favourite session of the whole conference was Eric Nicaise's brilliant paper on teacher talk.  He compiled a corpus of teacher talk in the ELT classroom by native and non-native speaker teachers (working in Belgium) to compare their use of language.  It revealed some fascinating differences which Eric was keen to stress weren't about making any kind of judgment, but about raising both groups of teachers' awareness of the type of language they use which might be providing a model for their students. Just two of the findings he had time to present were around the use of modals and phrasal verbs. He found that the NS teachers used a much wider range of modals in the classroom and used them for a wider range of functions. So, for example, while NS teachers would say things like “Would you like to … (turn to page …)” and "Shall we ... (look at …)?", NNS teachers would often go for much more direct imperatives “Turn to …” and “Repeat …” (examples from memory).  And perhaps unsurprisingly, when it came to phrasal verbs, the NS teachers again used far more scattered throughout their speech, whereas the NNS teachers used fewer and they tended to be mainly when they we were consciously teaching and exemplifying phrasal verbs.

For me, this set my head whizzing with all kinds of ideas. Eric mentioned the idea of language that's perhaps more suitable for teacher talk at lower levels where NS teachers might want to consciously use simpler forms and it also struck me that avoiding the trickier forms might apply to learners in ELF contexts ... why use confusing phrasal verbs if you're a French L1 speaker talking to an Italian and you'll both find Latinate verbs much easier?! Then from the opposite perspective, if learners are underexposed to a full range of modals, for example, they risk coming across to native speakers they communicate with (and expert NNSs too) as overly direct and even rude. This has implications for NNS teachers who might want to try and model more of these forms in their TT, especially at higher levels, and for learners, who need to be aware of the effect their language choices can have on the impression they create.

Collaboration between researchers and materials writers: One of the key points I tried to make in my own session was about how we can feed insights from corpus research into classroom materials and this was a theme that cropped up again and again in other talks with various researchers talking about the importance of translating their research into practical applications. For some, with very specific research objectives, that meant applying it directly to their own teaching context, but several also expressed the need for more collaboration between researchers and mainstream materials writers. In the Q&A at the end of Ute Römer's great plenary, I ventured to suggest a direct collaboration between corpus researchers and MaWSIG (the IATEFL materials writing group) ... a suggestion that seemed to be received enthusiastically. 

Conference selfie via @uroemer

So often it seems to me that freelance ELT writers would like to keep up with the latest research but they come up against various barriers (a point picked up too by Clare Maas at the recent MaWSIG/Oxford Brookes event). Most academic research is behind paywalls which make it difficult to access if you're not attached to a university – I couldn’t, for example, access the papers on which some of the talks mentioned above were based in order to recheck details. Of course, there are ways around this if you know what you're particularly interested in (I could have emailed the researchers directly), but without access to the kind of search facilities found in university libraries, how do you know what to look for? Finding out about relevant research becomes rather pot luck, down to happening across things shared on social media, reviewed in open access sources (such as ELT Research Bites) or mentioned at conferences, and for each person is usually restricted to the highlights in their own specialist area. And then there's time ... as a freelancer, time is money and most of us simply can't afford to spend lots of unpaid hours ploughing through long academic articles in search of relevant insights that we might be able to make use of when we have writing deadlines to meet. It seems to me that more direct contact between researchers and writers would be really helpful in sharing knowledge in a way that's friendly and accessible. Hearing about research face-to-face with the option to chat and ask questions, to clarify any unfamiliar academic terminology, to share ideas on applicability and tease out any important caveats would be really helpful. I don't know quite what form this might take or who we could invite, but I'll definitely be talking to the MaWSIG crew about it as a possibility.

Four Favourite Corpora

Recently, I gave a 10-minute talk at the ELT Freelancers’ Awayday in Oxford about “Simple corpus hacks for ELT editors”. I only had time to look at one corpus and a handful of searches, but I promised to share some of my other favourites in a blog post. So here goes…

1 Monco: In my talk, I looked at the Monco corpus. I chose it because it’s a monitor corpus, so it monitors current usage, updating with new data daily and as such, I find it useful for answering language questions that haven’t yet made it into conventional reference sources like dictionaries. For example, in my talk, we looked at how wellbeing (spelled as a single word) may be catching up with more traditional hyphenated form (well-being) that you’ll find in most dictionaries (simply by typing well-being|wellbeing into the search box). The split was 35% – 65% in Monco compared with 17% – 83% in the British National Corpus (with data from the 1980s and 90s). We also turned up some potentially useful verb collocates for newsfeed, including scroll through and pop up, which won’t have yet made it into a collocations dictionary. One of my favourite features of Monco, especially for the corpus novice, is its user-friendly search screen and its nice graphics for results.

On the downside, Monco’s data is drawn from entirely online news sources which means that it’s really only reflective of journalism, rather than language usage in general. And although it includes sources from the UK, US, Canada and Australia, it isn’t balanced, so there’s significantly more data from some sources than others – a factor to bear in mind that can skew the results.

2 Brigham Young University: Not strictly a single corpus, but a collection of different corpora available via the same site and the go-to source for lots of queries. Personally, I tend to use COCA (the Corpus of Contemporary American English) for checking US usage. It’s a large corpus containing a nice variety of contemporary sources (1990 – present), including radio & TV transcripts, fiction, newspapers, magazines and academic data. Through BYU, you can also find host a specialized corpora including a corpus of Wikipedia entries and even, slightly weirdly, the Hansard corpus of British parliamentary proceedings, should that happen to fit your purpose!

My main grumble with BYU is that I find the interface clunky and frustrating to use, especially with its rather distracting colour-coding.

3 BAWE and BASE: The British Academic Written English corpus (BAWE) and the British Academic Spoken English corpus (BASE) are composed of written and spoken data collected from university students at a number of British universities. The written corpus contains essays and other coursework which received a good pass mark and the spoken data includes lectures and seminars. I particular like these corpora because they’re an example of language as it might be used by the peers of the students we’re aiming at, rather than text produced by professional writers, journalists, academics, etc. which doesn’t necessarily provide an appropriate model for the average ELT student. This is obviously university-level language, so is especially relevant for EAP, but I think BAWE could be useful for any advanced students who need to write formal essays (IELTS, CAE, Proficiency). And if you’re looking for US academic equivalents, you could also check out MICUSP and MICASE.

BAWE and BASE are actually available via several sources, but I wanted the excuse to get you to experience Sketch Engine, for me, the gold standard when it comes to corpus tools and the interface used by all the major dictionary publishers for their large corpora.

4 Spoken BNC2014: I admit this is the corpus on my list that I’ve probably used least so far, but I’m including it because it’s one I’m quite excited about finding uses for. Slightly contrary to its name, it was only released in 2017 and is the result of a massive project to collect data about current spoken English used in everyday contexts. If you’re working on speaking materials, looking at evidence from written English is not going to tell you anything terribly useful, because we just don’t speak how we write. So I think this could become the go-to corpus for anyone who wants to know how people actually say things.

Unfortunately, the Spoken BNC2014 doesn’t have the most user-friendly interface and getting access involves a bit of a faffy sign-up process which could be off-putting for the casual user. If spoken language is your thing though, I think it’s worth investing the time and effort to check it out, not least because some of the content is just really funny!

A note about corpora and copyright: It’s important to remember that, in general, the data that appears in a corpus is liable to all the usual copyright restrictions. That means you can’t just pull a big chunk of language from the corpus and use it in your activity, especially not if it’s for commercial publication. Occasionally, of course, you come across very short, ‘vanilla’ examples which could have come from almost anywhere (A young woman opened the door. The traffic was particularly bad.), but to be honest, these are few and far between. Generally, when I search for a particular language item, I’ll scan through the examples and jot down a ‘frame’:

I/you scroll through my/your (Facebook) newsfeed to see/searching for/on the train …

Then I’ll use my notes as the basis for an example that keeps the feel and pattern of the ones I’ve looked at, but fits my teaching purpose … and doesn’t infringe copyright.

There are lots of different corpora out there and corpus fans will have their personal favourites. If you’re new to corpora though, I’d say pick one or two to check out, play around with a few simple searches, use the help to get you started, and see what’s most useful for you. Be warned though, it can be addictive!

Lexicographic leeway: the case of quagmire

Recently, I chatted to Lindsay Clandfield for an episode of the TEFL Commute podcastabout dictionaries. When I listened back to the full episode, lots of things that came up in the chat between Lindsay and his co-presenter, Shaun Wilden had me wanting to chip in. Perhaps one of the most interesting was Shaun's comment that he always tests a new dictionary by looking up the entry for 'quagmire' to see whether the literal, concrete sense (a wet, boggy area of ground) or the metaphorical sense (a messy situation) is listed first. It's an interesting test and, I think, a case worth explaining from a lexicographer's perspective.

The frequency principle:

In general, modern learner's dictionaries are based on the principle of frequency at all levels. Whether it's which words to include, how to order the different senses of a word or which collocations and patterns to illustrate in the example sentences, the most frequent typically take priority. The thinking being that the most frequently used words and senses are likely to be most useful to a learner, so they should get priority. That frequency information comes from a corpus (a computerised database of language consisting of hundreds of millions of words from books, newspapers, magazines, TV, radio, websites, conversation, etc. which we use to represent the language as a whole). The frequency of individual words can be retrieved quite easily, but determining the frequency of different senses comes down to the judgement of the individual lexicographer; computers can't yet distinguish meaning. So when compiling an entry, you might get up a sample of, say, 500 examples and scan through to work out which is the most frequent sense. Looking at 'quagmire' on the British National Corpus (BNC) the split is roughly 54% concrete uses vs. 46% metaphorical uses. Undoubtedly, that would vary for different corpora, but it's clearly a close call in terms of frequency.

Meaning and metaphor:

There are, however, cases where other factors override the general frequency rule. In the case of words with a concrete and a metaphorical meaning, the question is raised as to whether it's easier for a learner to understand a metaphorical usage by explaining the physical one first. Is it easier to understand the idea of a group of politicians bogged down in a metaphorical quagmire, if you can picture them stuck in a load of squelchy mud? Some vocabulary acquisition research argues that encouraging learners to create visual representations of words, either literally as drawings or at least in their mind's eye, aids both comprehension and retention. And of course, there's all the language that comes along with the original metaphor. To understand why we talk about people being bogged down in, sinking into or wading through quagmires (all collocations that came up in my corpus search at metaphorical uses), does it help to understand the concrete, physical sense too?

Styleguides and individual judgments:

Each dictionary will have rules about these types of cases set out in a huge document, called a styleguide, which lexicographers refer to as they're compiling entries. In some cases, that will be a hard-and-fast rule (frequency always first or concrete before metaphor), but often the styleguide will leave it down to the judgement of the individual compiler to weigh up the relative frequencies of the two senses and how useful the physical sense is in understanding the metaphorical one. That's why you'll find a different treatment of 'quagmire' in different learner's dictionaries (Oxford and Cambridge go for concrete first, COBUILD and Macmillan start with the metaphorical usage); it's one of those cases where you could just argue either way.

I'll leave you with a few more to think about yourself - if you were explaining the metaphorical uses of these in the classroom, would you start by describing the concrete, physical sense first or not? Which factors would influence your decision?

inundate, swamp, battle, grasp, tweet, mammoth, giant, nightmare