Idiom Teacher
Problems in designing a CALL Package

Upper secondary, adult
Hélène Stengers, June Eyckmans, Arnout Horemans, Frank Boers, Erasmus College , Brussels

Introduction

In 2001 a team of language teachers and researchers started developing an on-line instrument to help their students of modern languages and translation at the Erasmus College, Brussels comprehend and remember a comparatively large number of English idiomatic expressions. The instrument, which they have called Idiom Teacher, includes about 400 idioms taught using different types of exercise. Each type of exercise targets about 25 expressions as a series before the same expressions are targeted again in another type of exercise.

The first type of exercise is a multiple-choice task in which students are asked to indicate which 'source domain' they think the expression is derived from. For example, the students are presented with the expression Be up in arms and are asked whether the origin of this idiom lies in "entertainment", "health" or "war". As feedback they are informed (on the computer screen) that arms in the expression refers to weapons and that the source domain is warfare. The second type of exercise is a more traditional multiple-choice task in which students are asked to match the idiom with the correct definition of its figurative meaning. For example, Be up in arms is matched with "being very angry and protesting strongly" rather than any of the distracters. The third type of exercise is a gap-fill task in which the idiom is embedded in a meaningful context and the students are asked to fill in the missing keyword, for example in "The unions were up in ___________ when the management announced the factory was to be closed down".

The team were inspired to include the first type of exercise (i.e., the 'identify-the-source-domain' task) by three theories:

1. Cognitive Linguistics (e.g., Langacker, 1991), and more specifically Cognitive Semantics (e.g., Lakoff 1987), the central tenet of which is that language is not arbitrary but rather 'motivated'. Applied to idioms this means that, while the lexical make-up and meaning of idioms is not fully predictable, it can nonetheless often be 'explained' in retrospect. For example, the figurative meaning of many idioms can often be inferred from its original, literal usage (like in the Up in arms example). This 'motivated' nature of many idioms invites insightful learning.
2. Levels-of-processing theory (e.g., Cermak & Craik, 1979), which holds that information that is processed at a deep cognitive level, for example through investing cognitive effort in making associations, is better retained in memory than information that is processed at a more shallow level, for example through rote learning or blind memorisation. The team were convinced that inviting their students to invest cognitive effort first in trying to trace back the idioms to their source domains and subsequently in trying to infer the figurative meaning of the idioms from their original, literal usage would promote deep processing and thus retention in memory.
3. Dual Coding theory (e.g., Paivio, 1986), which holds that associating a verbal form with a mental image will facilitate recollection. The team believed that informing their students of the literal, original usage of an idiom would call up a mental image of a concrete scene, which could then be stored in memory alongside the verbal form, as recommended by Dual Coding theory.

A dummy runⁱ

When Idiom Teacher was first implemented in 2002, the first type of exercise, i.e., the 'identify-the-source-domain' multiple choice, offered five options (one correct and four distracters). Students were required to click options until they got the correct one. Only then did they receive feedback (i.e., information about the origin of the expression) and only then could they proceed to the next item. The team assumed this set-up would maximise their students' investment of cognitive effort (the assumption being that students would need to think especially hard to trace back an idiom to its origin if there were a lot of distracters and that they would be especially intrigued if their first choice was rejected). It would soon become clear this was a rather naïve assumption.

The on-line exercises produced very encouraging results with regard to students' recollection of idioms whose source domains they had found rather easily. For about 70% of the expressions, the students needed one or two attempts in the 'identify-the-source-domain' multiple choice task to be allowed to move on to the feedback and the next item. These were significantly more likely to be recollected in the gap-fill exercise than the idioms whose source domains students had found more difficult to identify (three or more attempts). The team then had a closer look at the idioms whose source domains had proved to be so opaque to their students. There were several reasons why students had been unable to guess their origins, including the following:

1. Some keywords were low-frequency items and the students simply did not know the literal meaning of them: e.g., At the end of one's tether; Be at loggerheads; Get short shrift.
2. Some keywords were false-friends and the students confused them with words in their mother tongue: e.g., the word 'contention' in A bone of contention was mistakenly interpreted as 'happiness' because 'content' means 'satisfied' in Dutch (and hence the idiom was interpreted as 'a great source of satisfaction' in the second type of multiple-choice exercise; the word 'harness' in Work in harness was mistakenly interpreted as 'suit of armour' because it resembles the Dutch word 'harnas' (which happens to mean 'suit of armour').
3. Some keywords had more common literal senses, belonging to other source domains than the one the idioms were derived from: e.g., 'grain' in Go against the grain (here the line of fibre in wood, and nothing to do with the source domain of agriculture); 'suit' in Follow suit (here a type of playing card, and nothing to do with the source domain of clothes).

Idioms like these, whose source domains the students failed to identify in one or two attempts, were less well retained, despite the fact that the feedback with information about the original, literal usage of these expressions should have called up mental images and thus promoted the mnemonic effects of dual coding. Although the students had spent more time (and the team assumed also more effort) trying to get through the multiple-choice items on these particular idioms, they were less well retained. This seemed to contradict Level-of-processing theory.

It was time to talk to the students.

Picking the students' brainsⁱⁱ

The team carried out five informal interviews with students who had had ample experience with the on-line exercises. Although the students expressed their appreciation of Idiom Teacher in general, and of the information about the literal, original usage of the idioms in particular, they were quite sceptical about having to go on clicking the multiple-choice options until the right choice was made. If the source-domain was fairly transparent, the task was felt to be gratifying and the feedback about the idiom's origin was quickly taken on board as a confirmation of what the student had suspected. If the source-domain was opaque, simply because the student did not know the literal meaning of the keyword (e.g., At the end of one's tether), the student resorted to blind guessing until the correct choice was made by coincidence. Blind guessing, of course, hardly qualifies as deep-level processing. This part of the activity was actually felt to be a waste of time. If the source-domain looked transparent because of the presence of a keyword whose meaning the student thought she knew, such as in Work in harness and Follow suit, the student kept obstinately clicking the deceitfully attractive distracter (i.e., 'war' and 'clothes', respectively), as if convinced that "the computer was not listening". When eventually the student gave up and randomly clicked until she got to read the feedback, the reaction was often one of frustration and feeling tricked: "How could I possibly have known this?" Some of the students actually confessed that they had not bothered to read the information about the origins of the idioms if it had taken them too long to get the multiple-choice task over with. Especially comparatively long explanations (more than three lines on the computer screen) had been skipped in order to "get the exercise over with".

In short, the informal interviews with students brought home a simple but painful message: the team had overlooked the risk of negative affect when it came to idioms whose source domains were experienced as opaque.

Back to the drawing boardⁱⁱⁱ

To dampen the risk of negative affect and to enhance the probability of students' taking in the feedback with information about the idioms' literal origins (and thus to maximise the likelihood of dual coding through mental imagery), the team decided to make a couple of relatively small changes to the task design in the 'identify-the-source-domain' multiple choice:

1. The number of options was reduced to only three (one correct option and two distracters)
2. Feedback with information about the literal, original usage of the idiom was given already after one attempt at identifying the source domain
3. Longish explanations in the feedback were shortened where possible.

Back to the drawing boardⁱⁱⁱ

Idiom Teacher was then implemented again in 2003, with a new generation of students. Item analyses of the new version revealed the proposed method might work across the idiom board, after all: the gap-fill scores showed no more difference in the tool's effectiveness between idioms whose source domains were experienced as transparent and those whose source domains were harder to identify. Both categories of idioms were recollected equally well in the gap-fill exercises: average scores of around 69% on expressions that the students were previously unfamiliar with. It appears that giving the (brief) feedback faster and allowing students to proceed to the next item faster was sufficient to avoid frustration and to put students in the right mind to really process the information along lines recommended by the (still firmly standing) Levels-of-processing theory and the Dual Coding theory.

Conclusion

Summing up, the story of Idiom Teacher could be read as a dual reminder:

1. It demonstrates how important it can be not to be concerned solely with so-called cognitive variables when designing CALL tasks, but to give due consideration to affective variables, too.
2. It demonstrates how CALL designers can sometimes fail to fully anticipate exactly how students will experience the tasks they are given, and that it can therefore be very helpful to actually talk (and listen!) to them about this.

References

Boers, F., M. Demecheleer, & J. Eyckmans (2004). Etymological elaboration an a strategy for learning idioms. In Vocabulary in a second language: selection, acquisition and testing, Paul Bogaards and Batia Laufer (eds.), 53-78. Boers, Frank, June Eyckmans, and Hélène Stengers (2005, forthcoming). Presenting figurative idioms with a touch of etymology: more than mere mnemonics? Language Teaching Research. Cermak, L. and F. Craik (1979). Levels of processing in human memory. Hillsdale: Erlbaum. Lakoff, George (1987). Women, fire and dangerous things: what categories reveal about the mind. Chicago: Chicago University Press. Langacker, R. (1991). Foundations of Cognitive Grammar. Vol. 2: Descriptive Applications. Stanford: Stanford University Press. Paivio, A. (1986). Mental Representations. Oxford: Oxford University Press.

ⁱIdiom Teacher has served various learning experiments over the past three years. A more detailed and in-depth account of these experiments can be found in Boers, et al. (2004), and Boers, et al. (2005, forthcoming).
ⁱⁱOne of the students who gave feedback about how she and her classmates had experienced the on-line exercises and what simple changes in design could dampen negative effect has meanwhile joined the research team and is actually the first author of this paper.
ⁱⁱⁱIdiom Teacher is upgraded and expanded on a regular basis. At the time of writing this paper, we were still piloting different versions with our student population at the college, with a view to making the most satisfactory version available to the outside world. For an update on the project, please contact the authors.