Testing Times

Geoff Tranter, Germany

The UEFA Football Championships that took place in Poland and the Ukraine this summer have produced numerous new and innovative ideas that can be easily adapted and transferred to language testing. Equally, many of those matches have revealed a number of quality deficiencies that need to be eliminated before the next competition comes round in four years time.

In a similar vein, strengths and weaknesses are constantly being discovered in the field of both formal and informal language assessment systems within the constraints of the CEFR.

As a result, informed sources in Strasbourg have reported that for these reasons UEFA and the Council of Europe are planning to set up a joint committee to harmonize the quality procedures used in European football and European language testing.

The ideas under consideration relate to a higher degree of objectivity and standardisation in terms of playing the game (= teaching and learning) and scoring (= assessment). As the following agenda shows, it is intended to set up a joint UEFA/CEFR Manual Project to follow up and further develop these innovations.

The following proposals have been put forward to be discussed by the new committee.

A] Proposals for UEFA

1. Realising the problems that have occurred in connection with controversial decisions regarding ‘goal or no goal’, ‘offside or not offside’, ‘over the line, not over the line’, UEFA will consider implementing high-quality standards for match officials.
   
   To this end it is intended to investigate the idea of adopted a similar scheme to the Council of Europe and introduce a COMMON EUROPEAN FRAMEWORK OF REFEREES (CEFR). This Framework will cover a six-scale system of definitions from A1 (Referees for club games) all the way up to C2 (Referees for international games).
   
   A number of categories have been established for this Framework including:
   • Spoken Interaction (= sub-skill ‘goal-oriented cooperation’)
   • Listening Comprehension (= sub-skill ‘understanding insults’)
   • Interactive Strategies such as ‘turntaking’ = ‘initiating and closing the game’ – (This will also cover multi-media features such as whistle-blowing and red and yellow cards)
   • Compensation Strategies such as compensating for one wrong penalty decision by making the same wrong decision for the other team later in the game.
   • Mediation (= “making communication possible between players who are unable, for whatever reason, to communicate with each other directly” - cf. CEFR 2.1.3 adapted)
   • Socio-linguistic Appropriateness (= subskill: understanding non-verbal communication, such as head-butting, the semantics of the middle finger or alternatively two fingers, etc.)
2. UEFA will also consider establishing some form of reliability checks for their match officials adopting the proven CoE procedures of inter-rater and intra-rater reliability.
   
   In keeping with the principle of test reliability, there will also be checks to ensure that given the same type of competition/test, the same result will be achieved regardless of venue. That the UEFA have already reached a relatively high degree of reliability is demonstrated by the fact that
   • the last three international competitions - 2008 in Austria/Switzerland, 2010 in South Africa and 2012 in Poland/Ukraine - were all won by the same national team;
   • Italy has nearly always beaten Germany in the semi-finals of major competitions;
   • England almost never win in penalty shoot-outs.
3. All qualifying games for future competitions will be known as pre-testing sessions. The results will be statistically analysed along the lines of the Classical True Score (CTS) theory’s hypothesis that an actual score consists of two components: a true score that is due to an individual’s level of ability and an error score that is random and due to factors such as visually handicapped referees.
4. There have also been calls for a greater focussing on validity in UEFA Championship games in the sense that many supporters do not consider the methods used by some teams to listlessly move the ball around the pitch to be a valid representation of what is commonly understood by the concept of football. This is particularly controversial when it comes to the specific area of “content validity” when fans are in no way content with the performace of the teams on the pitch.
   
   Such assessments however tend to have a high degree of subjectivity, a good example of which was provided by the Champion’s League Final in Munich in May 2012 when the FC Bayern fans were far from content with the result and disputed the content validity of the seemingly random ball movements that Chelsea undertook on the pitch. However, here once again can be seen the goal-oriented discrepancy between content validity, content relevance and content coverage (=face validity) on the one hand and the action-based, real-world, goal- and result-oriented construct validity with which Chelsea achieved the score-sheet they had planned for.
5. In the interest of multi- and plurilinguism across Europe, all international players will have to have a good command of at least three European languages. For this, a joint UEFA/CoE committee is reported to be developing a new syllabus containing words and phrases that are most frequently used during and after football matches, i.e. four-letter words, insults and obscenities.
6. UEFA is said to be considering the CEFR test option of allowing teams to re-take a test as many times as possible until they win. This proposal, said to have come from the English Football Association, will probably be rejected by a large majority.

B] Proposals for CEFR-Based Language Testing

1. To ensure that candidates are not assessed unfairly, the use of videos will be permitted in test rooms to make sure the assessors know whether the candidates really have crossed the line in terms of grammatical accuracy or are offside in terms of repertoire.
2. In keeping with UEFA philosophy, all tests will be goal-oriented with an appropriate score-sheet. And in keeping with the student-centred and “bottom-up” approach of modern language teaching the candidate’s “own goals” will also be recognised.
   
   To ensure that this approach is implemented and adhered to, the CoE will establish a new committee, whose members will be known as the “goal-keepers”.
3. Language errors and mistakes will in future be known as fouls and candidates will be awarded a yellow card should their performance be borderline and a red card if the number of fouls warrants failure in the test. B1 candidates who fail to make the grade will be relegated to A2, those producing the highest performance levels will be promoted to the next highest competence level, e.g. B2>C1.
4. Borderline candidates in danger of failing the examination after normal time will have to accept a period of extra time in order to have another chance of passing. If after extra time there is still no clear result, a system of five shots at a difficult target question will ensue. If all five shots are successful (i.e. do not miss the goal) the candidate will be given the appropriate certification. However, this proposal is not expected to be implemented as it will probably be vetoed by the English.
5. Candidates who suffer an attack of nerves or worse during the test will in the interest of fairness be entitled to five minutes injury time. For such cases, all exam rooms will have to be equipped with a first-aid kit for resuscitation purposes. All such incidents will be video-recorded to ensure that no candidates “take a dive” when confronted a difficult task.
6. Oral tests will be extended to include a post-exam joint evaluation session to assess the candidate’s competence in the new “foul language, swear words, obscenities and insults” syllabus being especially designed for adult learners. This additional test phase will allow both examiners and candidates to express their opinions of each other after the test. This will be known as the EFL test (=English as a Foul Language). A number of publishers have allegedly already put in a bid to produce such EFL materials and colleges are queueing up with teacher-training courses in TEFL (Teaching English as a Foul Language).
   
   As was clearly shown by comments made after certain teams lost out in the European Championships, this will particularly accommodate the needs of, amongst others, the French.
7. Official test-based terminology and procedures could be changed as follows:
   • All tests will be known as games, and all test centres will be re-named stadiums.
   • All examiners, raters and assessors will be required to wear black shirts and white shorts.
   • All test examiners will be issued with a CEFR whistle to make sure the candidates know when they have to start the tasks and when the task has been completed.
   • In paired-format test situations, a coin will be spun to decide who sits on which side of the assessors and who should first “tackle” the initial task. Halfway though each test,
   candidates will ”change ends” and swap seats.
   • One change that will not be accepted is the principle of reserves on the bench. Candidates will not be allowed to use a stand-in as a reserve candidate during the examination.
8. In view of the increasing popularity of so-called “public viewing” of international games, it is intended to allow the video-screening of all language examinations throughout Europe. By this means it is hoped to motivate more and more people in all CoE countries to volunteer to take language tests. If accepted, test venues could be extended to include beer tests and beer gardens and cheerleaders may be hired to celebrate all correct answers.
9. In keeping with the behaviour of most international football players and to establish the degree of emotional intelligence, male test candidates will be allowed to
   • kiss, hug and jump on top of each other on passing the test
   • shed tears of disappointment on failure.
10. In line with the English system of inviting WaGs (wives and girl-friends) to attend competitions, a similar system will be introduced for CEFR-based language testing. In the interest of gender-mainstreaming, this will be extended to HuBs (husbands and boyfriends), although not more than one per candidate will be allowed at any one test.

Please check the Improving English through Humour course at Pilgrims website.