The Dark Side of ESL Assessment

Béatrice Boufoy-Bastick

Dr. Boufoy-Bastick has taught second languages in Europe, Asia, Oceania and the Caribbean. Now lectures in French at the University of the West Indies, Department of Modern Languages and Literatures in Jamaica. E-mail: boufoybastick@gmail.com

Menu

A blessing in disguise
Administering assessment
The Caribbean Case
From research to practice

Universities have regulations excluding students from writing end-of-course examinations if they have failed the coursework. This research questioned the rationale and empirical bases of such an exclusion rule as applied to a university English course with an enrolment of 849 students. As is commonly the case, in-class tests were used as coursework and the exclusion rule was intended to encourage attendance. As the coursework pass rate was 95%, it was reasoned that students failing the coursework would have failed the examination. However, through an administrative mishap, 55 students who had failed the coursework wrote the examination. The results were surprising challenging the validity of course work marks in predicting the examination results of a university ESL course

All universities have official examination regulations and these vary greatly from one university to another. However, no matter how varied they are, they are expected to be fair, equitable and in the service of the public good. This research examined whether a typical university policy of excluding students from the course examination when they had failed the coursework, was really equitable. There are problems with accommodating the assessment of coursework and examinations in universities (Bone, 1999), and not all students are happy with the ways in which coursework is used (Norton, & Brunas-Wagstaff, 2000). The pedagogic and psychometrics conflicts centre on the common practice of using formative coursework for summative decisions. This arises because coursework is a widely accepted learning tool that is also used for assessment (Banister, 2004).

A Caribbean university had a student exclusion policy that excluded students from taking their final examination on the course 'Introduction to the use of English' (ENG115) unless they had passed the coursework. The coursework comprised three in-class tests spread throughout the semester and the examination was a traditional 3-hour paper and pencil test. The coursework was weighted 50%, the examination was weighted 50%, and the pass mark was 45% of the coursework and 45% the examination. Hence to pass, a student had to attain at least 22.5% in both the coursework and in the examination.

We now examine the rationale for the exclusion policy, for the purpose of deciding if it was equitable. The coursework and the examination had a common in-class test format and shared similar content. Hence, the purpose of the exclusion rule was not to ensure that students have a passing competence in two different content areas or competence in responding to different forms of assessment. The initial purpose of such an exclusion rule is to motivate students to work continuously throughout the semester (Medrich, Brown, & Henke, 1992). Only 55 (6.5%) of students failed the coursework. Initially there were 849 first year university students enrolled in this foundation course. The numbers are expected to rise to 1553 and the exclusion rule will be 'convenient' in saving on the marking of 100 examination scripts. Ostensibly, the purpose of the assessments is to ensure that students have a satisfactory level of competence at the end of the course. Clearly, the rationale of 'motivation' or 'cost-saving' are irrelevant to this purpose. The most direct demonstration of end-of-course competence would be for all students to write the end-of-course examination. However, the rule is maintained by the supposition that students who do not pass the coursework, which has an expected pass rate of approximately 95%, are almost certain to fail the examination, and hence, there is little chance of injustice in excluding them. This supposition is indirectly supported by the fact that students who achieve high coursework marks also seem to achieve high examination and those who only just pass the coursework seem to only just pass the examination.

In 2004 an administrative mishap resulted in 55 students, who had failed the coursework, being allowed to sit the examination. This gave the occasion to empirically test the supposition that such students were almost certain to fail the examination. Of the 849 students who wrote the examination 794 had passed the coursework and 55 had failed the coursework. Predictive correlations (Pearson's product moment correlations) of coursework with exam mark were calculated for (i) those who passed the coursework and for (ii) those who failed the coursework. The correlations were compared to find out whether the coursework was an equal predictor for both groups.

Results showed coursework was a medium strength predictor of the examination results for those who passed (r=0.469, p<0.001). Hence, for these students 22% (medium effect size) of the variation in the exam marks is predicted by the coursework result. However, for the 55 students who failed the coursework, their coursework mark did not predict their exam mark (r=0.074, p=0.594>0.05), the effect size being less than 1% (r2=0.55%). Further, 28 students who failed the coursework passed the final examination with a maximum mark of 64%.

The research shows that the suppositions maintaining the exclusion rule are empirical. In addition, the suppositions are also theoretically untenable because they assume the sole reason a student fails the coursework is because he or she is extremely lacking in competence. However, competent students may have failed by being absent on the day of one or more in-course tests. It is concluded that the exclusions rule as it stands should be modified or rescinded as the research has clearly demonstrated that it is not equitable.

References

Banister, P. (2004). Assessment as a tool for fostering key skills. Psychology Learning and Teaching, 3(2), 109-113.
Bone, A. (1999). Ensuring Successful Assessment. Coventry, England: National Centre for Legal Education
Medrich, E.A., Brown, C.L. & Henke, R.R. (1992, January). Overview and Inventory of State Requirements for School Coursework and Attendance. Research and Development Report (NCES 92-663). U.S. Government Printing Office, Washington, DC
Norton, L. & Brunas-Wagstaff, J. (2000, June). Students' perceptions of the fairness of assessment. Paper presented at the first annual conference of the Institute for Learning and Teaching in higher education (ILT), College of Ripon and York St John, York.

Please check the Testing course at Pilgrims website.