A New Direction in Feedback
Russell Stannard, UK
Russell Stannard is a principal lecturer at the University of Westminster where he teaches on the MA in TESOL and ICT/Multimedia courses. He recently won the Times Higher award for “Outstanding Initiative in ICT “for his website www.teachertrainingvideos.com. The website offers free training material for teachers interested in integrating technology into their teaching. His work on feedback has created coverage in the national press and is being researched in many countries around the world. E-mail: russellstannard@btinternet.com, www.russellstannard.com
Menu
Abstract
Introduction
Why do we give feedback?
A lot of feedback is ineffective
What method?
Direct or indirect?
Content or form?
Students want conferencing
Video feedback
Summative versus informative
Conclusions
Appendix 1
Appendix 2
References
This paper looks at an innovative way of providing feedback for students on their written assignments. Screen capture software allows you to record all screen activity on your computer screen as if you had a video camera pointed at it. If a microphone is attached to your computer it can also record your voice. The result is that students work can be opened onto a computer screen, the screen capture software can be turned on and the teacher can begin to correct the word on the screen, highlighting mistakes, underlining problems and writing comments. The resulting video can then be sent back to the students , resulting in a ‘live’ recording of the tutor correcting the students work. The student can play back the video and listen and watch as the tutor goes through their written assignments. The idea was originally reported on in the THES in December 2006 (Stannard 2006)
Screen capture software is nothing new. It is usually used for teaching computer software. Since the screen recorder software simply records the screen of your computer it can be used to demo things very simply. Imagine for example you wanted to know how to reduce the size of an imagine using Adobe Photoshop. Instead of reading through notes or the help page, your tutor could simply make a video where he/she records the screen of their computer while they actually use Photoshop and then send you the resulting video. The attraction is obvious. You can both hear and see what your tutor is doing resulting in a form of tuition which appeals to a variety of learning styles and is both visual and oral. What is more you can play, pause and repeat the video as often as you like. One site that offers many of these types of training videos for ELT teachers is www.teachertrainingvideos.com
However using the technology for feedback is a very recent development. In this study a small group of students was used on a “feasibility test” to see how technically the idea might work as well as gauging the student’s reaction through questionnaires and interviews. Feedback from students has been very positive but probably more interesting is some of the insights that have been revealed through the research and possible directions in which the idea could be developed further. A second study is also mentioned.
A simple example (not a real example taken from a student) of how video feedback might work can be seen here: www.russellstannard.com/king/king.html
At the bottom of any reasoning is the fundamental belief that by making the students aware of the mistakes they make and by getting them to act on those mistakes in some way, then it is believed that the students will assimilate the mistakes and eventually not make them in future. In other words correction is closely linked to language acquisition and particularly to the idea of accurately acquiring the language. This statement itself is making sweeping statements about everything from the quality of the feedback to the ability for students to acquire language through the feedback we offer them. However, it seems to be the heart of why we do feedback. Indeed you could argue that institutions and students demand feedback and that this is the reason why we provide it, but even these two groups fundamentally believe that correction in someway leads to language acquisition and demand it for that reason. Writers such as Ashwell (2001) have pointed out that with the increased importance of the writing process there may be other reasons for including feedback and these include improving the “communicative effectiveness” of a given written piece. In other words we correct students and guide students so that the written piece they eventually produce communicates their ideas as effectively as possible. This is linked closer to feedback that looks at the content of a written piece rather than the grammatical forms within it. He also points out that a 3rd reason for correcting students work is simply because “formal accuracy of a written piece matters” and that people generally are a lot less accepting of mistakes in written pieces.
This is a sweeping statement to make. Though if you read the work of Truscott (1996) then it is anything but sweeping. “Rather the teacher should conclude that correction is simply not effective” or 9 years later “The goal of this paper is thus to show that research evidence points strongly to the ineffectiveness of correction” (Truscott 2005). What is clear even to those who believe in feedback is that it is incredibly inefficient. It takes a long time to mark a piece of work and yet the research would suggest that students largely ignore the information we give back to them.
Cohen (Cohen 1987) pointed out that students only took a mental note of about half the mistakes the teacher corrected and that they only made use of about 10% of the corrections in their re-drafts. Even if students do use our corrections it is not even clear that they follow them. Research has shown that corrections are often contradictory, confusing and vague Zamel (1985), Crawford (1992), Goldstein & Kohls (2002). Crawford (Crawford 1992) even found that students used teacher’s feedback without actually understanding what they were correcting
Teachers misread students work and evoked abstract rules. This is perhaps not surprising since in most cases teachers have to make “plausible” reconstructions of what students were trying to say as they rarely have to the chance to sit down with the student and go through the work in person. Some research has shown that students often ignored feedback because they simply couldn’t understand the corrections (Bartholomae,1980; Hyland, 2003).
Cohen(Cohen 1987) found that students felt that teachers gave insufficient grammatical explanations of mistakes. It seems that a few notes in a margin or at the end of a paper are simply not enough for students to be clear about the mistakes they have made. Sommers ( Sommers 1982) , Leki (Leki 1990) came to similar conclusions. Some researcher have even concluded that correction focuses too much on the negative and can be dysfunctional in the students pursuit of language competence (Cohen and Cavalcanti 1990).
In an attempt to deal with the varied problems of feedback researchers have turned to systems or methods that would make the marking scheme more consistent and comprehensible. A common one is to provide code for the corrections ( ie G for grammar, V for vocabulary etc) while leaving it up to the student to actually correct the errors.Rob ( Rob et al., 1986) compared four different forms of feedback. Each control group was given a particular type of feedback. One group was given explicit error correction, one group was given codes to show where and what type of mistake was made, one group was simply given a tally of mistakes in the margin and final group just had mistakes underlined. They found hardly any differences in the number of corrections completed in re-drafts. However, since the researchers did not have a ‘no feedback’ control group, we have no way of knowing whether some feedback is better than no feedback. Ferris et al. (Ferris and Robins 2001) found similar results. They had 3 control groups. They found that explicit marking and offering codes ( 5 different types depending on the error) had a similar impact on the student’s re-drafts. Both had fewer mistakes than the third group which was a no feedback group. This study would seem to suggest at least that some feedback is better than none. Other studies support this view by Ashwell (2000), and Fatham and Whalley (1990)
There have been several longer studies undertaken. Among these Chandler( Chandler 2003) undertook a study where one group were set a weekly essay and given back the corrected work before they wrote the next one, while the other group was given no feedback on their essays during the experiment. It was found that both groups improved in terms of fluency but the feedback group improved more in terms of accuracy. Unfortunately other longitudinal studies didn not come to similar results leaving the picture still unclear (Sheppard,1992; Goring-Kepner,1990; Semke,1984)
Even if we do conclude that some feedback is better than none we still need to concern ourselves with what and how much. Do we focus on surface areas or turn our attention to the writing process (form v content)? Do we give explicit correction or just direct students to their errors? Do we correct less but provide more information in the hope that perhaps students take greater notice of the mistakes we focus on?
Direct feedback means explicitly correcting the mistake so the students do not need to do much more than copy the correction in the new draft. Most EFL/ESL practitioners opt for indirect forms of correction, where the mistake might be highlighted but not explicitly corrected. The hope is that since the students have to think about the mistake and correct the work themselves, it is more likely to be ‘memorised’ hopefully leading to acquisition. Lalande (Lalande 1982) demonstrated that direct correction ( actually correcting the mistake) had less impact than indirect correction (ie. pointing out the mistake and giving a code but not correcting it). Ferris et al (Ferris et al 2001) came to an interesting set of conclusions. They in contrast found that direct error correction led to fewer mistakes in re-drafts, however over time they found that those students who received indirect feedback reduced their error frequency more. In other words, forcing the students to process their mistakes led to greater language acquisition in the long term. Rob et al (Rob et al 1986) did not find that the direct feedback was any more effective than the three other indirect methods used.
Lee (1997) found that indicating a mistake in the margin or giving no feedback at all led to fewer corrections in re-drafts than students whose errors were underlined. Perhaps then it is at least important to direct the student to the mistake even if we do not explicitly correct the work. Chandler ( 2003) also found that underlining mistakes or direct corrections led to the most accurate revisions.
How we correct is one problem, what we correct is another. The interest in the writing process has led to shift away from correction on form to a focus on the content and organisation of writing. Fatham and Whalley (1990) had three control groups. One receiving feedback on form, one receiving feedback on content and one receiving both feedback on form and feedback on content. All 3 groups had similar gains in accuracy. Ashwell (2000) came to similar conclusions. Goring-Kepner (1990) had two control groups one receiving feedback on form and one on content. The results showed little differences over time in their long term accuracy. Fazio (2001) had three groups, one receiving feedback on form, one receiving feedback on content and one receiving both. Results suggested there were little difference between the 3 control groups. Semke (1984) and Sheppard (1992) came to similar conclusions.
It is not all bad news. Student feedback that was accompanied by sessions of student – teacher conferencing has been found to be more effective. Bitchener Young and Cameron (2005) concluded that there was a “significant effect on the combination of written feedback and conferencing’. What is clear is that students like conferencing ie student to teacher meetings (Fregeau,1999). Fregeau (1999) found that it helped both students and teachers to trace their mistakes and made the written feedback that teachers offer students more understandable.
One thing that is rarely mentioned in the feedback debate is the fact that nearly all feedback is written. As Sugita (Sugita 2006) writes “For many teachers, however, handwritten commentary on students drafts is the fundamental method of response”.
Yet work in the last 15 years has shifted towards a consideration of learning styles and a need to accommodate a variety of different learners. There has been a greater interest in things like peer-reviewing, taping feedback and a whole variety of computerised formats, but the fact remains that most feedback is written.
Outside of ELT, work by Richard Mayer (2001) whose primary focus is on the power of multimedia to help learning, is very revealing. His research has shown that a combination of animation and commentary is the most memorable form of input for students. In tests students recall more information from this form of input than others. If input is more memorable when it is a combination of commentary and visuals then presumably feedback that combines animation and commentary would be better than the current text based approach most teachers still use.
Research has also shown that teachers often don’t understand the mistakes that student’s make. Corder (1981) speculated that teachers could correctly guess the intended meaning of learners in a large number of errors. Yet recent research has suggested that this is not true. Hamid (2007) found that a much higher number of errors were misunderstood by teachers. If the teachers had recorded the marking sessions then at least the students would be aware of when a teacher had misunderstood their intended meaning.
The research covered in this study, draws together many of the drivers for the introduction of video feedback.
- Feedback is often misunderstood by students. It is often unclear what the tutor has written or corrected.
- There is a lack of interest in the feedback process and a real need to find new ways of providing feedback that will bring it back into the centre of learning, where it should be.
- Students want conferencing.
- Teachers don’t always interpret the mistakes their students have made and this leads to students being confused about the corrections they make.
- Feedback is text based. It is biased to one type of learner ie the learner that likes reading as their primary focus of input.
- The amount of information that can be conveyed in the written form is very limited. Teachers tend to give short, inadequate feedback in many cases.
The first study was done with a group of nine Chinese students on an EAP course. An example feedback video was shown to the students in class. It was explained that they would not be receiving conventional feedback but rather they would be sent a video where they would be able to see a recorded session of the tutor marking and commenting on their work. The video could be played, paused and rewinded and the students should to use the video to re-write their essays.
Students sent their work by email. The tutor opened their work onto the screen, turned on the screen recorder software and began to correct the students work. The videos were then saved, compress and sent back to the students. Students watched the videos and re-drafted their essays based on the feedback in the videos.
Observations from the first test
- Students returned their re-drafts incredibly fast. Within about 2 days I started receiving the essays.
- It immediately became clear that a certain amount of preparation needed to be done before a tutor starts to create the feedback. It is vital to quickly read the essay through on the screen and mark with a screen highlighter (or write numbers on the paper) the points that the tutor wants to develop. If not there are periods of silence while the tutor searches for errors that he/she wants to focus on.
- Another danger is that if the tutor does not read through the essay at least once that they will tend to read aloud as they go through the essay. This can confuse the students. Preparation is vital.
- The tutor was surprised to see how much information could be provided. You can speak around 200 words in a minute so a two minute video could be as much as 400 words. This is equivalent to about a sheet of A4 feedback in written form (depending on the tutor’s handwriting).
- Related to this point, there is a tendency to perhaps provide too much feedback. A consideration of some sort of approach or methodology is vital for this idea to succeed. In this particular case the tutor did not actually correct any of the errors but rather commented on them and provided information as to why there was a mistake and what the student needed to consider but it was up the student to actually correct the work.
- The visual element needs a certain amount of training. Remember anything that is done on the screen is recorded in the video since the screen capture software simply records the screen. So if the tutor used the highlighter tool, colours in parts of the screen, selects, underlines etc, this all comes out in the video and offers a visually rich form of correction as well as making use of the oral/aural channel. In fact within the software package used for this experiment (Camtasia) there is actually a whole array of drawing tools that can be used to facilitate the correction
- Correction was quick but to email the videos to the students it is necessary to compress the videos and this is time consuming. In this first experiment the tutor corrected the work, compressed the video and then sent it to the student (doing this a total of nine times). It was later discovered that it is possible to batch compress the videos. So all the videos can be made first and then compressed at the same time. This saves a considerable amount of time.
In the next lesson an informal discussion took place with the students and each student was given a questionnaire to complete. The questionnaire consisted of the following questions. Students were given the following questionnaire and then an informal group discussion then took place
- What advantages to video feedback do you think there are?
- What disadvantage to video feedback do you think there are?
- Can you rate how good you think video feedback is?
- Not useful at all
- Ok- but no better than using traditional approaches
- A very good way of providing feedback
- An excellent way of providing feedback
- Did you find the video feedback easy to follow? Would you say it is easier to follow the video feedback than follow the corrections when they are written on your essays?
- If you had to choose. Would you prefer video feedback or traditional forms of feedback?
- Can you explain how you used the feedback? What did you do with the videos?
- Are there any other comments you would like to make?
A summary of the student’s reactions included the following points:
- The feedback was very positive. All the students thought the method was excellent or very good. However this could easily be due to the novelty factor and the control group is too small to make conclusions from. ( later on-going studies have been very positive too but not quite as much as this first study)
- The main reasons for liking the videos were that:
- The students liked the fact that the information was oral and visual (one
student commented that it was like having the teacher with them),
- They felt more information was being provided (which is true as some students
were probably getting in access of 400 words of feedback),
- They could play and replay the videos as much as they liked.
- Students commented that it was much easier to follow the corrections. In fact
seven of the nine students made this comment. Interestingly however in the
informal discussions some students commented that too much information was
provided. Again pointing to the need to develop some sort of methodology or
coherent approach.
- Most of the students had watched the videos more than once before re-drafting
their essays.
- In the informal discussions students asked that when the video was sent to the
students that the tutor also send back the actual file that had been corrected on
the screen too, with all the markings. In this first experiment only the videos had
been forwarded to the students.
This was very much a “feasibility test” in the possibilities of using screen recorder feedback and there are many flaws in this initial piece of research (including the size of the study itself). In a second experiment the tutor decided to provide video feedback at an earlier stage in the writing process. It was felt that though the idea was good but the focus was on the product and not on the process. Exactly the same approach was taken, with students sending in their essays plans rather than their completed drafts and the tutor “correcting” or “guiding” the students and “commenting” on their essay plans.
The experiment had to be abandoned as so many of the plans were so weak that there was very little to comment on. For example, students wrote things like “Introduce idea” as their plan for the introduction. However two students did write fairly detailed plans and several points emerged from the abandoned experiment (the student videos were never actually sent to the students) which have actually had a major influence on the direction of current research taking place.
When tutors comment on ideas rather than surface error mistakes they are much more likely to make greater use of the video feedback. Simple surface error mistakes like spellings or grammar, in most cases, require one or two sentences ie “You have made a spelling mistake here. Think carefully about what prefix to use with possible to negate it” However, when you are commenting on ideas or structure the tutor is quite likely to want to develop and elaborate on ideas. This is much easier to do using video feedback and of course much quicker. Could it be that in many ways the video feedback is least suited to correcting surface errors in language work and much more suited to correcting well formed essay plans or indeed other areas of the curriculum where “ideas” are always the key focus? Is it possible that Russell Stannard’s original idea (2007) is in fact the least appropriate way of actually using video feedback? This is the current area of this researcher’s focus.
This idea caused so much interest that the THES published an article about video feedback before any real research had even been collated. This meant that the idea was in the public domain at a very stage allowing several organisations including Coventry University and Edinburgh University to begin their own research. It is interesting to see how the directions of their work have all been different; highlighting the importance of the idea and just how many factors will need to be considered if video feedback is really to emerge as a viable solution.
Edinburgh University have been interested in doing experiments on a large scale and looking at ways of making the method as rapid and effective as possible. They have also focused on student’s reactions which have fully confirmed student’s reactions in this study. However, Edinburgh focus was on biology students and there work is of yet unpublished. Coventry University undertook research with German translation students and again came to similar conclusions. There work is just about to be published and their focus is now moving towards how tutors can be trained to provide video feedback and the barriers to entry in terms of the technology.
This researcher’s focus is more on the approach itself. Questions under consideration or for future study include the following
- Is video feedback suited to certain types of learners? Are there students with certain learning styles that don’t like the feedback?
- Is the feedback more memorable? Can we prove that students memorise more of the mistakes than they do when they are provided in written form?
- Do students actually make use of more of the corrections on re-drafts is they are provided by videos than in written form? Though this is no proof of long term language acquisition it would indicate greater interest or motivation in the process. It may indicate that the students are following more of the mistakes.
- How can be guarantee that we maximise the use of both channels? It is vital to make the correction as visual as well as oral as possible. There is a danger that the tutor speaks to much and makes little use of the powerful visual tools that are also available.
There are many types of screen recorder software on the market. Probably the best known product is Camtasia. This is a fairly simple product to use and there are training videos that show you how to use the product
www.techsmith.com
Another product is Screen Recorder by Matchware
www.matchware.com/en/products/screencorder/default.htm
Both of this products can be downloaded and used for 30 days free
Individual feedback
A fictitious example of correcting a student’s piece of work which demonstrates the idea can be viewed at:
www.russellstannard.com/king/king.html
Classroom feedback
The other two are real examples of ‘classroom” feedback’ ( as outlined as my second idea). This is general feedback on their performance in presentations and also on writing an essay.
www.russellstannard.com/present/present.html
www.teachertrainingvideos.com/iwb/iwb.html
Ashwell, T. (2000), Patterns of teacher response to student writing in a multiple-draft composition classroom: Is content feedback followed by form feedback the best method?, Journal of Second Language Writing 9 (2000), pp. 227–257.
Barholomae, D. (1980) The study of error College Composition and Communication 31:253-69
Bitchener, J. Young, S. Cameron, D The effects of different types of corrective feedback on ESL students writing. Journal of Second Language Writing. October 2005 pgs 191-205
Chandler J (2000), The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing, Journal of Second language writing
Cohen, A. (1987) Student processing of feedback on their compositions. In A. Wenden and J Rubins Learner strategies in language learning (pp 55-69)
Cohen, A & Calvcanti, M. (1990) Feedback on compositions: Teacher and student verbal reports. In: B. Kroll, Editor, Second language writing: Research insights for the classroom, Cambridge University Press, Cambridge (1990), pp. 155–177
Crawford, J. (1992). Student response to feedback strategies in an English for academic purposed program, ARAL, 45-62
Fathan, A.k and Walley, E (1990) Teacher response to student writng:Focus on form versus conent. In B Kroll . Second Language Writing (pp 57-68). Cambridge UK: CUP
Fazio,L. (2001)The effect of corrections and commentaries on the journal writing accuracy of minority- and majority-language students, Journal of Second Language Writing 10 (2001), pp.
Ferris, D and Roberts, B. (2001) Error feedback in l2 writing classes: How explicit does it need to be? Journal of Second Language Writing 10(3) 161-184
Ferris, D. (1997) “The influence of teacher commentary on students’ revision. TESOL Quarterly 31 pp 315-39
Fregeau ,L.A. (1999). Preparing ESL students for college writing: Two case studies. The Internet TESL Journal [on-line]5 (10).
Goldstrein, L and Kohls, S (2002) Writing, commenting and revising: The relationship between teacher feedback and student revision on-line. Paper repsented at the American Association of Applied Linguistics Conference Salt Lake City, UT
Goring-Kepner, C.(1990) , An experiment in the relationship of types of written feedback to the development of second-language writing skills, The Modern Language Journal 75 (1991), pp. 305
Hyland, D(2003) Focusing on form: Student engagement with teacher feedback, System 21 (2003), pp. 217–230
Lalande, F.J (1982) e, Reducing composition errors: An experiment, The Modern Language Journal 66 (1982), pp. 140–149
Lee, I (1997), ESL learners’ performance in error correction in writing: Some implications for college-level teaching. System 25 (1997), pp. 465–477
Lizotte, R. (2001, Winter). Quantifying progress in an ESL writing class. MATSOL Currents, 27(1), 7, 17. Additional data and information was supplied in a personal communication.
Leki. I. 1990 Coaching from the margins: Issues in written response. In B Kroll . Second Language Writing (pp 57-68). Cambridge UK: CUP.
Mayer R 2001 Multimedia Learning. CUP
Rob, T, Ross S, Shortreed,I. (1986) Salience of feedback on error and its effect on EFL writing quality. TESOL Quartely , 20. 83-95
Semke, H. (1994) Effects of the red pen. Foreign Language Annuals , 17 , 195-202
Sheppard, K (1992) Two feedback types: Do they make a difference? RELC Journal, 23 103-110
Sommers N.(1982) Responding to Student writing College Compositoin and Communication 33/2 148-56
Stannnard (2006) THES December 2006
www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=207117
Stannard (2007) HEA English Subject Centre Commissioned Case Studies.
www.english.heacademy.ac.uk/explore/publications/casestudies/technology/camtasia.php
Sugita, Y (2006) The impact of teacher’s comments types on student’s revision. ELT Journal 60/1 January 2001
Truscott, T. (1996) The case against grammar correction in L2 Writing classes. Language Learning , 46:2 327-369
Truscott, T. (2007) The effect of error correction on learner’s abilility to write accuractely. The Journal of Second Language Writing 16 (2007) 255-272.
Zamel, V (1985) Responding to student writing. TESOL Quarterly, 19,pg 79-101
Please check the Humanising Testing course at Pilgrims website.
Please check the Expert Teacher course at Pilgrims website.
|
