About this Journal
Information for Authors
Related Publications
Online Language Teaching: Crises and Creativities
Insights into Teaching and Learning Writing
Insights into Autonomy and Technology in Language Teaching
Insights into Flipped Classrooms
Insights into Task-Based Language Teaching
Proceedings of the XXIst International CALL Research Conference
Insights into Professional Development in Language Teaching
Smart CALL: Personalization, Contextualization, & Socialization


This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

The processes of rating L2 speaking performance using an analytic rating scale – A qualitative exploration
Thuy Thai, Susan Sheehan
– In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012).
Author(s) | |
---|---|
Paper type | Regular Articles |
Pages | 34-51 |
DOI | |
Year |
Abstract
In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater variability impacts on rating decisions, much still remains unclear about what raters actually do when they assess speaking performances. This paper aimed to extend our understanding of the rating process by analysing think-aloud protocols provided by 13 VSTEP speaking raters while they used an analytic rating scale to assess 15 recorded performances with varying proficiency levels. The data suggested that although all the raters seemed to experience similar stages in the rating process, differences in rating behaviour between novice raters and experienced raters exist. To finalise the scores, the raters used five different ways of deciding the scores.
Suggested citation
Thai, T., & Sheehan, S. (2022). The processes of rating L2 speaking performance using an analytic rating scale - A qualitative exploration. Language Education & Assessment, 5(1), 34–51. https://doi.org/10.29140/lea.v5n1.777