Examining Rater Effects in a WDCT Pragmatics Test

سال انتشار: 1393
نوع سند: مقاله ژورنالی
زبان: انگلیسی
مشاهده: 74

فایل این مقاله در 16 صفحه با فرمت PDF قابل دریافت می باشد

استخراج به نرم افزارهای پژوهشی:

لینک ثابت به این مقاله:

شناسه ملی سند علمی:

JR_IJLT-4-1_004

تاریخ نمایه سازی: 16 بهمن 1400

چکیده مقاله:

Written Discourse Completion Task (WDCT) has been used in pragmatics tests to measure EFL learners’ interlanguage pragmatic knowledge. In a WDCT, the students give their responses to situations designed to elicit certain pragmatic functions, so human raters are required to rate the students’ performance. When decisions are made based upon such ratings, it is essential that the assigned ratings are accurate and fair. As a result, efforts should be taken to minimize the impact of rater inaccuracy or bias on ratings. This paper reports a study of rater effects in a WDCT pragmatics test. Based on the Myford& Wolfe (۲۰۰۳; ۲۰۰۴) model and corresponding retrospective interviews, four types of rater effects were investigated and discussed quantitatively and qualitatively: leniency/severity, central tendency, halo effect, and differential leniency/severity. Results revealed significant differences in terms of rating severity, with a general tendency towards severity. Though the raters could effectively and consistently employ the rating scales in their ratings, some of them showed certain degrees of halo effect. Most raters were also found to exhibit certain bias across both traits and test takers. Possible reasons behind the rater effects were analyzed. Finally suggestions were raised for rating training.

کلیدواژه ها:

نویسندگان

Jianda Liu

Guangdong University of Foreign Studies, China,

Lijun Xie

Middle School of Xiamen, Fujian, China.