Reliability of writing scores has always been a
longstanding concern in second-language assessment.
Due to the myriad sources of variability
contributing to writing scores, such as tasks,
raters, scoring rubrics, genres, topics, and traits
to be evaluated, writing exams have relatively low
score reliability. Classical methods of estimating
reliability fall short of accounting for different
sources of error since only one undifferentiated
error term is provided in this context.
Generalizability theory is perceived as a better
alternative to these classical methods given its
ability to provide information about the different
sources of error. In addition, it gives insights into
how these variables interact in an assessment
context. More importantly, generalizability theory
can propose possible assessment scenarios to achieve
maximum measurement precision. This book focuses on
two of these facets, tasks and raters, in order to
investigate their relative contribution to writing
scores using generalizability analysis. The book
adopts an interdisciplinary perspective where both
applied linguistics and psychometrics research is
brought together.
longstanding concern in second-language assessment.
Due to the myriad sources of variability
contributing to writing scores, such as tasks,
raters, scoring rubrics, genres, topics, and traits
to be evaluated, writing exams have relatively low
score reliability. Classical methods of estimating
reliability fall short of accounting for different
sources of error since only one undifferentiated
error term is provided in this context.
Generalizability theory is perceived as a better
alternative to these classical methods given its
ability to provide information about the different
sources of error. In addition, it gives insights into
how these variables interact in an assessment
context. More importantly, generalizability theory
can propose possible assessment scenarios to achieve
maximum measurement precision. This book focuses on
two of these facets, tasks and raters, in order to
investigate their relative contribution to writing
scores using generalizability analysis. The book
adopts an interdisciplinary perspective where both
applied linguistics and psychometrics research is
brought together.