|Carsten Roever is a senior lecturer in Linguistics and Applied Linguistics, School of Languages & Linguistics, at the University of Melbourne, Australia. He earned his PhD in Second Language Acquisition from the University of Hawai'i at Manoa in 2001. He has worked in test validation at the head office of Educational Testing Service in Princeton, NJ, USA from 2001 to 2002. In 2005 he published Testing ESL pragmatics and in 2006 he co-authored Language testing: The social dimension with Tim McNamara. His research fields include second language acquisition, interlanguage pragmatics, language testing, and cross-cultural communication. This interview was conducted in person in March-April 2008.|
[ p. 21 ]A lot of procedures to have been suggested investigate the internal aspects of test validity, most likely because that is an area with which test validation researchers feel comfortable. Psychometricians like statistical procedures. You can't investigate the social dimension simply through statistics. Such an investigation requires other procedures, and that reason could account for the minimal amount of work conducted so far, because there are very few suggestions on how to conduct such research.
|"The establishment of procedures to validate the social dimension of language testing is a very complicated issue."|
[ p. 22 ]
|"Rasch analysis in and of itself does not buy you validity. It only shows you whether an item fits or doesn't fit, but just because the item fits, that does not mean that the item measures the construct."|
[ p. 23 ]What books are helpful depends on your general approach. I always like Hughes's Language Testing for Language Teachers because I think it is a good introduction. J. D. Brown's Testing for Language Programs is also quite good. There's another good one published in 1995, Language Test Construction and Evaluation by Alderson, Clapham, and Wall. Tim McNamara's Language Testing represents a more conceptual approach. Of course, it's quite abbreviated because it's the shortest introductory book, but that can be a good starting point for broader readings. I think for people to really get into language testing, I would probably teach a two-semester course – first introducing language testing, and then exploring advanced issues in language testing – because to digest so many things in one semester would be quite a challenge. If many graduate students are also language teachers, it would also be a good approach to combine testing and teaching practices. I think that could be, for example, a project component of a testing class. They could be asked to think about the context that they themselves work in and some of the tests that they either use or should use, and then look at those from a theoretical perspective or develop a new test given the material that has been covered in class. That's probably much more useful in the conceptually-oriented than in the statistically-oriented class.
[ p. 24 ]Lassche, G. (2007). Rasch & quality control: Controlling data, forgetting quality? In T. Newfields, I. Gledall, P. Wanner, & M. Kawate-Mierzejewska. (Eds.) Second Language Acquisition - Theory and Pedagogy: Proceedings of the 6th Annual JALT Pan-SIG Conference. May. 12 - 13, 2007. Sendai, Japan: Tohoku Bunka Gakuen University. (pp. 42 - 55) Retrieved March 28, 2008 from http://jalt.org/pansig/2007/HTML/Lassche.htm.