Extraneous variables and the washback effect
James Dean Brown
University of Hawai'i at Manoa
[ p. 10 ]For the most part, extraneous variables are a threat to the internal reliability and validity of a research project. Essentially, such extraneous variables, if not controlled, or otherwise accounted for in a study, are all potential intervening variables (i.e., unanticipated variables that could explain the outcomes of a study as well as the conclusions drawn by the authors). As I put it in my 1988 book, "In statistical studies, there are a number of problems that can arise – both within a study and from outside of it that may create major flaws in its validity, i.e., the degree to which a study and its results correctly lead to, or support, exactly what is claimed. The problems themselves result from extraneous variables that are relevant to a study but are not noticed or controlled."
[ p. 11 ]From the point of view of testing, thinking about washback can help us to think about test validity. Washback becomes negative washback when there is a mismatch between the construct definition and the test, or between the content (e.g., the material/abilities being taught) and the test. Given that the definition of validity is the degree to which a test is measuring what it claims to measure, any such mismatch between the construct or content that a test is designed to measure and the test, would be a threat to the test's validity.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115-129.
Brown, J. D. (1988). Understanding research in second language learning: A teacher's guide to statistics and research design. London: Cambridge University Press.
Brown, J. D. (1999). The roles and responsibilities of assessment in foreign language education. JLTA Journal, 2, 1-21.
Brown, J. D. (2000). University Entrance Examinations: Strategies for creating positive washback on English language teaching in Japan. Shiken: JALT Testing & Evaluation SIG Newsletter, 3(2), 4-8. Retrieved March 1, 2001 from the World Wide Web: http://jalt.org/test/bro_5.htm
[ p. 12 ]Cheng, L. & Watanabe, Y. (2004). Washback in Language Testing: Research Contexts and Methods. Mahwah, NJ: Lawrence Erlbaum Associates.
Gates, S. (1995). Exploiting washback from standardized tests. In J. D. Brown & S. O. Yamashita (Eds.), Language Testing in Japan (pp. 101-106). Tokyo: Japanese Association for Language Teaching.
Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test Validity (pp. 33-45). Hillsdale, NJ: Lawrence Erlbaum Associates.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed.) (pp. 13-103). New York: Macmillan.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241-256.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language Testing, 13, 298-317.
|Where to Submit Questions:|
|Please submit questions for this column to the following address:|
Department of Second Language Studies
University of Hawai'i at Manoa
1890 East-West Road
Honolulu, HI 96822 USA
[ p. 13 ]