Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 2 No. 2 Oct. 1998 (p. 11 - 14) [ISSN 1881-5537]
PDF PDF Version


Can we validly translate questionnaire items
from English to Japanese?

Dale Griffee

The purpose of this paper is to argue that items from a test instrument originally written in English, if translated into Japanese, cannot result in a valid test in Japanese simply by being translated. Researchers such as Shimizu (1995) as well as Widdows and Voller (1991) have had questionnaire items written originally in English translated into Japanese and then used data resulting from the translated items in order to make both statistical and qualitative inferences. None of the researchers gave reasons for the translation, although we can suppose that they wanted to ensure item comprehension by the Japanese participants in their studies. Their translated instruments were aimed at making inferences about students' needs or attitudes. Widdows and Voller wanted to know if their Japanese students' needs were being met by traditional teaching methods. Shimizu wanted to know if her students had different attitudes toward Japanese teachers of English as opposed to foreign teachers of English. Such research raises the issue of what it means for an instrument to have been validated. Put simply, to validate an instrument such as a test or a questionnaire, three things have to be done (for a more complete description see Griffee, 1997a). First, the construct has to be defined, which is to say "before developing a test of any construct, one should clearly and explicitly express what one wants to test" (Most and Zeidner, 1995, p. 482). This is sometimes known as defining your terms, but it probably should be known as defining your construct. Second, you have to give some evidence that the instrument has consistency or reliability (Griffee, 1996). Third, the test maker has to provide some evidence that the test is measuring what the test maker claims it to be measuring (Brown, 1988, p. 101).
Three questions are raised in this paper. The first has to do with the validation of the original test instrument items. The second has to do with the possibility of the equivalence of the two instruments. The third question has to do with the validation of the L2 document. I will refer to the original English instrument as the E-doc and the resulting Japanese translation instrument as the J-doc. Terms such as "test", "instrument", and "questionnaire" are also used interchangeably in this article.
Three questions that every researcher working between languages should ask are:

[ p. 11 ]

1. Was the E-doc validated?
2. Is it possible for test items to be validly translated?
3. Was the J-doc validated?
Now let us examine each of these three questions.

Was the E-doc validated?

In many cross-linguistic research papers, constructs are not defined, reliability information is not provided, and no evidence is provided to show whether the instrument is measuring what it purports to. Though the papers by Shimizu (1995) and Widdows and Voller (1991) fail in each of these respects, this is not unusual in our field in Japan. For example, as far as I know, no article with questionnaire data in The Language Teacher has ever had adequate cross-linguistic validation information. Moreover, most articles in The JALT Journal with questionnaire data also lack this feature. If the examples supplied the original language were not validated carefully, the translation upon which the resulting documents was based cannot be considered valid.

Is it possible for test items to be validly translated?

Suppose that the E-doc were validated. Then what? Widdows and Voller themselves suggest the difficulty, if not impossibility, of a valid translation. They state, "It is interesting to note that certain concepts quite fundamental to current EFL methodology proved impossible to render into straightforward Japanese" (1991, p. 128). They add that another difficulty arose from Japanese cultural understanding dealing with an item asking about learning styles. The item wanted to know if the student learned better when the teacher took an interest in them as a person. The problem was with the word "interest" because "it was impossible to eradicate entirely the connotation of sexual interest in the Japanese version" (Widdows and Voller, 1991, p. 128).
Sakamoto (1996) investigated Hyland's (1994) use of translated questionnaire items originally from Reid's (1987) learning style preferences questionnaire. Sakamoto's participants were similar to the students used by Shimizu and Widdows and Vollers. The students were two groups of Japanese women aged 20 to 22 years of age at Bunka Women's University in Tokyo. Hyland had Reid's items translated and Sakamoto used these translated items, but she retranslated four of the items she thought were misleading. Sakamoto administered both the English items and the translated items to her students allowing time between administrations to reduce the possibility of remembering answers. She then compared the answers on the two questionnaires to see if the students answered the Japanese version differently than the English version and found that "about half of the 65 participants answered the same questionnaire statements differently in English and Japanese" (Sakamoto, 1996, p. 83). Sakamoto concludes:

[ p. 12 ]

Clearly, then, there were differences between the questionnaire results in English and Japanese. The high discrepancy in this study warms us that the researcher should not simply consider translation as the answer to help the respondent understand the questionnaire better. (p. 87).
Was the J-doc validated?

Now comes the real kick in the head. I want to argue that the translated J-doc becomes an instrument in itself and has to be validated as a separate instrument. The fact that the E-doc has been validated does not carry over to the J-doc. We have to have evidence that the J-doc is measuring what it purports to be measuring. The mere fact that the students speak the Japanese language as their L1 and that the J-doc has been translated into the Japanese language is not enough to provide validation. If you are a native English speaker, you can ask yourself two questions. The first question is, "Have you ever read an English sentence which was translated from Japanese (or another language) and been left wondering what it was trying to say?" The second question is, "Have you ever read an English sentence written originally in English by a writer you presumed to be an English L1 writer and still been left wondering what it meant?" Most of us can answer "yes" to both questions. That this is the case is exactly why validation of information is required in the first place. Even if the J-doc had been written originally in Japanese by a Japanese L1 writer, it would require validation.
I would like to conclude with two points. First, I am not trying to find fault with the two instruments cited. When Widdows and Voller published their article I did not have the foggiest idea what reliability (much less validity) was about. Furthermore, I was quite happy to write and have published my own unvalidated questionnaire (Griffee, 1997b). Nevertheless, we are now more informed as to what is involved in the issues of reliability and validity. Second, we now have a Testing & Evaluation SIG and a newsletter. I think that one of the uses we can put our newsletter to is to discuss issues such as the one I have raised. In that sense I invite feedback, comments, and discussion on this article either for or against which can be sent to the editor of this publication.

[ p. 13 ]


Brown, J. D. (1988). Understanding research in second language learning. Cambridge: Cambridge University Press.

Griffee, D. T. (1996). Reliability and a learner style questionnaire. In G. van Troyer, S. Cromwell & H. Morikawa (Eds.). Proceedings of the JALT 1995 International Conference on language Teaching/Learning. (pp. 283-292). Tokyo: The Japan Association for Language Teaching.

Griffee, D. T. (1997a). Validating a questionnaire on confidence in speaking English as a foreign language. The JALT Journal, 19 (2), l 77- I 97.

Griffee, D. T. (1997b). Where are we now? Trends, teachers, and classroom research. In D. Griffee & D. Nunan (Eds.), Classroom teachers and classroom research. Tokyo: The Japan Association for Language Teaching.

Hyland, K. (1994). The learning style of Japanese students. JALT Journal, 16 (1) 55-74.

Most, R., & Zeidner, M. (1995). Constructing personality and intelligence instruments. In D. H. Saklofske & M. Zeidner (Eds.), International handbook of personality and intelligence. (pp. 417-446). New York: Plenum Press.

Reid, J. M. (1987). The learning style preferences of ESL students. TESOL Quarterly, 21 (2) 87-109.

Sakamoto, M. (1996). The effect of translating survey questions. TUJ Working Papers in Applied Linguistics, 9, 82-88.

Shimizu, K. (1995). Japanese college student attitudes towards English teachers: A survey. The Language Teacher, 19 (10), 5-8.

Widdows, S., & Voller, P. (1991). PANSI: A survey of the ELT needs of Japanese university students. Cross Currents, 18 (2), 127-141.

Newsletter: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next
HTML:   /   PDF:

[ p. 14 ]