Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 13 No. 1. January 2009. (p. 31 - 34) [ISSN 1881-5537]
PDF PDF Version

Assessment Literacy Self-Study Quiz #6
by Tim Newfields

To encourage readers to become more familiar with key concepts and terminology in the field of language testing, this column explores questions about testing, statistics, and assessment in a quiz format Suggested answers to the problems below are online at

Part I: Open Questions

  1. Consider the following graph from the Japanese Ministry of Education, Culture, Sports, Science and Technology website, reputedly describing the "reasons for selecting a particular university" among high school students in Japan who appear to have successfully entered a Japanese university. What key information is missing from this graph? What problems are likely to arise in interpreting this graph? How should this graph be revised to minimize such problems?

    Figure 2.3 - Reasons for selecting a particular university

    [ p. 31 ]

    Source: Ministry of Education, Culture, Sports, Science and Technology of Japan. (n.d.). Remaking Universities: Continuing Reform of Higher Education: Chapter 2, Section 1, Sub-Section 3: Approaches to the Improvement of University Entrance Examinations. Retrieved December 1, 2008 from

  2. In the context of multiple-choice testing, what does "proportionality" refer to and why is this concept important to test designers?

  3. A well-known publisher and test developer has attempted to explain how its test correlated to two other famous tests by offering the following instructions:

    Follow these steps to convert Longman English Assessment percentage scores in Listening , Writing , and Reading into an approximate TOEFL® or TOEIC® score.

    Note that you must take all 3 Longman English Assessment (LEA) skill tests (Listening, Writing and Reading) to obtain a TOEFL or TOEIC correlation, and that the 3 skill tests must all be at the same level (Beginning, Intermediate, or Advanced).
    1. Gather your LEA score for each skill test. For example:

      Beginning Listening percentage score 70%
      Beginning Writing percentage score 80%
      Beginning Reading percentage score 90%
    2. Add the three scores to obtain a sum of the percentage scores.

      Sum of the percentage scores 240
    3. Divide the sum by 3 to obtain an average of the percentage scores.

      Average of the percentage scores 80%
    4. Use the correct table (depending on whether you took the Beginning, Intermediate, or Advanced skill tests) to determine your approximate TOEFL or TOEIC score. Approximate TOEFL score 440 Approximate TOEIC score 475

    LEA / TOEFL / TOEIC COrrelation Tables
    Source: Source: Pearson Education, Inc. (2003). Correlations between Longman English Assessment and TOEFL and TOEIC. Retrieved December 1, 2008 from

    [ p. 32 ]

    What ethical issues have been compromised in reporting this information? What information needs to be included to enable teachers and school administrators to make an informed decision about how well the exam developed by Pearson Education actually correlates with the other two ETS exams?

  4. Mention one concrete situation for which each of the following tests would be appropriate:
    1. A mastery test
    2. A power test
    3. A speed test
    4. A -ze test

    Part II: Multiple Choice Questions

    1. Which of these statements below is generally true of ANOVA tests? (Hint: Think a bit and don't expect a single, clear answer.)

    2. Which statement about the test item below is true?

      Figure 2 - An examination of item R29
      Source: Nakamura, Y. (2007). A Rasch-based analysis of an in-house English placement test. In T. Newfields, I. Gledall, P. Wanner, & M. Kawate-Mierzejewska. (Eds.) Second Language Acquisition - Theory and Pedagogy: Proceedings of the 6th Annual JALT Pan-SIG Conference. Retrieved December 16, 2008 from

      [ p. 33 ]

    3. Please look at the survey item below:

      What are the three main reasons that you have for studying English?
      Rank the most important one as 1 and the least important as 3.
      • Because I want to be able to understand Western music
      • Because I want to be a 'kokusai-jin' [English: "international person"]
      • Because I want to speak English when I go traveling
      • Because I have to as part of my course
      • Because I want to be able to teach my children English
      • Because I enjoy taking English classes
      • Because I am interested in other cultures by learning English
      • Because I want to be able to understand western movies
      • Because if I speak English other people will respect me more

      Source: Jenkins, A. C. (2005). EFL motivations and teacher preferences: A survey of Japanese university students in Niigata. Unpublished master's thesis, Department of Linguistic, Cultural & Translation Studies, School of Arts, University of Surrey, Guildford, Surrey, U.K. Retrieved December 16, 2008 from

      What type of scale is it based on?

      Also, how should this scale be revised to enhance its validity?

    4. In most 3-parameter IRT tests, the third parameter refers to _____________?

    5. At what point can a Pearson product-moment correlation be interpreted as "strong"?


    NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
    TEVAL SIG: Main Page Background Links Network Join
    last Main Page next
    HTML:   /   PDF:

    Quiz #1: Qs   / As   *    2: Qs   / As   *    3: Qs   / As   *    4: Qs   / As   *    5: Qs   / As   *    6: Qs   / As   *    7: Qs   / As   *    8: Qs   / As   *    9: Qs   / As   *    10: Qs   / As   *    11: A HREF="SSQ11.htm">Qs   / As  

    [ p. 34 ]