Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 13 No. 3. Nov 2009. (p. 26 - 28) [ISSN 1881-5537]
PDF PDF Version

Assessment Literacy Self-Study Quiz #8
by Tim Newfields

This ongoing column features questions about testing, statistics, and assessment in a quiz format to promote greater assessment literacy.
Suggested answers to the problems below are online at

Part I: Open Questions

Figure 1 from the Higginbotham article
  1. An article by Higginbotham (2009, pp. 15-18) in a recent issue of The Language Teacher attempted to highlight the value of extended task based instruction. Support for this instructional approach was based on a single questionnaire about a carol singing activity using that instructional framework. One month after the activity, 69 of of the 80 students who participated in that activity completed a Likert questionnaire and vocabulary test with merely five words from the Christmas carols they had supposedly memorized. Based on those results, the author suggests activities such as this lead to "deep processing of the language" (p. 15).

    What confounding issues are present in this study? How could the validity and reliability of studies such as this be enhanced? Also, how should the graph to the right probably have been revised for academic publication?

    Source: Higginbotham, G. (2009). Event-based learning: The benefits of positive pressure. The Language Teacher. 33 (1) 15-18.

  2. [ p. 26 ]

  3. At most (if not all) universities in Asia, multiple entrance pathways exist. One widely used pathway is the entrance by recommendation system Suisen Nyugaku In this system, a limited number of students from a select number high schools are allowed to recommend canditates for admission to a specific university. Those select schools are often, but not always, affiliated with each given university in some way. In many cases, admission decisions are based on a submitted essay and a very short screening interview. Applicants generally have a month or so to compose their essays and they usually solicit the help of parents and/or teachers when composing and revising their essays. Moreover, most of the oral interview questions are known in advance and many students are able to churn out memorized answers to the most common questions.

    What specific test validity issues are compromised through this process? What viable alternatives to the current entrance by recommendation (suisen nyuushi) system exist?

  4. What are the pros and cons of balancing the correct answer choice sequences in the answer sheets of fixed-response exams? What alternative ways of organizing multiple-choice key answers currently exist?

    Source: Harrell, F. & Banfield, J. (2005, March 8). R Graph Gallery Database: Box-Percentile plot. Retrieved April 5, 2009 from
  5. TOEIC Bridge - Math Test Correlation
  6. In an ETS-funded research report, the TOEIC Bridge (Reading Section) was found to have a .84 correlation with a government run test of mathematics ability among 198 high school students in Columbia as the graph below suggests. What would be an appropriate interpretation of these results?

    Source: Sinharay, S. et al. (2008). Establishing the Validity of TOEIC Bridge Test Scores for Students in Colombia, Chile, and Ecuador. ETS Research Report 08-58. Retrieved on November 12, 2009 from


Part II: Multiple Choice Questions

  1. One source of confusion in the field of language testing is that the symbol p is used to denote two entirely different concepts. Which concepts below are commonly signified by p by different authors?

  2. [ p. 27 ]

  3. Which of the following statements are true about the KR-21? (Note: More than one correct response is possible.)

  4. Which of the following statements are not true about Wald tests? (Note: More than one correct response is possible.)

  5. The following grading scheme suggested by Davies (2002) is an example of
    Grading Scheme by Davies

  6. Which of the following statements are true about parameter drift? (Note: More than one correct response is possible.)

    NOTE: At least two answers for each sub-question are possible.


NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next
HTML:   /   PDF:

Quiz #1: Qs   / As    *    2: Qs   / As    *    3: Qs   / As     *    4: Qs   / As    *    5: Qs   / As   *    6: Qs   / As    *    7: Qs   / As    *    8: Qs   / As    *    9: Qs   / As   *    10: Qs   / As   *    11: A HREF="SSQ11.htm">Qs   / As  

[ p. 28 ]