Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 15 No. 2. October 2011 (p. 30 - 42) [ISSN 1881-5537]
PDF PDF Version

Assessment Literacy Self-Study Quiz #11
by Tim Newfields

This column about testing, statistics, and assessment is in a quiz format and designed to promote greater assessment literacy. Suggested answers to all of the problems below are online at

Part I: Open Questions

  1. At most universities in Japan, how do the AO entrance examinations tend to differ from regular entrance exams and the National Center Test?

  2. What is a "false consensus effect" (FCE)? How might it skew some social science data and how should it be avoided?

  3. What unstated assumptions are inherent in the TOEIC "Can Do" statement chart below? What issues need to be considered when interpreting this chart? How could this information be more ethically presented?
  4. TOEIC score / English can do / job placements
    Source: Ingles Personalizado. (2011). TOEIC. Retrieved from
  1. Illustrate how a study with a regression discontinuity design could be structured to ascertain the effects of a specific intervention on English language reading fluency among a group of EFL high school students.

[ p. 30 ]

  1. What additional information should have been mentioned in the chart below to help readers to interpret the data more meaningfully? Also, how would you interpret this data distribution?
SPOT Scores
Source: Northwood, B. & Thompson, C. K. (2010). Why stop studying Japanese? A case in Australia. In A. M. Stoke (Ed.) JALT 2009 Conference Proceedings. (p. 821). Retrieved from
  1. In factor analysis, how does a communality differ from a commonality?

Part II: Multiple Choice Questions

  1. A tendency of respondents to concur with the opinions implicitly expressed in a given rating scale (and avoid offering counter-opinions) is a known as ________________.

  2. NOTE: One of these choices is most relevant; two are somewhat relevant, and two are irrelevant.

  3. IWhat is one difference between the standard error of measurement (SEM) and standard error of difference (SED)??

  4. NOTE: Just one answer is considered fully correct.

  5. What sort of test would be most appropriate to ascertain whether a large sample of male and female university students differed significantly in their attitudes towards study abroad according to a Likert scale with 4 or 5 options??

    [ p. 31 ]

  6. NOTE: : Two answers are viable, but one answer will yield more information than the other.

  7. Which of the following points is not problematic about the avowed ranking of these universities in the following chart?
    University Rankings
    Source: 2011 University Web Ranking. (2011). Universities in Japan. Retrieved from


NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Join
last Main Page next
HTML:   /   PDF:


Quiz #1: Qs   / Qs   / As   *    2: Qs   / As   *    3: Qs   / As   *    4: Qs   / As   *    5: Qs   / As   *    6: Qs   / As   *    7: Qs   / As   *    8: Qs   / As   *    9: Qs   / As   *    10: Qs   / As   *    11: Qs   / As  

[ p. 32 ]