#### Shiken: JALT Testing & Evaluation SIG NewsletterVol. 13 No. 2. May 2009. (p. 24 - 34) [ISSN 1881-5537]PDF Version

Assessment Literacy Self-Study Quiz #7
by Tim Newfields

This ongoing column features questions about testing, statistics, and assessment in a quiz format to promote greater assessment literacy. Suggested answers to the problems below are online at http://jalt.org/test/SSA7.htm.

### Part I: Open Questions

1. Please examine the following test item and then answer these questions:(1) What skills is this task likely tapping into? (2) In what ways (if any) might this item be inappropriate in a general university undergraduate EFL entrance exam?

This item was taken from South Korea’s 2008 College Scholastic Ability Test [2008 Daehak suhakneungnyeok siheom] – one of the three types of exams that high school applicants seeking university admission must take.

The task is to determine the correct sequence of three passages (A-C) that follow a 41-word paragraph opening.

Source: Korea Institute of Curriculum and Evaluation. (2008). 2008 Haknyeondo Daehaksuhakneungryeoksihum Munjeji Oegukeo (Yeongeo) Yeongyeok – Holsuhyeong Oegukeo (Yeongeo) Yeongyeok 7. [2008 College Scholastic Ability Test Foreign Language (English) Field 7]. Retrieved April 7, 2009 from http://www.kice.re.kr/ko/board/view.do?article_id=64464&menu_id=10089&board_id=10153

2. [ p. 24 ]

3. Explain the difference between these concepts: (1) local test item independence and (2) test item unidimensionality.

4. Look at the box-percentile plots below, then briefly describe how the two groups differ. Also, what advantages do these projections have over common boxplots? Finally, how can such box-percentile plots such be drawn?

Source: Harrell, F. & Banfield, J. (2005, March 8). R Graph Gallery Database: Box-Percentile plot. Retrieved April 5, 2009 from http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=77

5. In 2005 Hirsch proposed a simple index as one way of quantifying faculty research output and granting promotions. This index, known as the h-index, combines two metrics: (1) the number of papers one has written [k] and (2) the number of times those papers have written have been cited [d]. The precise formula is:

Hence an author with a h-index of 12 has at least 12 publications, all of which have each been cited at least 12 times. That author might have a few publications with more citations, and many with less – but 12 is the largest number of articles-with-equal-number-of-citations for this person.

Source: Hirsch, J. E. (2005, September 1). An index to quantify an individual's scientific research output. Physics and Society, (DOI: 10.1073/pnas.0507655102) Retrieved April 3, 2009 from http://arxiv.org/abs/physics/0508025

[ p. 25 ]

### Part II: Multiple Choice Questions

1. Which of the following statements is true about Poisson distributions?

2. The graph below is an example of a

3. Which of the following is least likely to improve the reliability of an examination?

[ p. 26 ]

4. One difference between the KR-20 and KR-21 is:

5. What is an ideal item facility index (a.k.a. “p-value” or “item difficulty index”) for a 4-option, multiple-choice norm-referenced test item? What about a true-false norm-referenced test item? What if both these items were designed for a criterion-reference test?

NOTE: At least two answers for each sub-question are possible.