Here are some suggested answers for the questions about testing, statistics, and assessment from the April 2008
issue of SHIKEN.
If you feel an answer is unclear or conclusion is incorrect, please contact the editor.
Basabas-Ikeguchi, C. (1988). Analysis of reading and listening comprehension skills in different language environments. Unpublished master's thesis, Dokkyo University. ERIC #: ED355807.
Burns, E. (1998). Test accommodations for students with disabilities. Springfield, IL: Charles C. Thomas.
ETS. (2007) 2007-2008 Bulletin supplement for test takers with disability. Retrieved April 12, 2008 from www.ets.org/disability/
Holt, J., Hotto, S., Cole, K. (1994). Demographic aspects of hearing impairment: Questions and answers. (Third Edition). Retrieved April 12, 2008 from http://gri.gallaudet.edu/Demographics/factsheet.html#Q1/
[ p. 46 ]2 Q: How should an oral proficiency interviewee with a likely stuttering disorder be rated?
ELSA. (2000). ELSA Links – Discrimination. Retrieved April 13, 2008 from http://www.stuttering.ws/links/discrim_eu.htm
Tyrer, A. (2007, September 23). Oral assessments, and assessed presentations. Retrieved April 13, 2008 from http://www.stammeringlaw.org.uk/education/oral.htm
[ p. 47 ]
Anderson, L. W. (2002, November) Curricular alignment: A re-examination. Theory Into Practice, 41, (4) 255 - 260. ERIC Document # EJ667162.
Barrie, S., Brew, A., McCulloch, M. (1999). Qualitatively different conceptions of criteria used to assess student learning. Paper presented at the 1999 Australian Association for Research in Education. Retrieved April 14, 2008 from http://www.aare.edu.au/99pap/bre99209.htm
[ p. 48 ]
Garcia, P. A. (1987). The competency testing mine field: Validation, legal and ethical issues with implications for minorities. ERIC Document # ED336967
Kun, T. (2007). American regional accent map. Retrieved April 15, 2008 from http://freeshells.ch/~xavier/accentmap/
Saar, H. (2005, January 17). Validation guidelines for test developers. Retrieved April 15, 2008 from www.qalspell.ttu.ee/Validation%20Guidelines%20for%20Test%20Developers.doc
University of Arizona Language Samples Project. (2001). Varieties of English. Retrieved April 15, 2008 from http://www.ic.arizona.edu/~lsp/main.html
Stivers, T. (2001). Negotiating who presents the problem: Next speaker selection in pediatric encounters. Journal of Communication, 51, 252-282.
Stivers, T. (2002). Presenting the problem in pediatric encounters: "Symptoms only" versus "candidate diagnosis" presentations. Health Communication, 14, 299-338.
[ p. 49 ]Ten Have, P. (2000, July 3). Methodological issues in conversation analysis. Retrieved April 16, 2008 from http://www2.fmg.uva.nl/emca/mica.htm
TESOL Quarterly. (n.d.). Qualitative research: Conversation analysis guidelines. Retrieved from April 16, 2008 from http://www.tesol.org/s_tesol/sec_document.asp?CID=476&DID=2154
West, C. (1984) Routine complications: Trouble with talk between doctors and patients. Bloomington: Indiana University Press.
Wieder, D. L. (1993). On the compound questions raised by attempts to quantify conversation analysis' phenomena, part 2: The issue of incommensurability. Research on Language and Social Interaction, 26 (2) 213-26. ERIC #: EJ464150.
[ p. 50 ]Bremner, S. (1997, Autumn). Language learning strategies and language proficiency: causes or outcomes? Perspectives (9). Retrieved from April 18, 2008 from http://sunzi1.lib.hku.hk/hkjo/view/10/1000125.pdf
Cohen, A. D. & Upton, T. A. (2006). Strategies in responding to the new TOEFL reading tasks. TOEFL Monograph No. MS-33. Princeton, NJ: ETS. Retrieved from April 17, 2008 from http://www.ets.org/Media/Research/pdf/RR-06-06.pdf
Cohen, A. D. (2007) The coming of age for research on test-taking strategies. In J. Fox, et al (Eds.) Language Testing Reconsidered. Ottawa, Ontario: University of Ottawa Press., pp. 89 - 112.
Edwards, B. (2003, August). An examination of factors contributing to a reduction in race-based subgroup differences on a constructed response paper-and-pencil test of achievement. Unpublished Ph.D. thesis at Texas A&M University. Retrieved from April 17, 2008 from http://txspace.tamu.edu/bitstream/handle/1969.1/128/etd-tamu-2003B-2003062513-Edwa-1.pdf?sequence=1
Gu, P.Y. (1996). Robin Hood in SLA: What has the learning strategy researcher taught us? Asian Journal of English Language Teaching, 6, 1-29.
Lessard-Clouston, M. (1997, December) Language Learning Strategies: An Overview for L2 Teachers. The Internet TESL Journal, 3 (12). Retrieved from April 17, 2008 from http://iteslj.org/Articles/Lessard-Clouston-Strategy.html
Mahamed, A., Gregory, P., Austin, Z., & Dan, L. (2006, December). Testwiseness among international pharmacy graduates and Canadian senior pharmacy students. American Journal of Pharmaceutical Education, 70 (6), p. 131. Retrieved from April 17, 2008 from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1803693
Millman, J., Bishop, C. H., & Ebel, R. (1965). An analysis of test wiseness. Educational and Psychological Measurement, 25, 707-726.
Rogers, W. T.; Bateson, D. J. (1991, April). The Influence of Test-Wiseness on Performance of High School Seniors on School Leaving Examinations. Applied Measurement in Education, 4, 159 - 183.
Tarone, E. (1983). Some thoughts on the notion of 'communication strategy'. In C. Faerch & G. Kasper (Eds.), Strategies in Interlanguage Communication (pp. 61-74). London: Longman.
[ p. 51 ]
[ p. 52 ]JD Brown (2008, p. 36-41) offered two examples of how p-value results could be misleading in this issue of SHIKEN. If you look through the literature closely, it is not hard to find plenty of others that exist.
Brown, J.D. (2008, April). Statistics Corner. Questions and answers about language testing statistics: Effect size and eta squared. Shiken: JALT Testing & Evaluation SIG Newsletter, 12 (2) 36 - 41. Retrieved from April 18, 2008 from http://jalt.org/test/bro_28.htm
Dixon, P. (2003, September). The p-value fallacy and how to avoid it. Canadian Journal of Experimental Psychology, 57, 189-202. Retrieved from April 18, 2008 from http://www.psych.ualberta.ca/~pdixon/Home/Preprints/pValue.pdf
Dixon, P. (2000, July). The p-value fallacy: Why inferential statistics don't describe results. Paper presented at the joint meeting of the Experimental Psychology Society of Great Britain and the Canadian Society for Brain, Behaviour, and Cognitive Science, Cambridge, UK. Retrieved from April 18, 2008 from http://www.psych.ualberta.ca/~pdixon/Home/Presentations/pValues/pValues.htm
Killeen, P. R. (2005, May). An Alternative to Null-Hypothesis Significance Tests. Psychological Science, 16 (5) 345-353. Retrieved from April 18, 2008 from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1473027
Wright, D. (2008, March 3). Killeen's prep. Retrieved from April 18, 2008 from http://www.sussex.ac.uk/Users/danw/masters/statistical%20analysis/killeen.htm [inactive link]
[ p. 53 ]A: According to Trochim (2006) and Jacobs (2006) effect size – Option (C) – does have an impact on statistical power.
Becker, L. (2000, March 21). Effect size. Retrieved from April 19, 2008 from http://web.uccs.edu/lbecker/Psy590/es.htm
Jacobs, R. (2006, December 19). The concepts of statistical power and effect size. Retrieved on April 19, 2008 from http://www83.homepage.villanova.edu/richard.jacobs/EDU%208603/lessons/stastical%20power.html
Meta Analysis. (2002). In S. A. Mousavi An Encyclopedic Dictionary of Language Testing. (3rd Ed.). (pp. 411-413). Taipei: Tung Hua Book Company.
Trochim, W. M.K. (2006). Research methods knowledge base: Statistical power. Retrieved on April 19, 2008 from http://www.socialresearchmethods.net/kb/power.php
[ p. 54 ]