Curriculum Innovation, Testing and Evaluation:
Proceedings of the 1st Annual JALT Pan-SIG Conference


by Alan S. Mackenzie & Tim Newfields

These proceedings present a record of some of the papers presented at the JALT Pan-SIG Mini-Conference held on May 11th-12th 2002 at the Kyoto Institute of Technology. Two of the SIGs involved: CUE (College and University Educators) and TEVAL (Testing and Evaluation) combined their efforts to produce this volume because they saw a firm link between their themes, Curriculum Innovation and Testing and Evaluation in the 21st Century.
Most universities and colleges in Japan are changing their curricula as a whole and their foreign language curricula in particular for a number of reasons. There is increased competition for students due to the changing population structure. Being seen as innovative and producing new and interesting curricula is an attempt to attract more students. There is also a realisation that students go through university language programs without much change in their language ability. Many institutions are trying to rectify this through structural and methodological changes at curricular, course and class delivery levels. A third influence is the Japanese Ministry of Education, Culture, Sports, Science and Technology's desire to improve tertiary level education standards through policy changes at tertiary, secondary and elementary levels. Integral to the innovations they are recommending are changes in entrance exams, placement procedures for levelling or streaming students and assessment procedures for monitoring student progress. The papers in this volume are divided into four sections: curriculum innovation issues, pro-autonomy curriculum innovations, entrance examinations and placement, testing within the curriculum. A brief summary of each section follows.

Curriculum innovation issues

Richard Blight opens this section with a description of the processes involved in moving from grammar-translation-based teaching to communicative methods in a national university. This paper illustrates that simple methodological changes imply alternative structural arrangements for schools and deep policy shifts in order to be effective.
Next, Anna Seabourne explicates the changes in a private university attempting to move toward a content-based program. She also recounts what was learned from this methodological shift, emphasizing how all changes within the curriculum should be monitored and evaluated.
Gregory D. O'Dowd then explores different ways in which curriculum change can be implemented and presents a case study of the evaluation of curriculum innovation at a private university.
Exploring concerns teachers have about curriculum change, Neil Cowie details the impact of curriculum change on a faculty in a national university. He emphasises the need to develop more collaborative cultures in the staffroom in order to involve faculty members more in the curriculum change process. Similarly, Charles Kowalski investigates the causes of teacher stress and suggests ways in which schools can become more caring places that listen to teachers rather than merely tell them what to do.

[ p. 1 ]

Finally, Alan S. Mackenzie explains the changes currently underway in Thailand's national curriculum providing a different perspective on how another country is reforming EFL teaching and learning.

Pro-autonomy curriculum innovations

Echoing the 2001 CUE conference theme of Developing Autonomy, some presenters discussed curriculum innovations that focus on development of autonomy in its many guises.
The first paper by Dr. Alan Brady moves from personal reflection to detailed exposition concerning autonomy in higher education. He outlines a framework for encouraging students to explore, think critically and study authentic language use. Stacy Vye et al continue this theme from the teacher's viewpoint by describing a workshop in which they facilitated from the audience ways to become aware of curriculum constraints and how we can operate within them on a personal level. Finally Ikuko Kondo and Carolyn Obara and Robert Kirkpatrick demonstrate classroom- level techniques for respectively stimulating personal interest through videos and bringing religion into the classroom.

Entrance examinations and placement

JD Brown starts this section by describing how his attitudes toward Japan's entrance examination system have changed. Emphasizing that foreign teachers should not be afraid of taking pro-active roles in voicing concerns about unprofessional testing practises, he encourages people to consider test impact more deeply. Brown then outlines six general problems with Japan 's testing system and some specific ways to remedy the situation.
Tomoyasu Akiyama then considers how the English section of the high-school entrance exams in Tokyo are conducted. He mentions a proposal to make such exams more interactive and authentic by including direct speaking tests as an exam component. Akiyama asserts that this would enhance the construct validity of the exam and create more beneficial washback. He further stresses the need to develop more data about junior high school student performance and build up a roleplay task bank.
The next three papers in this section came from the CUE Roundtable on Leveling, Streaming and Placement Testing. Deborah Bollinger first describes how Tokai University's streaming system was developed, listing questions administers need to reflect on in implementing a similar system. After this, Amanda Gillis-Furutaka and Nobuko Sakurai describe how the curricula at their university changed recently as streaming was introduced. The need to inform students clearly about the purpose of streaming and to obtain their consent when changing levels is underscored. Finally, Alan S. Mackenzie mentions how his school evolved from a two-level into a three-level streamed program in order to meet the needs of students with limited foreign language skills. He points out that streaming is worthwhile if the criteria for each level are clear and well understood by students.

Testing within the curriculum

The next five articles explore the testing-curriculum interface further, and also mention some innovative approaches to language assessment.
Richard Blight's article points out how one section of a test of English proficiency for Australian immigrants was evaluated. Underscoring that test revision should be conceived of as a cyclical process, he emphasises the need to constantly reanalyze the degree to which test results, test specifications, and program objectives correlate.

[ p. 2 ]

After this, Mohd. Sallehhudin Abd. Aziz specifies how a test for law students at a university in Malaysia was designed, constructed, and validated. Factors to consider in developing a large-scale EAP proficiency test are discussed in detail. The model for test development outlined in this article provides many helpful tips about how to accurately assess English academic ability.
Alistair van Moere and Francis C. Johnson provide an overview of how a video- mediated proficiency test at a private university relates to the overall curriculum and academic goals of that institution. The test they outline is used not only for streaming, but also to allow students to bypass some courses. Moreover, the school uses it to set performance standards for required courses. Ways that the curriculum at their school is being individualized as a result of this streaming process are also recounted.
After overviewing how verbal reporting (VR) techniques are generally used to obtain respondent data, Bob Gibson mentions some of the shortcomings of such procedures. He then suggests an alternative approach to obtaining feedback by having informants categorize their behaviors in terms of a set of a priori set of codings. The benefits and drawbacks of this procedure are then outlined. Related to this, Ellen Head briefly overviews how cloze tests differ from C-tests, mentioning some resources to learn more about C-tests.
The final article is this volumes concerns how to best assess EFL writing compositions. Since it is often impractical to have multiple raters and seldom reliable to have only one rater, Yuuji Nakamura suggests a technique of using paired raters. Various factors related to the rating process are then statistically analyzed through a many-faceted Rasch measurement (FACETS) model. The results indicate that a paired rating system can provide a practical and reliable way to assess compositions.


What this book attempts to do is outline some of the curricular innovations taking place in Japan today and demonstrate how the connection between testing and curriculum is both far-reaching and interactive. When so many institutions are becoming change- oriented, it is important to monitor those changes and examine the processes they, their faculties and their students go through. As educators, we should be striving for congruence between what we purport to teach and how we purport to measure the results of that teaching. If there is a high degree of alignment between these two important facets, then beneficial washback can be enhanced. The fact that there is often a lack of congruence between curricular content components and testing is borne out by many of the articles herein. Nonetheless, some innovative and practical ways to foster that connection are underscored. This volume demonstrates that the relationship between curriculum and testing is a vibrant research area. It also indicates many key areas in the testing-curriculum matrix which remain to be explored.

2002 Pan SIG-Proceedings: Author Index Main Index Page Index Title Index Topic Index
Complete Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index


[ p. 3 ]