Cover of Text

Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 10 No. 1. Mar. 2006. (p. 18 - 19) [ISSN 1881-5537]

Book Review

Issues in Testing Business English (Studies in Language Testing 17)
By Barry O'Sullivan (2006) [ISBN 0-521-01330-5]
Cambridge, U.K.: Cambridge University Press (394 pp.)

The main purpose of this book is fairly broad: not just to discus business English testing, but "to provide a framework for classifying and understanding specific purpose language assessment." (p x). More specifically, about a third of the text is devoted to an analysis of eleven different business language tests. Finally, the last half of the book deals with the revision of the Business English Certificate (BEC), an examination created in 1993 by the UCLES for the Chinese National Education Examinations Authority (NEEA).

Chapter 1

Chapter 1 lays the theoretical foundations. A brief historical introduction sets the scene, giving examples of language learning for business use, and an overview of business language tests. The author then summarizes the theory of business language testing, suggesting that business language testing is a type of testing language for specific purposes (LSP). However there are degrees of specificity, and he suggests the concept a "continuum of specificity" (p 14) to explain the variety of LSP tests.
This brings us to a refreshingly concise summary of Douglas's (2000) ideas, together with the objections which have been raised to them. Douglas's main concept is authenticity. An LSP test is "one in which . . . test tasks and content are authentically representative of tasks in the target situation" (quoted on p 3). The two underlying ideas are firstly that "language performance varies with the context of that performance" (p 3), and also that they are distinguished from general-purpose tests in their language. Douglas says LSP tests are situationally authentic because the language comes from a specific "language domain". They are also interactionally authentic, because the tasks replicate target tasks.
O'Sullivan then attacks two main problems with LSP – and therefore also business English – tests: Can LSP tests be clearly distinguished from general English tests? Can LSP tests be made both situationally and interactionally authentic? His answer to the first question is that we know from studies of the role of background knowledge in tests that specific areas are clearly definable, and that on a highly specific test, subject experts outperform laymen.
As for authenticity, situational authenticity is possible, especially in highly specific tests such as for air traffic controllers. Interactional authenticity is still a rather unsatisfactory concept. The test should result in a interaction between the task and a linguistic skill, but "insufficient work has been done to link context-based validity elements to theory-based processing" (p 7) This section concludes with a discussion of how to assess performance by various rating scales.
The final part of the lengthy first chapter is a review of major business language tests, including those for English, French, Italian and Japanese. A nine-point scheme is used to classify and evaluate the tests, and the discussion is helped by a liberal use of charts and sample exam questions.
The most familiar of these tests to readers of this journal will probably be TOEIC®, which is described as "problematic" for many reasons. It is too general to be classified as a business English test. There is a contradiction between what the writers claim that the test shows – "TOIEC test scores indicate how well people can communicate in English with others in the global workplace." (quoted on p 18) and what it really allows us to say about the candidate. This is because "There is clearly some confusion as to the underlying construct of the TOIEC" (p 18). Another problem is that the test is based on a theory of separate skills. O'Sullivan criticizes the writers for "the failure . . . to respond to changes in theoretical perspectives of language competence" (p 19) The skills coverage is inadequate because language knowledge rather then language use is being tested. The score reporting is also unsatisfactory since the score is related to the population who took that administration of the test, rather than to a criterion level.

[ p. 18 ]

Chapters 2-5

The remainder of the text deals with the BEC test. Firstly, we read a history of the revision process, an intriguing story for most of us who have never been involved in producing a large-scale test. The UCLES test development method (described in Saville, 2002) is shown in practical terms. We are introduced to the construct of business English and how it is arrived at practically by considering data on the test takers. The process of assessing and using three types of validity are dealt with: theory-based validity, content-based validity and scoring validity (reliability). Then the four qualities of the Cambridge ESOL tests – validity, reliability, impact and practicability – are explained. Finally the practical issues of administration and grading are addressed. The last few pages outline the revision process. This process was based on "an extensive consultative exercise", (p 117) and examination qualities were identified using the VRIP system: "validity, reliability, impact and practicality." (p 117)
Major changes to the suite is the title of the third, fairly brief, chapter. This deals with general changes such as the way results are reported, the weighting of the components, both within each section of the test and also within each BEC level, and the changes to the speaking content.
Following these high-level changes, the details and rationale for revisions of sections of the BEC test are analyzed in Chapter 4. In short, "the most significant changes have come in response to criticism of the Speaking paper, while the other papers . . . were left relatively untouched." (p 171).
The final chapter returns to an overview perspective. A welcome summary of the book's essential argument is presented. This leads us to a discussion of specificity, authenticity and validity. These theoretical discussions are balanced and clarified by a section on practical implications: reliability issues and the use of computers. The author concludes with a thought-provoking agenda for LSP testing. For example, with regard to testing of speaking, it is a good idea to include many different tasks. He suggests, "research is required into the practicality of awarding scores of individual testing operational conditions." And he recommends, "It is necessary to investigate the impact on performance of factors such as interlocutor variables (e.g. sex and age) or candidates' perception of the interlocutor/audience . . ." (p 192)
The four appendices, running to nearly 200 pages, give illustrative sample papers of the pre-and post revision BEC test in addition to other relevant tests: BULATS, CEIBT, and CEFL.
There is some reference to Japan, where the author worked for a number of years: the JETRO test of Business Japanese is described. Moreover, eight pages are devoted to the TOEIC test. However, the language testing world is very much an international community, and this is the overriding impression of this text.
The style is generally fluid, knowledgeable yet lucid, using the first person pronoun. In addition, the diagrams are very helpful aids to exegesis of quite complex ideas. However, occasionally this reviewer stumbled in reading and had to backtrack. This is due in part to the author's conversational writing style, whereby at the end of a sentence an extra thought is added after a dash. This literary mannerism bloats many sentences, as for example two successive sentences in the third paragraph on p 15. Moreover, perhaps a more energetically edited text would have avoided double negatives, such as "a not incompatible expansion" on p 14. A final minor point there is at least one run-on sentence, on p 192, the third complete sentence.


This text is recommended both to both specialists and generalists. Those who are involved in test production will be enlightened by the theoretical discussion of LSP tests, and the rigorous analyses deployed in the case study section of Chapters 2 - 4. Generalist teachers preparing students for the large-scale LSP examinations such as the TOEFL® or TOEIC, will find this work both accessible and balanced. For native English speaker students taking a testing module, it provides an up-to-date alternative to Douglas when considering LSP testing.

- Reviewed by Daniel Dunkley
Aichi Gakuin University

Works Cited

Douglas, D. (2000). Assessing languages for specific purposes. Cambridge: Cambridge University Press.

Saville, N. (2002). The process of test development and revision within UCLES EFL. In C. Weir & M. Milanovic (Eds.) Continuity and innovation: Revising the Cambridge Proficiency English Examination 1913-2002. Cambridge: Cambridge University Press. p 57-120.

Weir, C. (2005). Language testing and validation: An evidence-based approach. London: Palgrave.

NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next

[ p. 19 ]