Below the surface of written exams there is a complex structure of theory and technical knowledge which ensures they are accurate and fair.
The credibility of Cambridge ESOL examinations relies to a large degree upon firstly being able to test people reliably to a consistent standard, and secondly, that standard being an accurate representation of a real-life language situation.
For example, if the Writing element of a test was not well defined, it would be difficult to support the claims Cambridge ESOL makes about its usefulness. It would also be difficult to show that tests are not too narrowly focused or are not susceptible to external factors such as unreliable scoring or cultural differences.
Cambridge ESOL has developed a comprehensive base of theoretical models for the way tests are developed and how distinctions are made between different levels of tests.
Having a clear theoretical approach to the underlying structure of General English examinations (KET, PET, FCE, CAE and CPE) is also necessary to guide the review process for Writing tests and to provide sound reasons for any proposed changes.
For practical assessment purposes, these theoretical models have to deal satisfactorily with the triangular relationship between three vital elements:
- the test taker’s cognitive abilities
(what he or she understands) - the task and context
- the scoring process
By maintaining a strong focus on these three components and by undertaking careful analysis of our tests in relation to them, we can provide robust, theoretical and empirical evidence to support our claims about the validity, quality and usefulness of our exams.
The key to defining an appropriate Writing component of a test is in identifying what factors make up the real-world language use, and which
of those factors are necessary to be measured.
But no single definition of language ability is applicable for all purposes, and each testing situation must take into account the test takers,
the purpose of the test and the real-life situation the test simulates.
For this reason, the information gathered about test takers (e.g. from candidate questionnaires) is of great importance and feeds directly into decisions about test design across the different exam suites and proficiency levels.
Much has already been achieved towards a better understanding of second language writing proficiency and how it can be assessed. As part of Cambridge ESOL’s commitment of ongoing review and improvement, the programme of research continues.
This article is adapted from: Defining the constructs underpinning Main Suite Writing Tests: A Socio-Cognitive Perspective by Cyril Weir and Stuart Shaw in Research Notes 26. To find out more about this subject, you can download Research Notes 26 free from: www.cambridgeesol.org/rs_notes/rs_nts26.pdf
You can also read this book on the subject by the same authors: Examining Second Language Writing: research and practice, by Stuart Shaw and Cyril Weir: Studies in Language Testing, Volume 26: Cambridge University Press.