Cambridge ESOL’s world-leading Research & Validation team leads studies of the constantly evolving field of English language testing, providing an objective basis for the work of the whole organisation – and delivering the kind of investigation and knowledge that sets Cambridge ESOL exams apart from many other providers. The team is continually seeking ways to enhance fairness, security and accessibility for candidates. Research & Validation is also involved in extending the range of exams while maintaining consistent global standards.
The team is working on a range of projects that will have a long-term impact on the nature and delivery of qualifications, and which are a core part of Cambridge ESOL’s uncompromising commitment to quality.
Harnessing new technologies
As part of an overarching strategy for the production and use of test materials, the Research & Validation team has been exploring the potential for developments and enhancements based on new technology. An online test delivery engine is under development. In addition to providing computer-based tests direct to centres, the engine would be linked into Cambridge ESOL’s back office for entries, marking and results, expanding the potential for far more flexibility, more test sessions, shorter lead-times and quicker turnaround for results. The team is also looking at new approaches to data collection so that future tests take into account trends and issues from previous test takers.
Supporting computer-based testing
The introduction of computer-based tests, such as the computer-based PET, has demanded close attention to issues of comparability between computer-based and pencil and paper test formats. Large-scale research and pretesting on computer- based versions of PET and IELTS, as well as data from BULATS and other computerised tests, allows us to align the different versions so that standards are consistent.
A general finding from the trials has been that ultimately some candidates feel more comfortable with a computer test and others prefer pencil and paper. For this reason, any computer-based tests are also offered in both formats so that candidates may choose the mode of delivery most suitable for them.
Supporting new test development
Test reliability – ensuring that scores or results produced by the test are accurate/ consistent and as free as possible from errors of measurement – is a crucial element of quality for existing exams, and is critically important to the support for new ESOL products: TKT (Teaching Knowledge Test), ESOL Skills for Life and ILEC (International Legal English Certificate).
Work currently in progress on TKT is an investigation into the relationship between language ability and TKT performance based on language tests given to candidates in live examination situations. A study of the performance of native and non-native speakers of English, as well as specialists and non-specialists, has shown that there is no real difference in terms of levels of performance between the groups. The results provide an indication of the fairness of the TKT test, which is intended to be suitable for candidates from different backgrounds.
Planned research projects on ESOL Skills for Life include establishing what analyses are required to monitor effectively the Writing and Listening/Speaking modes and respective examiners and markers to ensure score reliability and compliance with Cambridge ESOL’s own quality standards.
With ILEC due its first administration in May this year, pretesting was conducted in December 2005. This followed an earlier trial of Reading, Writing and Listening components on a sample of candidates, which provided positive feedback on the exam’s format, rubrics and construction. ILEC is designed to test the level of English in a legal setting for candidates at B2 and C1 (Council of Europe’s Common European Framework of Reference for Languages).
Expanding the learner corpus
Alongside Cambridge University Press we continue to build the Cambridge Learner Corpus (CLC), which now contains 23 million words taken from 65,000 exam scripts written by candidates taking our exams all over the world. This unique resource helps us to validate existing tests and develop new exams, also feeding into a number of other research projects. For example, the CLC is being used with our collection of speaking tests to help define the core vocabulary of candidates operating at Common European Framework B1 level in writing and speaking. We hope to add Skills for Life and BULATS scripts to the CLC in the coming year.
You can find an archive of research articles from the Cambridge ESOL Research & Validation team at www.CambridgeESOL.org/researchnotes