Research and validation
Research: at the heart of what we do
Cambridge ESOL is a not-for-profit department of the University of Cambridge. We believe we have a role and responsibility to develop language learning and assessment to help people achieve their life goals.
As leaders in the field of language assessment, quality and fairness are at the centre of everything we do:
- Quality - this means validity, reliability, impact and practicality.
- Validity - the extent to which exam marks can be considered a true reflection of underlying ability.
- Reliability - the extent to which test results are consistent, accurate and therefore dependable.
- Impact - the positive effect of a test on candidates and other users, including society as a whole.
- Practicality - the extent to which a test is workable in terms of the resources needed.
Fairness
All the exams are specifically designed to be fair to all test takers, whatever their age, gender, nationality, first language, culture or ethnic background.
Research
To be certain of both quality and fairness, we have to accurately measure how our tests perform and compare them with the latest thinking in language assessment. Both of these require extensive research.
Quality is rigorously monitored across all aspects of the exam process, and our Research & Validation Group works closely with the Assessment & Operations Group on test development, construction, administration, assessment and results processing, and exam review and evaluation.
These are just a few of the many research projects we undertake each year.
Test development
To ensure our assessment is learning-led, our Research & Validation Group invests heavily in a continual programme of performance monitoring, which includes validation at the development stage, data analysis, results interpretation, and exam review and evaluation.
For example, we have undertaken extensive research into the testing of language for special purposes (ILEC and ICFE exams). We have also conducted empirical studies into the underlying language proficiency needs of FCE and CAE candidates as part of the exam review research programme. As part of our new tests research, we are monitoring and analysing TKT test takers' language proficiency to see whether it may affect performance over time.
We also study the changes in the test user population and test taker characteristics which could lead to further test developments.
Test validation
We conduct on average 70 validation studies each year on existing and proposed exams as well as on assessment approaches and theories to assist test development and production. Validity studies include bias, impact on stakeholders, rater-reliability, and accommodations.
Test technology
With technology fast becoming central to the process of exams and testing, research into the use of technology and its impact on test takers is essential.
We regularly undertake such research to ensure our exams — through whatever means they are delivered — remain relevant, reliable, accurate and fair. Research into the post-launch phases of computer-based testing is ongoing.
Support for stakeholders
Research and validation provides information for external stakeholders who need to be aware of the quality and fairness of the exams and who need to have details of research and validation work carried out by Cambridge ESOL.
Cambridge ESOL & CEFR
Today the Common European Framework of Reference for Languages (CEFR) plays a central role in language and education policy not only within Europe but worldwide. For language testers it has become imperative to make the case that their exams are aligned to the CEFR. The Council of Europe has attempted to facilitate this by providing a toolkit of resources, including a draft pilot Manual for relating language examinations to the CEFR and a technical reference supplement to this (Council of Europe 2003, 2004). Cambridge ESOL is one of many language testers to have piloted the Manual, and in December 2007 hosted a seminar in Cambridge on behalf of the Council of Europe where all these case studies were reported, and experiences and recommendations shared.
More on the relationship between Cambridge ESOL and the CEFR.
Pretesting
To make certain our exams are valid, reliable, practical to administer and an accurate test on the subject, we invest heavily in exam trialling for all new exams and exam modifications. This takes place in many countries around the world. Find out more about pretesting.
Publishing
We share much of the results of our research with the wider language assessment community. Find out more about our research publications.
