Leading standards in assessment
Cambridge ESOL views itself as an ethical business with a social role to lead in standards of language learning through assessment to help individuals achieve their life goals. To ensure that its assessment is learning led, much of the work of its Research and Validation Group focuses on test development and validation to ensure its exams fulfil the criteria of fairness and quality.
In the rigorous monitoring of quality controls across the entire process of exam provision, the research teams work closely with the Assessment and Operations Group on test development, construction and administration, assessment processing and results, and post exam reviews. Although these activities form the backbone of the work, there is also a considerable programme of other research projects every year.
Test development
Cambridge ESOL invests heavily in trialling new exams to make sure the exams test what they purport to test, and are valid and reliable, practical to administer, and have a beneficial impact on teaching practices.
Language for Special Purposes testing research
With the introduction of ILEC and the development of ICFE, Research and Validation has undertaken various studies of the language needs of particular occupational groups. Constructing these specific purpose tests raises several considerations: content range and authenticity, the relationship between background knowledge and language knowledge, and test specificity (the degree to which a test contains tasks and language truly representative of the target language).
Exam reviews research
Empirical studies of the underlying language proficiency needs of FCE and CAE candidates were central to the FCE/CAE review process. The impact of this research will have a positive effect; broadening the understanding of the true nature of the writing ability of learners of a second language will help teachers to teach writing skills in a different or more enlightened way. For the YLE review – in addition to the test trials – an in-depth study of primary coursebooks ensured that wordlists and topics are up to date and reflect current classroom practice.
New tests research
By analysing TKT candidate levels of English, the research team monitors test takers’ language proficiency over time to determine the extent to which it may affect performance. The team is also investigating the difference in performance between candidates with less or more teaching experience. A study of changes in the test user population and test taker characteristics was also carried out which may lead to further test developments. Similar research of Skills for Life candidates was undertaken to ensure that the tests remain fair and relevant to all test takers.
Test validation
About 70 validity studies are conducted annually on existing and proposed exams as well as on assessment approaches and theories to assist test production. Validity studies include impact studies, trial analyses, viability investigations and reliability studies which examine test items to ensure they are fit for purpose. Several studies were undertaken for BULATS (for example, research into marker reliability to determine whether there are variances in interpretation of mark schemes by the markers of the exam papers), BEC (grading methods among other issues), YLE and IELTS.
There were also performance analyses of test takers and exam sessions as part of particular government education projects, and various data analyses at the design phase of a potential call centre test.
Test technology
Research into the post launch phases of two computer-based products (CB IELTS, CB PET) continued with performance monitoring of exam materials and candidate responses focusing on comparability between paper-and-pencil and computer-based responses. There was also research into test technology that could support automated test assembly and item cloning.
Enriching the learning environment
Conferences
Experts delivered papers on a range of subjects at 50 conferences around the world during the year as part of Cambridge ESOL’s commitment to share its knowledge and insights.
Research Notes
This year, the quarterly Research Notes included generic assessment subjects; for example, setting and monitoring professional standards, assessment systems (conceptual, human, technological), evaluating the impact of word processed text on writing quality, and items on specific product research.
Studies in Language Testing
Cambridge ESOL published (with Cambridge University Press) four volumes (17, 21, 22 and 24) of the academic series during the year.
These examined subjects such as testing of language for specific purposes, the impact of high-stakes testing on classroom teaching and the effects of the powerful influence of language teaching and testing programmes exerted on a wide range of stakeholder groups.