Abstract

As validity pertains to test use rather than the test itself, using a test for unintended purposes requires a new validation program using additional evidence from relevant sources. This small-scale study contributes to the validation of the use of originally academic language tests-the International English Language Testing System and the Test of English as a Foreign Language-for assessing skilled immigration eligibility. Data were collected from 39 immigration-seeking test-takers, who are arguably under-represented in validation research. Analysis was informed by contemporary validity theory, which treats validity as a unitary concept incorporating score reliability, score interpretation, score-based decisions and their consequences. Results showed that the test-takers’ perceptions varied widely. The evidence supporting this use included generally positive perceptions of the scores’ reliability, washback effect, and fairness of score-based decisions. The refuting evidence concerned factors perceived to interfere with test-takers’ performance and the complex consequences for the test-takers in aspects other than washback. However, overwhelmingly, as test-takers found the score-based decisions as fair, the validity judgement appeared tilted towards the positive side from the perspectives of these key stakeholders. Although the ultimate validity judgement requires the examination of evidence from other significant stakeholders as well, the present study has contributed valuable and unique evidence and bears important implications for research, practice, and policy particularly in high-stakes contexts such as immigration.

Suggested citation