Investigating the Quality of Language Tests; Calculating the Reliability Coefficient of Placement Test as a Case Study
DOI:
https://doi.org/10.26750/Vol(10).No(1).Paper11Keywords:
Reliability Coefficient, Split-half Method, Placement Test.Abstract
Language measurement tools should be valid and reliable by which the test scores obtained help drawing meaningful inferences and, fairly enough, significant decisions are made. This study attended to the reliability of an English language placement test. The aim of the study is to find out the reliability estimate of that type of test. Based on the Classical Test Theory, the reliability coefficient of the test should be computed to find out the range of the measurement error and achieve an estimate of the candidates’ true scores. To this end, the data obtained from the administration of University of Raparin’s Language and Development Center Placement Test[i], which was administered to 889 freshmen students, was analyzed. The method is descriptive and exploratory in which, by using the split half method, the three formulas – Kuder-Richardson 20 and Spearmen-Brown Prophecy – were used to calculate the reliability estimate of the test. The results showed that the reliability estimate of the test is 0.81. They also showed that the number of the items of the test should be doubled in order to achieve the desired reliability of 0.9 and above. Thus, based on the results obtained, it is recommended that the test structure should be improved to achieve a higher rate of reliability estimate. It is also recommended that the test should be modified; test items need to be increased and the content amended to cover all the language skills. Additionally, it is recommended that other sources of unreliability, namely, test administration and individual test items need to be studied to find the source of measurement error.
References
ARON, A. & ARON, E. N. 2003 (3rd ed.) Statistics for psychology. United States of AMerica: Prentice-Hall/Pearson Education.
ARYADOUST, V. & RAQUEL, M. 2019. Quantitative data analysis for language assessment Volume I: Fundamental techniques. USA: Routledge; Taylor & Francis Group.
BACHMAN, L. F. 1990. Fundamental considerations in language testing, Oxford, England, Oxford University Press.
BACHMAN, L. F. 2004. Statistical analyses for language assessment, Oxford, England, Oxford University Press.
BACHMAN, L. F. 2005. Building and supporting a case for test use. Language Assessment Quarterly: An International Journal, 2, 1-34.
BACHMAN, L. F. & PALMER, A. S. 1996. Language testing in practice: Designing and developing useful language tests, Oxford University Press.
BACHMAN, L. F. & PURPURA, J. E. 2008. 32 Language Assessments: Gate-Keepers or Door-Openers? The handbook of educational linguistics, 456-469.
BROWN, H. D. 2004. Language Assessment: Principles and Classroom Practices: United Stated of America. A Pearson Education. Inc.
BROWN, J. D. 2005. Testing in language programs: a comprehensive guide to English language assessement, New York, McGraw-Hill College.
CARMINES, E. G. & WOODS, J. A. 2005. Reliability Assessment. In: KEMPF-LEONARD, K. (ed.) Encyclopedia of Social Measurement. New York: Elsevier.
CHAPELLE, C. 2013. Reliability in language assessment. In: CHAPELLE, C. A. (ed.) The Encyclopedia of Applied Linguistics. Oxford: Blackwell/Wiley.
FAN, J. & JIN, Y. 2020. Standards for language assessment: demystifying university-level English placement testing in China. Asia Pacific Journal of Education, 40, 386-400.
FULCHER, G. 2013. Practical language testing, Routledge.
GREEN, A. 2012. Placement testing. In: C. COOMBE, P. D., & B. O’SULLIVAN (ed.) The Cambridge guide to second language assessment. New York: Cambridge University Press.
HARRIS, D. P. 1969. Testing English as a Second Language.
HUGHES, A. 2003. Testing for language teachers (2nd Edition), Ernst Klett Sprachen.
KUDER, G. F. & RICHARDSON, M. W. 1937. The theory of the estimation of test reliability. Psychometrika, 2, 151-160.
LADO, R. 1961. Language Testing: The Construction and Use of Foreign Language Tests. A Teacher's Book.
LONG, A. Y., SHIN, S.-Y., GEESLIN, K. & WILLIS, E. W. 2018. Does the test work? Evaluating a web-based language placement test. Language Learning & Technology, 22, 137-156.
MERIAM-WEBSTER DICTIONARY. n.d. Reliability coefficient [Online]. Available: https://www.merriam-webster.com/dictionary/reliability%20coefficient [Accessed February 4 2021].
MOUSAVI, S. A. 2012. An encyclopaedic dictionary of language testing, Tehran: Iran, RAHNAMA PRESS.
PIEDMONT, R. L. 2014. Reliability Coefficient. In: MICHALOS, A. C. (ed.) Encyclopedia of quality of life and well-being research. Springer Netherlands Dordrecht.
RAHMAN, M. S. 2020. The advantages and disadvantages of using qualitative and quantitative approaches and methods in language “testing and assessment” research: A literature review.
RAZAVIPOUR, K. & FIROOZI, T. 2021. Placement Decisions in Private Language Schools in Iran. Challenges in Language Testing Around the World: Insights for language test users, 267.
WEBB, N. M., SHAVELSON, R. J. & HAERTEL, E. H. 2006. Reliability Coefficients and Generalizability Theory. Handbook of Statistics, 26.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of University of Raparin
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
