Classical test theory and item response theory: two understandings of one high-stakes performance exam

Teoría clásica de la evaluación y teoría de respuesta al ítem: dos comprensiones de un examen avanzado de proficiencia

Authors

  • Gerriet Janssen University of Hawai‘i at Manoa
  • Valerie Meier
  • Jonathan Trace University of Hawai‘i at Manoa

Keywords:

evaluación de alfabetización, teoría de evaluación clásica, teoría de respuesta al ítem, evaluación de lenguas, análisis Rasch (es).

Keywords:

Language Testing, Classical Test Theory, Rasch Analysis, Assessment Literacy, item response theory (en).

Downloads

Abstract (en)

Language testing professionals and teacher educators have articulated the need for a broad variety stakeholders––including classroom teachers–– to develop assessment literacy. In this paper, we argue that when teachers are involved in local assessment development projects, they can expand their assessment knowledge and skills beyond what is necessary for conducting principled classroom assessments. We further claim that a particular analytic approach, Rasch analysis, should be considered as one possible element of this expanded assessment literacy. To this end, we use placement exam data from one Colombian university to illustrate how analyses from item response theory perspectives (Rasch analysis) differ from, and can usefully complement classical test theory.

Abstract (es)

Evaluadores de lengua y formadores de maestros argumentan que los involucrados en el campo de la educación, incluyendo los maestros de aula, deben desarrollar un conocimiento profundo en el tema de la evaluación. Planteamos que los profesores, a la hora de estar involucrados en el desarrollo de proyectos de evaluación, puedan expandir sus conocimientos y habilidades para ir más allá de la evaluación tradicional del aula. Para alcanzar este fin, proponemos que la herramienta de análisis Rasch sea considerada como una parte de esta expansión de conocimiento. En este ensayo, a través de los datos obtenidos de un examen de clasificación de lengua aplicado en un contexto universitario colombiano, ilustramos cómo el análisis Rasch puede complementar la teoría clásica de la evaluación.

References

Bachman, L. (2004). Statistical analyses for language assessment. New York, NY: Cambridge University Press.

Bachman, L. & Palmer, A. (1996). Language testing in practice. New York, NY: Oxford University Press.

Bachman, L. & Palmer, A. (2010). Language assessment in practice. New York, NY: Oxford University Press.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2nd edition.

Brennan, R. L. (2001). GENOVA. University of Iowa: Center for Advanced Studies in Measurement and Assessment.

Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. New York, NY: McGraw-Hill.

Brown, J. D. (2014). Classical theory reliability. In A. Kunnan (Ed.), Companion to language assessment, Vol. 3. Hoboken, NJ: Wiley Blackwell.

Ebel, R. L. (1979). Essentials of educational measurement, 1st edition. Upper Saddle River, NJ: Prentice Hall.

Henning, G., Hudson, T., & Turner, J. (1985). Item response theory and the assumption of unidimensionality for language tests. Language Testing, 2(2), 141–154.

Janssen, G., & Meier, V. (2013). Establishing placement test fit and performance: Serving local needs. Colombian Applied Linguistics Journal, 15(1), 100–113.

Linacre, J. M. (2010). Winsteps (Version 3.70.0.1). Chicago, IL: MESA Press.

Linacre, J. M. (2012). A user’s guide to Winsteps [software manual]. Chicago, IL: MESA Press.

López Mendoza, A. A., & Bernal Arandia, R. (2009). Language testing in Colombia: A call for more teacher education and teacher training in language assessment. PROFILE, 11(2), 55-70.

Marcoulides, G., & Ing, M. (2014). The use of Generalizability Theory in language assessment. In A. Kunnan, (Ed.), The companion to language assessment, Vol. 3 (pp. 1207–1223). New York, NY: John Wiley & Sons, Inc. DOI: 10.1002/9781118411360.wbcla014

McNamara, T. (1996). Measuring second language performance. New York: Longman.

Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory Into Practice, 48(1), 4-11. doi:10.1080/0040584080257753

Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21-36. doi:10.1017/S026719050909003

Wright, B. D., & Linacre, J. M. (1989). Observations are always ordinal; Measurements, however must be interval (MESA Research Memorandum No. 44). MESA Psychometric Laboratory. Retrieved November 10, 2013, from www.rasch.org/memo44.htm.

How to Cite

APA

Janssen, G., Meier, V., & Trace, J. (2014). Classical test theory and item response theory: two understandings of one high-stakes performance exam. Colombian Applied Linguistics Journal, 16(2), 167–184. https://doi.org/10.14483/udistrital.jour.calj.2014.2.a03

ACM

[1]
Janssen, G., Meier, V. and Trace, J. 2014. Classical test theory and item response theory: two understandings of one high-stakes performance exam. Colombian Applied Linguistics Journal. 16, 2 (Sep. 2014), 167–184. DOI:https://doi.org/10.14483/udistrital.jour.calj.2014.2.a03.

ACS

(1)
Janssen, G.; Meier, V.; Trace, J. Classical test theory and item response theory: two understandings of one high-stakes performance exam. Colomb. appl. linguist. j 2014, 16, 167-184.

ABNT

JANSSEN, G.; MEIER, V.; TRACE, J. Classical test theory and item response theory: two understandings of one high-stakes performance exam. Colombian Applied Linguistics Journal, [S. l.], v. 16, n. 2, p. 167–184, 2014. DOI: 10.14483/udistrital.jour.calj.2014.2.a03. Disponível em: https://revistas.udistrital.edu.co/index.php/calj/article/view/4950. Acesso em: 20 may. 2022.

Chicago

Janssen, Gerriet, Valerie Meier, and Jonathan Trace. 2014. “Classical test theory and item response theory: two understandings of one high-stakes performance exam”. Colombian Applied Linguistics Journal 16 (2):167-84. https://doi.org/10.14483/udistrital.jour.calj.2014.2.a03.

Harvard

Janssen, G., Meier, V. and Trace, J. (2014) “Classical test theory and item response theory: two understandings of one high-stakes performance exam”, Colombian Applied Linguistics Journal, 16(2), pp. 167–184. doi: 10.14483/udistrital.jour.calj.2014.2.a03.

IEEE

[1]
G. Janssen, V. Meier, and J. Trace, “Classical test theory and item response theory: two understandings of one high-stakes performance exam”, Colomb. appl. linguist. j, vol. 16, no. 2, pp. 167–184, Sep. 2014.

MLA

Janssen, G., V. Meier, and J. Trace. “Classical test theory and item response theory: two understandings of one high-stakes performance exam”. Colombian Applied Linguistics Journal, vol. 16, no. 2, Sept. 2014, pp. 167-84, doi:10.14483/udistrital.jour.calj.2014.2.a03.

Turabian

Janssen, Gerriet, Valerie Meier, and Jonathan Trace. “Classical test theory and item response theory: two understandings of one high-stakes performance exam”. Colombian Applied Linguistics Journal 16, no. 2 (September 22, 2014): 167–184. Accessed May 20, 2022. https://revistas.udistrital.edu.co/index.php/calj/article/view/4950.

Vancouver

1.
Janssen G, Meier V, Trace J. Classical test theory and item response theory: two understandings of one high-stakes performance exam. Colomb. appl. linguist. j [Internet]. 2014Sep.22 [cited 2022May20];16(2):167-84. Available from: https://revistas.udistrital.edu.co/index.php/calj/article/view/4950

Download Citation

Visitas

1099

Dimensions


PlumX


Downloads

Download data is not yet available.

Metrics

Metrics Loading ...