DOI:
https://doi.org/10.14483/22487085.22116Published:
2025-07-01Issue:
Vol. 27 No. 1 (2025): CALJ: january-juneSection:
Research ArticlesEvaluating an Online Language Assessment Course
Colombian Teachers’ Voices
El análisis de un curso en línea de evaluación de idiomas
Keywords:
language assessment, language assessment courses, language assessment literacy, teachers’ professional development (en).Keywords:
cursos de evaluación de lenguas, desarrollo profesional docente, evaluación de lenguas, literacidad en evaluación de lenguas (es).Downloads
Abstract (en)
Language assessment literacy (LAL) has emerged as a relatively recent focus in language testing, especially when it comes to fostering this dimension among stakeholders. There are few reports on online assessment courses for English language teachers that discuss their usefulness regarding teachers’ LAL. However, there is a growing call to advance LAL pedagogies and to investigate how assessment courses contribute to teachers’ professional development. In this study, we employed a qualitative research design to ask six Colombian English language teachers to evaluate the contents and activities of an online assessment course specifically designed for them. Data were collected through semi-structured interviews. Our findings indicate that teachers engaged in critical reflection on the course contents, activities, and —considering both what was effective and what was not—by referencing their own assessment practices. Moreover, the findings suggest that course activities, especially quizzes, were instructionally sound, as they promoted a formative assessment approach that, in turn, contributed to development of the teachers’ LAL.
Abstract (es)
La literacidad en la evaluación de lenguas (LEL) es un tema de discusión relativamente reciente en el campo de la evaluación de idiomas. Son escasos los estudios que exploran la utilidad de los cursos de evaluación en línea para el desarrollo de la LEL de docentes de inglés. No obstante, existe un creciente llamado a problematizar las pedagogías de la LEL con el fin de comprender cómo este tipo de cursos contribuye al desarrollo profesional de estos docentes. Este estudio adoptó un diseño cualitativo y convocó a seis profesores de inglés en Colombia para que analizaran los contenidos y actividades de un curso en línea de evaluación diseñado específicamente para ellos. Con este propósito, se utilizaron entrevistas individuales semiestructuradas. Nuestros hallazgos indican que los docentes tuvieron en cuenta sus contextos de evaluación para reflexionar críticamente sobre los contenidos, las actividades y los retos del curso. Además, los hallazgos sugieren que las actividades del curso, en especial los quizzes, fueron pedagógicamente apropiadas, ya que promovieron la evaluación de enfoque formativo, lo que en últimas conllevó al desarrollo de la LEL de estos docentes.
References
Arias, C. I., Maturana, L. M., & Restrepo, M. I. (2012). Evaluación de los aprendizajes en lenguas extranjeras: hacia prácticas justas y democráticas. Lenguaje, 40(1), 99–126. https://doi.org/10.25100/lenguaje.v40i1.4945
Baker, B. (2021, June). Unpacking the concept of language assessment literacy for all key stakeholders [Conference presentation]. New Directions Colombia 2021 https://americas.britishcouncil.org/new-directions/programme
Baker, B. A., & Riches, C. (2018). The development of EFL examinations in Haiti: Collaboration and language assessment literacy development. Language Testing, 35(4), 557–581. https://doi.org/10.1177/0265532217716732
Boyd, E., & Donnarumma, D. (2018). Assessment literacy for teachers: A pilot study
investigating the challenges, benefits and impact of assessment literacy training. In D. Xerri & P. Vella Briffa (Eds.), Teacher involvement in high-stakes language testing (pp. 105–126). Springer. https://doi.org/10.1007/978-3-319-77177-9_7
Brindley, G. (2001). Language assessment and professional development. In C. Elder, A. Brown,
E. Grove, K. Hill, N. Iwashita, T. Lumley, T. McNamara, & K. O’Loughlin (Eds.), Experimenting with uncertainty: Essays in honour of Alan Davies (pp. 126–136). Cambridge University Press.
Butler, A. (2022). Exploring the idea of “assessment with learners” in discussing language assessment literacy. Language Teaching Research Quarterly, 29, 92–101. https://doi:10.32038/ltrq.2022.29.06
Coombe, C., Vafadar, H., Mohebbi, H. (2020). Language assessment literacy: What do we need to learn, unlearn, and relearn? Language Testing in Asia, 10, 1-16. https://doi.org/10.1186/s40468-020-00101-6
Coombs, A. & DeLuca, C. (2022). Mapping the constellation of assessment discourses: a
scoping review study on assessment competence, literacy, capability, and identity. Educational Assessment, Evaluation and Accountability, 34, 279–301. https://doi.org/10.1007/s11092-022-09389-9
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. SAGE Publications.
Creswell, J. W., & Poth, C. (2018). Qualitative inquiry and research design: Choosing among
five approaches. SAGE Publications.
Csépes I. (2021). The evolving concept of (language) assessment literacy: Implications for
teacher education. Central European Journal of Educational Research, 3(1), 120–130. https://doi.org/10.37441/CEJER/2021/3/1/9360
Cui, X., Liu, Y., Yu. H., & Gao, Y. (2022). Developing English teachers’ language assessment
literacy in an EAP reform context through test design: A case study. System, 109, 1-12. https://doi.org/10.1016/j.system.2022.102866
Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327–
https://doi.org/10.1177/0265532208090156
Cote-Parra, G., & López, A. A. (2024). Examining the assessment practices of foreign
language novice teachers. Profile: Issues in Teachers’ Professional Development, 26(1), 97–113. https://doi.org/10.15446/profile.v26n1.106384
DeLuca, C., Searle, M., Carbone, K., Ge, J., & LaPointe-McEwan, D. (2021). Toward a
pedagogy for slow and significant learning about assessment in teacher education. Teaching and Teacher Education, 101, 1-12. https://doi.org/10.1016/j.tate.2021.103316
Deygers, B. (2019). Fairness and social justice in English language assessment. In X. Gao (Ed.),
Second handbook of English language teaching (pp. 541-570). Springer.
Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment
Quarterly, 9(2), 113-132. https://doi.org/10.1080/15434303.2011.642041
Fulcher, G., Panahi, A., Mohebbi, H.3 (2022). Glenn Fulcher’s thirty-five years of contribution to language testing and assessment: A systematic review. Language Teaching Research Quarterly, 29, 20–56 https://doi:10.32038/ltrq.2022.29.03
Giraldo, F. (2018). Language assessment literacy: Implications for language teachers. Profile: Issues in Teachers’ Professional Development, 20(1), 179-195. https://doi.org/10.15446/profile.v20n1.62089
Giraldo, F., & Murcia, D. (2019). Language assessment literacy and the professional development of pre-service language teachers. Colombian Applied Linguistics Journal, 21(2), 243–259. https://doi.org/10.14483/22487085.14514
Giraldo, F. (2023). Fostering pre-service teachers' language assessment literacy. Editorial
Universidad de Caldas.
Giraldo, F. (2024). Language assessment literacy for in-service teachers: Online course
planning and implementation. Editorial Universidad de Caldas.
Giraldo, F. (in press). Designing an assessment course for in-service English language teachers
in Colombia. Folios, 62.
González, E. F. (2021). The impact of assessment training on EFL writing classroom assessment:
Voices of Mexican university teachers. Profile: Issues in Teachers’ Professional Development, 23(1), 107–124. https://doi.org/10.15446/profile.v23n1.85019
Herrera, L., & Macías, D. (2015). A call for language assessment literacy in the education
and development of teachers of English as a foreign language. Colombian Applied
Linguistics Journal, 17(2), 302-312. https://doi.org/10.14483/udistrital.jour.calj.2015.2.a09
Hill, K. (2017). Understanding classroom-based assessment practices: A precondition for teacher
assessment literacy. Papers in Language Testing and Assessment, 6(1), 1–17.
Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on
language assessment courses. Language Testing, 25(3), 385–402. https://doi.org/10.1177/0265532208090158
Inbar-Lourie, O. (2012). Language assessment literacy. In C. Chapelle (Ed.), The encyclopedia
of applied linguistics (pp. 1-9). John Wiley & Sons. https://doi.org/10.1002/9781405198431.wbeal0605
Inbar-Lourie, O. (2017). Language assessment literacy. In E. Shohamy, I. G. Or, & S. May
(Eds.), Language testing and assessment: Encyclopedia of language and education (3rd ed., pp. 257–268). Springer. https://doi.org/10.1007/978-3-319-02261-1_19
Kleinsasser, R. C. (2005). Transforming a postgraduate level assessment course: A second
language teacher educator’s narrative. Prospect, 20(3), 77–102.
Koh, K., Burke, L., Luke, A., Gong, W., & Tan, C. (2018). Developing the assessment literacy
of teachers in Chinese language classrooms: A focus on assessment task design. Language Teaching Research, 22(3), 264–288. https://doi.org/10.1177/1362168816684366
Kremmel, B., Eberharter, K., Holzknecht, F, & Konrad, E. (2018). Fostering language
assessment literacy through teacher involvement in high-stakes test development. In D. Xerri & P. Vella Briffa (Eds.), Teacher involvement in high-stakes language testing (pp. 173–194). Springer. https://doi.org/10.1007/978-3-319-77177-9_10
Kremmel, B., & Harding, L. (2020). Towards a comprehensive, empirical model of language
assessment literacy across stakeholder groups: Developing the language assessment literacy survey. Language Assessment Quarterly, 17(1), 100–120. https://doi.org/10.1080/15434303.2019.1674855
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310
Lao, H. & Yan, Z. (2024) Using lesson study to promote teacher assessment literacy. In E. Cheng, & B. Lander (Eds.), Implementing a 21st century competency-based curriculum through lesson study. (pp. 42- 55). Routledge.
Levi, T., & Inbar-Lourie, O. (2019). Assessment literacy or language assessment literacy:
Learning from the teachers. Language Assessment Quarterly, 17(2), 168–182. https://doi.org/10.1080/15434303.2019.1692347
Lin, L., Ying, G., & Guo, X. (2024). Pre-service Chinese language teachers’ conceptions of assessment: A person-centered perspective. Language Teaching Research, 28(1), 273–295. https://doi.org/10.1177/1362168821996529
Lukin, L. E., Bandalos, D. L., Eckhout, T. J., & Mickelson, K. (2004). Facilitating the
development of assessment literacy. Educational Measurement: Issues and Practice, 23(2), 26–32. https://doi.org/10.1111/j.1745-3992.2004.tb00156.x
Nayernia, A., & Mohebbi, H., (2023). Teachers’ online language assessment literacy. In Z.
Tajeddin, & C. Griffiths (Eds.), Language education programs: Perspectives on policies and practices. (pp. 122 -140). Springer. https://doi.org/10.1007/978-3-031-38754-8
Malone, M. E. (2017). Training in language assessment. In E. Shohamy, I. G. Or, & S. May
(Eds.), Language testing and assessment: Encyclopedia of language and education (3rd ed., pp. 225–240). Springer. https://doi.org/10.1007/978-3-319-02261-1_16
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). SAGE
Publications.
Mohammadkhah, E., Reza Kiany, G., Tajeddin, Z., ShayesteFar, P. (2022). EFL teachers’ assessment literacy: A contextualized measure of assessment theories and skills. Language Teaching Research Quarterly, 29, 102–119. https://doi:10.32038/ltrq.2022.29.07
Montee, M., Bach, A., Donovan, A., & Thompson, L. (2013). LCTL teachers’ assessment
knowledge and practices: An exploratory study. Journal of the National Council of Less Commonly Taught Languages, 13, 1–31.
O’Loughlin, K. (2006). Learning about second language assessment: Insights from a
postgraduate student on-line subject forum. University of Sydney Papers in TESOL, 1, 71–85.
Pastore, S. (2023). Teacher assessment literacy: a systematic review. Frontiers in Education, 8.
https://doi.org/10.3389/feduc.2023.1217167
Randall, J., Poe, M., Slomp, D., & Oliveri, M. (2023). Our validity looks like justice. Does
yours? Language Testing, 41(1) 203–219. https://doi.org/10.1177/02655322231202947
Restrepo-Bolívar, E. M. (2020). Monitoring preservice teachers’ language assessment literacy
development through journal writing. Malaysian Journal of ELT Research, 17(1), 38–52.
Rezai, A. (2024). The role of teacher assessment literacy in job stress and job burnout in EFL contexts: a mixed-methods investigation. Asian-Pacific Journal of Second and Foreign Language Education, 9, 1-21. https://doi.org/10.1186/s40862-023-00225-1
Riazi, A. M. (2016). The Routledge encyclopedia of research methods in applied linguistics:
Quantitative, qualitative, and mixed-methods research. Routledge.
Saldaña, J. (2016). The coding manual for qualitative researchers. SAGE Publications.
Scarino, A. (2013). Language assessment literacy as self- awareness: Understanding the role of
interpretation in assessment and in teacher learning. Language Testing, 30(3), 309–327. https://doi.org/10.1177/0265532213480128
Scarino, A. (2017). Developing assessment literacy of teachers of languages: A conceptual and
interpretive challenge. Papers in Language Testing and Assessment, 6(1), 18-40.
Stabler-Havener, M. L. (2018). Defining, conceptualizing, problematizing, and assessing
language teacher assessment literacy. Working Papers in Applied Linguistics & TESOL, 18(1), 1–22.
Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238–
Taylor, L. (2013). Communicating the theory, practice and principles of language testing to test
stakeholders: Some reflections. Language Testing, 30(3), 403–412. https://doi.org/10.1177/0265532213480338
Vogt, K., & Tsagari, D. (2014). Assessment literacy of foreign language teachers: Findings of a
European study. Language Assessment Quarterly, 11(4), 374–402. https://doi.org/10.1080/15434303.2014.960046
Xu, Y., & Brown, G. (2016). Teacher assessment literacy in practice: A reconceptualization.
Teaching and Teacher Education, 58, 149-162. https://doi.org/10.1016/j.tate.2016.05.010
Yan, X. (2021, June 14-17). Towards a conceptual model of LAL development across
stakeholder groups and contexts [Conference presentation]. Language Testing Research
Colloquium, online.
Yan, X., Zhang, C., & Fan, J. (2018). “Assessment knowledge is important, but ...”: How
Contextual and experiential factors mediate assessment practice and training needs of language teachers. System, 74, 158-168. https://doi.org/10.1016/j.system.2018.03.003
Yan, X., & Fan, J. (2021). “Am I qualified to be a language tester?”: Understanding the
development of language assessment literacy across three stakeholder groups. Language
Testing, 38(2), 219–246. https://doi.org/10.1177/0265532220929924
Zu, X & Evans, C., (2024). Enhancing the development and understanding of assessment
literacy in higher education. European Journal of Higher Education, 14(1), 80-100. https://doi.org/10.1080/21568235.2022.2118149
How to Cite
APA
ACM
ACS
ABNT
Chicago
Harvard
IEEE
MLA
Turabian
Vancouver
Download Citation
Metrics
License
Copyright (c) 2025 Frank Giraldo, Lenny García
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
The journal allow the author(s) to hold the copyright without restrictions. Also, The Colombian Apllied Linguistics Journal will allow the author(s) to retain publishing rights without restrictions.
