Establishing placement test fit and performance: serving local needs

Establecimiento de ajuste y desempeño en exámenes de clasificación de acuerdo con necesidades locales

Authors

  • Gerriet Janssens Department of Second Language Studies University of Hawai’i, Mãnoa
  • Valerie Meier Department of Second Language Studies University of Hawai’i, Mãnoa

Keywords:

Programas de lenguas, análisis de exámenes (es).

Keywords:

Language programs, tests analysis. (en).

Downloads

Abstract (en)

Accountability in language education is often associated with top-down national policies unresponsive—or even hostile to—local needs; however, when accountability is driven by local stakeholders seeking to better understand and enhance their programs, it can foster productive cycles of action research and curriculum development. This paper reports on one such internally-motivated accountability effort, in which program insiders sought to determine the efficacy of a reading test being administered to a new population of students at one Colombian university. Descriptive statistics, reliability estimates, and item facility and discrimination measures were used to determine whether this test was sufficiently reliable and appropriately matched to test takers’ ability in order to warrant its use as part of a high-stakes English-language placement exam. A detailed analysis of this test is used not only to propose specific recommendations for revision but also to illustrate a useful set of statistical tools appropriate for test analysis in other language programs. Moreover, we conclude that the involvement of local instructors as part of an iterative, self-reflective, test development process provides opportunities for professional development and deeper engagement in accountability projects.

Abstract (es)

Frecuentemente se asocia el concepto de accountability en la enseñanza de lenguas con políticas de estado impositivas que pueden ser indiferentes ante (o ser hostiles a) las necesidades de la comunidad académica local. Sin embargo, cuando los proyectos de accountability son impulsados por la misma comunidad académica—buscando dentro sus programas curriculares un mejor conocimiento de si mismos—estos proyectos pueden fomentar ciclos productivos de investigación-acción y de desarrollo curricular. Este ensayo presenta parte de los resultados de una investigación en accountability motivada internamente dentro de un programa de lenguas en una universidad colombiana; como parte de ésta, los involucrados locales querían determinar en qué medida la prueba de lectura establecida por la universidad era apropiada para una nueva población de estudiantes. Se calcularon las estadísticas descriptivas, estimaciones de confiabilidad y medidas de item facility y item discrimination para determinar si el examen fue lo suficientemente confiable y si se adecuaba bien a las habilidades de los estudiantes para justificar el uso de este examen como parte de una prueba de clasificación de alto impacto. Se usó un análisis detallado de este examen para sugerir revisiones específicas y para ilustrar el uso de un grupo de herramientas estadísticas para el análisis de exámenes que pueden ser usadas dentro de otros programas de lenguas. Concluimos que la participación de los profesores locales dentro de un proceso de desarrollo cíclico y auto-reflexivo les da oportunidades para su desarrollo profesional y un nivel de compromiso más profundo durante proyectos de accountability.

Author Biographies

Gerriet Janssens, Department of Second Language Studies University of Hawai’i, Mãnoa

After completing his MA in Applied Linguistics and TESL (UCLA, 2005), he worked for six years at Universidad
de los Andes, which strongly supported his growing interest in curriculum development. Currently, he is a PhD candidate at University
of Hawai‘i, Mãnoa's Department of Second Language Studies, where he is pursuing his research interests in exam and program validity,
language test development, corpus linguistics, and quantitative research methodologies. He intends to return to Colombia and Los Andes
in order to finish his dissertation research and teach.

Valerie Meier, Department of Second Language Studies University of Hawai’i, Mãnoa

After completing an MA in Composition (SFSU, 2003) she taught a variety of academic writing and reading classes at
US universities and business English in Asia. She has just finished an MA program in Second Language Studies (2013) at the University
of Hawaiʻi, Mānoa, where she further developed her interests in academic literacies, EAP program development and evaluation, computermediated
communication, and language assessment.

References

Brown, J.D. (1995). The elements of language curriculum: A systematic approach. Boston, MA: Heinle

ELT.

Brown, J.D. (2005). Testing in language programs:

A comprehensive guide to English language assessment. New York: McGraw-Hill.

Bush, G.W. (2002).Public Law 107-110. An act to close

the achievement gap with accountability, flexibility,

and choice, so that no child is left behind. U.S.

Congress.

Cárdenas, M. L. (2006). Bilingual Colombia: Are we ready for it? What is needed? Paper presented at 19th

Annual EA Education Conference 2006. Retrieved

from http://www.englishaustralia.com.au/ea_conference2006/proceedings/pdf/Cardenas.pdf

Clavijo, A. (2004). Critical perspectives about bilingualism in the city of Bogota. Paper presented at

primer simposio internacional sobre bilingüismo y

educación bilingüe en Latinoamérica (Bilinglatam).

Buenos Aires: Argentina. Retrieved from http://www.

essarp.org.ar/bilinglatam/ingles/clavijo-olarte.html

Dick, W., Carey, L., & Carey, J.O. (2000). The systematic design of instruction, 5th edition. Boston, MA: Pearson.

Ebel, R.L. (1979). Essentials of educational measurement, 1st edition. Upper Saddle River, NJ: Prentice Hall.

Ebel, R.L., & Frisbie, D.A. (1991). Essentials of educational measurement, 5th edition. Upper Saddle River, NJ: Prentice Hall.

González, A. (2008). Bilingüismo en Colombia perspectivas globales y locales. Ministerio de Educación Nacional, Colombia. Retrieved from: http://www.mineducacion.gov.co/observatorio/1722/article-170864.html

Hudson, T. (in press). Research design and analysis

in Applied Linguistics (draft manuscript, cited with

author’s permission). University of Hawai‘i: Honolulu,

HI.

Kemmis, S., & McTaggart, R. (1988). The Action Research Planner (3rd Ed.). Geelong: Deakin University Press.

López, A., & Janssen, G. (2010). Validation study for

Colombia’s ECAES exam. Revista Lenguaje, 38(2),

–448.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.

Tabachnick, B., & Fidell, L. (2013). Using multivariate

statistics, 6th edition. Boston, MA: Pearson.

Truscott de Mejía, A. (2006). Bilingual Education in

Colombia: Towards a recognition of Languages, cultures and identities. Colombian Applied Linguistics

Journal, 8, 152–168.

Truscott de Mejía, A. M., Ordóñez, C. L., & Fonseca,

L. Universidad de los Andes, (2006). Estudio investigativo sobre el estado actual de la educación

bilingüe (inglés-español) en Colombia. Retrieved

from: ttp://cife.uniandes.edu.co/archivos/Lineamientos_

para_la_educacion_bilingue.pdf

Truscott de Mejía, A., López Mendoza, A., & Peña Dix,

B. (2011). Bilingüismo en el contexto colombiano:

Iniciativas y perspectivas en el siglo XXI. Bogotá,

Colombia: Ediciones Uniandes.

Usma, J. (2009). Globalization and language and education reform in Colombia: A critical outlook. Íkala,

revista de lenguaje y cultura 14(20), 19-42.

How to Cite

APA

Janssens, G., & Meier, V. (2013). Establishing placement test fit and performance: serving local needs. Colombian Applied Linguistics Journal, 15(1), 100–112. https://doi.org/10.14483/udistrital.jour.calj.2013.1.a07

ACM

[1]
Janssens, G. and Meier, V. 2013. Establishing placement test fit and performance: serving local needs. Colombian Applied Linguistics Journal. 15, 1 (Jan. 2013), 100–112. DOI:https://doi.org/10.14483/udistrital.jour.calj.2013.1.a07.

ACS

(1)
Janssens, G.; Meier, V. Establishing placement test fit and performance: serving local needs. Colomb. appl. linguist. j 2013, 15, 100-112.

ABNT

JANSSENS, G.; MEIER, V. Establishing placement test fit and performance: serving local needs. Colombian Applied Linguistics Journal, [S. l.], v. 15, n. 1, p. 100–112, 2013. DOI: 10.14483/udistrital.jour.calj.2013.1.a07. Disponível em: https://revistas.udistrital.edu.co/index.php/calj/article/view/4253. Acesso em: 20 may. 2022.

Chicago

Janssens, Gerriet, and Valerie Meier. 2013. “Establishing placement test fit and performance: serving local needs”. Colombian Applied Linguistics Journal 15 (1):100-112. https://doi.org/10.14483/udistrital.jour.calj.2013.1.a07.

Harvard

Janssens, G. and Meier, V. (2013) “Establishing placement test fit and performance: serving local needs”, Colombian Applied Linguistics Journal, 15(1), pp. 100–112. doi: 10.14483/udistrital.jour.calj.2013.1.a07.

IEEE

[1]
G. Janssens and V. Meier, “Establishing placement test fit and performance: serving local needs”, Colomb. appl. linguist. j, vol. 15, no. 1, pp. 100–112, Jan. 2013.

MLA

Janssens, G., and V. Meier. “Establishing placement test fit and performance: serving local needs”. Colombian Applied Linguistics Journal, vol. 15, no. 1, Jan. 2013, pp. 100-12, doi:10.14483/udistrital.jour.calj.2013.1.a07.

Turabian

Janssens, Gerriet, and Valerie Meier. “Establishing placement test fit and performance: serving local needs”. Colombian Applied Linguistics Journal 15, no. 1 (January 1, 2013): 100–112. Accessed May 20, 2022. https://revistas.udistrital.edu.co/index.php/calj/article/view/4253.

Vancouver

1.
Janssens G, Meier V. Establishing placement test fit and performance: serving local needs. Colomb. appl. linguist. j [Internet]. 2013Jan.1 [cited 2022May20];15(1):100-12. Available from: https://revistas.udistrital.edu.co/index.php/calj/article/view/4253

Download Citation

Visitas

429

Dimensions


PlumX


Downloads

Download data is not yet available.

Metrics

Metrics Loading ...