Improving Students’ Entry Profiles: Exploring English Ability Level, Affective Variables and Academic Performance in a Chilean TEFL Program.
Una exploración de los perfiles de ingreso de los estudiantes: nivel de dominio del inglés, variables afectivas y desempeño académico en un programa de formación de profesores de inglés en Chile
Keywords:éxito académico, evaluación diagnóstica, perfiles de ingreso de estudiantes de pedagogía en inglés, formación de profesores de inglés. (es).
Keywords:academic success, diagnostic assessment, English pedagogy entry profiles, TEFL teacher education (en).
Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
Alderson, J. C. (2005). Diagnosing foreign language proficiency. Continuum International Publishing Group.
Alderson, J. C., & Huhta, A. (2011). Can research into the diagnostic testing of reading in a second or foreign language contribute to SLA research? EUROSLA Yearbook, 11, 30-52. https://doi.org/10.1075/eurosla.11.04ald DOI: https://doi.org/10.1075/eurosla.11.04ald
Alderson, J. C., Brunfaut, T., & Harding, L. (2015). Towards a theory of diagnosis in second and foreign language assessment: Insights from professional practice across diverse fields. Applied Linguistics, 36(2), 236-260. https://doi.org/10.1093/applin/amt046 DOI: https://doi.org/10.1093/applin/amt046
Alderson, J. C., Haapakangas, E. L., Huhta, A., Nieminen, L., & Ullakonoja, R. (2015). The diagnosis of reading in a second and foreign language. Routledge. DOI: https://doi.org/10.4324/9780203071830
Alonso, F. & Ramos-Gálvez, C. (2018). Transición a la vida universitaria: programa de tutorías para estudiantes de primer año de pedagogía en inglés. Cuadernos de Educación, 80. Revista Virtual de la Facultad de Educación Universidad Alberto Hurtado.
Alrabai, F., & Moskovsky, C. (2016). The relationship between learners' affective variables and second language achievement. Arab World English Journal, 7(2), 77-103. http://dx.doi.org/10.2139/ssrn.2814796 DOI: https://doi.org/10.24093/awej/vol7no2.6
Law 20903 of 2016. Crea el sistema de desarrollo profesional docente y modifica otras normas. April 1st, 2016. http://bcn.cl/1uzzn
Bernaus, M., & Gardner, R. C. (2008). Teacher motivation strategies, student perceptions, student motivation, and English achievement. The Modern Language Journal (Boulder, Colo.), 92(3), 387-401. https://doi.org/10.1111/j.1540-4781.2008.00753.x DOI: https://doi.org/10.1111/j.1540-4781.2008.00753.x
Boo, Z., Dörnyei, Z., & Ryan, S. (2015). L2 motivation research 2005-2014: understanding a publication surge and a changing landscape. System, 55, 145-157. https://doi.org/10.1016/j.system.2015.10.006 DOI: https://doi.org/10.1016/j.system.2015.10.006
Bridgeman, B., Cho, Y., & DiPietro, S. (2016). Predicting grades from an English language assessment: the importance of peeling the onion. Language Testing, 33(3), 307-318. https://doi.org/10.1177/0265532215583066 DOI: https://doi.org/10.1177/0265532215583066
Cotton, F., & Conrow, F. (1998). An investigation of the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania. https://www.ielts.org/-/media/research-reports/ielts_rr_volume01_report4.ashx
Council of Europe. (2001). Common European framework of reference for languages: learning, teaching, assessment. Press Syndicate of the University of Cambridge.
Creswell, J., & Plano, V. (2017). Designing and conducting mixed methods research (3rd ed.). Sage.
Csizér, K., & J. Kormos. (2009). Learning experiences, selves and motivated learning behaviour: a comparative analysis of structural models for Hungarian secondary and university learners of English. In Z. Dörnyei & E. Ushioda (Eds.), Motivation, language identity and the L2 self (pp. 98–119). Multilingual Matters. DOI: https://doi.org/10.21832/9781847691293-006
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. Plenum. DOI: https://doi.org/10.1007/978-1-4899-2271-7
Dörnyei, Z. (2005). The psychology of the language learner. Routledge.
Dörnyei, Z. (2007). Research methods in applied linguistics: quantitative, qualitative and mixed methodologies. Oxford University Press.
Dörnyei, Z., & Ryan, S. (2015). Motivation. In Z. Dörnyei & S. Ryan (Eds.), The psychology of the language learner revisited (pp. 72-105). Routledge. DOI: https://doi.org/10.4324/9781410613349
Gardner, R. C., & Lambert, W. E. (1959). Motivational variables in second-language acquisition. Canadian Journal of Psychology/Revue canadienne de psychologie, 13(4), 266-272. https://doi.org/10.1037/h0083787 DOI: https://doi.org/10.1037/h0083787
Giaconi, V., Varas, L., Ravest, J., Martin, A., Gómez, G., Queupil, J. P., & Díaz, K. (2019). Fortaleciendo la formación inicial docente: experiencia universitaria en la implementación de la evaluación diagnóstica inicial para pedagogías. Proyecto FONIDE 170009. MINEDUC. https://demre.cl/estudios/documentos/2019-07-207-resumen-ejecutivo-final-FON1700009.pdf
González, E. (2015). Factores de contexto socioeconómicos y educativos en estudiantes de nivel superior, sugerencia para una realidad actual. Revista Interamericana de Psicología, 49(3), 399-412. https://doi.org/10.30849/rip/ijp.v49i3.10
Ingram, D., & Bayliss, A. (2007). IELTS as a predictor of academic language performance, Part 1. https://www.ielts.org/-/media/research-reports/ielts_rr_volume07_report3.ashx
Lee, Y., & Greene, J. (2007). The predictive validity of an ESL placement test. Journal of Mixed Methods Research, 1(4), 366-389. https://doi.org/10.1177/1558689807306148 DOI: https://doi.org/10.1177/1558689807306148
Hadfield, J. & Dörnyei, Z. (2014). Motivating Learning. Routledge. DOI: https://doi.org/10.4324/9781315833286
Harding, L., Alderson, J. C., & Brunfaut, T. (2015). Diagnostic assessment of reading and listening in a second or foreign language: Elaborating on diagnostic principles. Language Testing, 32(3), 317-336. https://doi.org/10.1177/0265532214564505 DOI: https://doi.org/10.1177/0265532214564505
Iwaniec, J. & Dunn, K. (2020). Measuring Motivation. In P. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 157-166). Taylor & Francis Group. DOI: https://doi.org/10.4324/9781351034784-18
Moskovsky, C., Racheva, S., Assulaimani, T., & Harkins, J. (2016). The L2 motivational self system and L2 achievement: a study of Saudi EFL learners. The Modern Language Journal, 16, 1-14. https://doi.org/10.1111/modl.12340 DOI: https://doi.org/10.1037/t65166-000
Pollitt, A. (1988). Predictive validity. In A. Hughes, D. Porter, & C. Weir (Eds.) ELTS research report 1 (ii). ELTS validation project: Proceedings of a conference held to consider the ELTS validation project report. The British Council/UCLES.
Saito, K., Dewaele, J., Abe, M., & In’nami, Y. (2019). Motivation, emotion, learning experience and second language comprehensibility development in classroom settings: a cross-sectional and longitudinal study. Language Learning, 68(3), 1-35. https://doi.org/10.1111/lang.12297 DOI: https://doi.org/10.1111/lang.12297
Taguchi, T., Magid, M., & Papi, M. (2009). The L2 motivational self-system among Japanese, Chinese and Iranian learners of English: A comparative study. In Z. Dörnyei & E. Ushioda (Eds.), Motivation, language identity and the L2 self (pp. 66-98). Multilingual Matters. DOI: https://doi.org/10.21832/9781847691293-005
Teimouri, Y., Plonsky, L., & Tabandeh, F. (2020). L2 grit: Passion and perseverance for second-language learning. Language Teaching Research, 1362168820921895. https://doi.org/10.1177/1362168820921895 DOI: https://doi.org/10.1177/1362168820921895
Weir, C. (2005). Language testing and validation: An evidence-based approach. Palgrave Macmillan. DOI: https://doi.org/10.1057/9780230514577
Weir, C. J., Chan, S. H. C., & Nakatsuhara, F. (2013). LTTC-GEPT Research Report: Examining the criterion-related validity of the GEPT advanced reading and writing tests: comparing GEPT with IELTS and real-life academic performance. https://www.lttc.ntu.edu.tw/lttc-gept-grants/RReport/RG01.pdf
How to Cite
Recibido: 7 de marzo de 2022; Aceptado: 11 de julio de 2022
Having valuable information about newly enrolled university students in the form of entry profiles is key to addressing their academic needs as they advance through their educational trajectories. In Chile, this is acknowledged by Law 20903, which mandates that all tertiary-level pedagogy programs implement a diagnostic assessment at the beginning of the studies. This, in order to even out students’ academic level and establish supportive measures for those who obtain low results. Aiming to enrich the entry profile of a group of Chilean English language teaching (TEFL) undergraduates, this research explores how factors such as English proficiency level, class attendance, and motivation influence their academic performance during the first semester of their university studies. Through the analysis of quantitative sources of data, the results suggest that class attendance and scores on an institutional English entry test provide valuable information about students’ performance but do not fully explain their academic success. Qualitative data from teacher interviews suggest that profiling students also involves looking into factors such as motivation. Findings should spark reflection on the need for more detailed entry profiles and comprehensive diagnostic assessments that transcend the assessment of English skills and linguistic knowledge in order to ensure better learning trajectories for university students.
Keywordsacademic success, diagnostic assessment, English pedagogy entry profiles, English teacher education, TEFL.
Contar con información valiosa sobre los perfiles de ingreso de estudiantes universitarios es clave para abordar sus necesidades académicas a medida que avanzan en su trayectoria formativa. En Chile, esta importancia es reconocida por la Ley 20903, que exige que los programas de pedagogía de nivel terciario apliquen una prueba diagnóstica al inicio de los estudios. Esto, en aras de emparejar el nivel académico de los estudiantes y establecer medidas de apoyo para quienes obtengan bajos resultados. Buscando enriquecer el perfil de ingreso de un grupo de estudiantes de pregrado de en pedagogía en inglés (TEFL), este estudio explora la manera en que factores como el nivel de dominio del inglés, la asistencia a clases y la motivación influyen en su desempeño académico durante el primer semestre de sus estudios. A través del análisis de fuentes cuantitativas de datos, los resultados sugieren que la asistencia a clases y los resultados obtenidos en la prueba diagnóstica institucional suministran información valiosa sobre el desempeño de los estudiantes pero no explican del todo su éxito académico. Los datos cualitativos de entrevistas con profesores sugieren que perfilar a los estudiantes incluye una mirada a factores como la motivación. Los hallazgos deberían suscitar una reflexión sobre la necesidad de contar con perfiles de ingreso más detallados y pruebas diagnósticas integrales que trasciendan la evaluación de aspectos lingüísticos para asegurar mejores trayectorias formativas en estudiantes universitarios.
Palabras claveéxito académico, evaluación diagnóstica, perfiles de ingreso de estudiantes de pedagogía en inglés, formación de profesores de inglés, TEFL.
The relevance of creating detailed and informative student entry profiles of university undergraduates cannot be understated. Knowing about both students’ academic and socio-affective learning needs can lead to better learning trajectories and more meaningful teaching practices. This is in fact acknowledged by Chilean Law 20903 of 2016, which establishes that, in order to inform curricular and instructional decisions, university pedagogy programs are required to diagnose all first-year undergraduates. However, although universities seem to value having an initial student profile, there is no clarity about how diagnostic practices actually take place, and less is known about whether they lead to any plan adjustments, mentoring programs, and/or leveling courses for students ( Giaconi et al ., 2019 ). In fact, Giaconi et al. (2019) report that current diagnostic practices across five Chilean universities have unclear purposes and important weaknesses in terms of design, development, validation, and use. This is in part explained by the law itself, which requires an initial diagnostic assessment but does not stipulate any parameters on how universities should proceed in order to comply with this requirement or with a diagnostic follow-up.
When it comes to creating profiles in subjectspecific disciplines such as the teaching of English as a foreign language (TEFL), the state of affairs might look grimmer. After all, this implies that pedagogy programs, where students acquire subject-of-teaching knowledge as well as teaching skills, would benefit from looking into diagnostic elements that contribute to both domains. For instance, in Chile, qualifying as an English teacher generally requires the successful completion of an undergraduate program which is eight to ten semesters long and combines English as a foreign language courses to develop students’ proficiency level and pedagogy-related courses. One challenge, however, is that English as a second or foreign language lacks a unifying diagnosis theory, and truly diagnostic instruments are almost non-existent ( Alderson, 2005 ). In turn, this posits problems for universities and students alike. On the one hand, Chilean universities in general and pedagogy programs in particular are required to ensure quality training while maintaining low student-attrition rates. On the other hand, undergraduates who fail courses face high consequences such as delays in their studies, an increase in their student debt and, in some cases, academic desertion. However, the current landscape is one where universities resort, at best, to the use of institutional placement tests or other outsourced instruments such as international standardized exams, with no local studies reporting on whether these are used in any informative way. Therefore, initial university-led assessments are at risk of becoming a ritualized administrative procedure instead of an opportunity for diagnosis, whereby students’ weaknesses and strengths are identified to offer them better learning opportunities. In this current scenario, providing a more detailed characterization of TEFL undergraduates’ needs is imperative, either in the form of principled diagnostic practices ( Alderson et al., 2015 ) or by using readily available resources (e.g., institutional placement tests, teacher educators’ expert judgements).
In the specific case of English pedagogy students, profiling their English needs is of paramount importance, as developing proficiency in English constitutes the cornerstone of their training and contributes to their future performance as teachers, as well as to students’ more immediate performance in core English-language subjects (which usually carry the greatest consequences) and in TEFL-related courses (specialization and pedagogy), some of which are also taught in English. In addition to English abilities, a great part of what TEFL undergraduates have to do as part of their learning, in-class participation, and future professional performance involves displaying communicational, personal, attitudinal, and motivational abilities. In fact, the direct relationship between successful language learning and motivation is well established ( Dörnyei, 2005. Dörnyei and Ryan, 2015. Lamb, 2017. Iwaniec and Dunn, 2020 ). Therefore, a better understanding of how these attributes affect students’ learning might also be desirable. Similarly, other factors such as personal commitment to studying, a positive learning environment, and a good relationship between teachers and students have also shown to be key for learning ( González, 2015 ), so this study considered these aspects in addition to the abovementioned variables.
Directly addressing the call for a better understanding of the needs of newly enrolled TEFL undergraduates, this study explored the extent to which students’ English proficiency, expressed in scores on an institutional English entry test and class attendance, relate to their academic performance in the English language course during the first semester of studies at one Chilean university. In addition to quantitative measures, the researchers also sought to understand what teacher educators consider to be relevant factors for student performance (e.g., motivation, risk taking, class participation, class attendance).
To the best of the researchers’ knowledge, this is the first attempt to use available resources to better understand and profile the needs of first-year TEFL undergraduates in Chilean universities.
Students’ motivation and foreign language academic success
Ever since the 1950s, motivation and its relation to language learning has been thoroughly studied (Gardner and Lambert, 1959), and its importance is nowadays well-established. In fact, in certain situations, “motivation to learn a language is as powerful, if not more powerful, than many or most cognitively based individual differences” (Iwaniec and Dunn, 2020, p. 157), and learners with sufficient motivation normally achieve working L2 proficiency regardless of their language aptitude (Hadfield and Dörnyei, 2014).
Researchers usually address motivation from Deci and Ryan’s (1985) Self-determination Theory or the L2 Motivational Self System (L2MSS) (Dörnyei, 2005), with the latter being the most heavily researched (Boo et al., 2015). For instance, Taguchi et al. (2009) found that the motivational role of the ought-to L2 self was limited in comparison to the ideal L2 self, while Csizér and Kormos (2009) show its role to be rather questionable. More recent studies focused on how L2 selves and language learning experience are associated with language proficiency gains have also led to inconclusive and somewhat contradictory results. For example, Moskovsky et al. (2016) found that, although L2MSS components effectively predicted students’ efforts, these components were not consistently correlated with L2 achievement. Meanwhile, Saito et al. (2019) concluded that students’ weaker ideal L2 selves were negatively related to their performance. Considering these discrepancies, Iwaniec and Dunn (2020) recommend using a wider variety of research methods to address the link between motivation and language abilities in a wider variety of contexts.
Other affective variables also known to be key for L2 learning are students’ attitude, autonomy, and self-esteem (Alrabai and Moskovsky, 2016). In one study, Bernaus and Gardner (2008) stated that motivational and affective variables such as anxiety and attitudes towards the learning situation were an important predictor of English achievement. A more recent study (Teimouri et al., 2020) explored how L2 grit (a combination of perseverance and passion for long-term goals) relates to students’ language learning motivation and achievement, finding a positive correlation between these and proposing that L2 grit is an important factor for L2 development. In this study, Teimouri et al. (2020) measured students’ grit in L2 settings by means of a language domain-specific grit scale, finding that gritty students had a persistent attitude towards L2 learning, paid more attention to class activities (involving teachers and classmates), felt good about using the target language, and enjoyed their language learning experience (Teimouri et al., 2020).
In conclusion, research suggests that motivation should be part of those features that profiles or diagnoses must take into consideration. This is why it is surprising that widely used diagnostic systems do not address motivation-related components (e.g., DIALANG, https://dialangweb.lancaster.ac.uk/; the Diagnostic English Language Needs Assessment, http://www.delna.auckland.ac.nz/; and The Diagnostic English Language Tracking Assessment, http://gslpa.polyu.edu.hk/eng/delta_web/).
English proficiency and academic performance
Given the importance of English proficiency in profiling the needs of TEFL undergraduates, a brief account of studies looking into its impact on academic success is provided herein. Several studies (e.g., Ingram and Bayliss, 2007. Lee and Greene, 2007. Bridgeman et al., 2016. Weir et al., 2013) have looked into the relationship between English proficiency test scores and other measures reflecting the construct or domain of interest (e.g., criterion validation studies) taken in conjunction with or after the test has been administered (Alderson et al., 1995). These measures are usually scores from other tests, students’ self-assessments, or teacher ratings. However, such studies rarely lead to consistent observations, mainly due to a lack of control of variables (Ingram and Bayliss, 2007) and the complexity of the construct of interest (e.g., academic performance). For instance, Lee and Greene (2007) reported that the “overall correlation between CEEPT score (the institutional ESL placement test) and student GPA was essentially zero” (p. 373), with statistically significant albeit weak correlations between scores and faculty evaluations (r = 0.14), as well as between scores and student self-assessments (r = 0.34). On the other hand, Weir et al. (2013) reported correlations between participants’ average IELTS reading and writing bands and their performance in four real-life tasks at r = 0.60 (p < 0.01). When taken separately, correlations stood at r = 0.55 (p < 0.01) for the reading component, and r = 0.40 (p < 0.01) for writing (Weir et al., 2013). In a different study, Bridgeman et al. (2016) found that the correlation between the Test of English as a Foreign Language (TOEFL) and GPA scores for 787 international undergraduates in the United States was r = 0.18 (p < 0.05). However, when controlling for subgroups, the results changed dramatically, with engineering students from China showing a correlation of r = 0.58 (p < 0.05) or r = 0.77 when corrected for range restriction (Bridgeman et al., 2016). Recent results seem to confirm what previous studies have found (see, for example, a summary of 18 studies by Graham, 1987). Overall, the findings suggest that the relationship between English proficiency and academic performance is less clear than one might have thought (Cotton and Conrow, 1998). To arrive at more meaningful interpretations, Weir et al. (2013) suggest using a variety of relevant external measures such as students’ course-related work, tests, and assignments, in addition to considering the contextual and cognitive parameters of an English test.
Despite the caveats of predictive validity studies and the inconclusive nature of correlations between English proficiency and academic success, adopting an exploratory stance, using course-related measures, and involving stakeholders (i.e., the approach adopted by this study) can lead to more beneficial interpretations. Therefore, this research explored English proficiency in relation to course performance, in addition to teacher educators’ views about important predictors of academic performance. This study took this approach partly due to practical reasons but also because diagnostic assessment literature suggests a close involvement of stakeholders and a more direct relationship with curricular objectives (Alderson and Huhta, 2011. Alderson et al., 2015). Also, as discussed in the previous section, affective variables have been prominently featured as relevant for foreign language learning.
Seeking to better understand the needs of newly enrolled TEFL undergraduates, the researchers set to explore the extent to which students’ English proficiency level, reflected on an institutional English entry test, their attendance to the language class, and other variables reported by teacher educators related to students’ academic performance in their English language class during the first semester of their university studies.
To this end, the study addressed the following research questions:
1. To what extent does Chilean TEFL undergraduates’ entry English proficiency level, as measured by an institutional English entry test, relate to their academic performance in the English language class?
2. To what extent does student attendance relate to academic performance in the English language class?
3. What variables do teacher educators think have an impact on students’ academic performance in the English language class during the first semester of their studies?
Overall research design
This study adopted a mixed-method exploratory research design (Creswell and Plano, 2017). In the absence of previous local studies, an exploratory approach is better suited for the collection and analysis of data as emerging from available sources of information such as the scores of the institutional English entry test, students’ course grades and attendance records, and teacher educators’ views.
Test scores and English language class attendance and grade records from 87 TEFL undergraduates (47 females and 40 males) at one Chilean university were used for this study. All undergraduates were newly enrolled in the TEFL program (i.e., they were in their first semester of studies), their ages were between 18-20, and their native language was Spanish.
Two teacher educators were interviewed. Both taught English language courses to develop undergraduates’ English proficiency at the TEFL program, and both taught all 87 undergraduates. Teacher educators were trained as teachers of English in a Chilean university and had an MA in Applied Linguistics. Each had five to seven years of teaching experience, and their native language was Spanish.
The English entry test
The institutional English entry test used for this study was designed and validated by a team of content experts (experienced language teachers with postgraduate studies in applied linguistics) and psychometricians. It is administered and used at the university level for the placement of almost all undergraduates regardless of their program (e.g., sociology, political sciences, pedagogy). However, the researchers had little information on the validation process of the English entry test.
English pedagogy students must enroll in all of their English language courses regardless of their English entry level, which is why placement test results are discarded. The test itself was designed based on the Common European Framework of Reference (CEFR) ( Council of Europe, 2001 ), and it covers the A1, A2, A2+, and B1 proficiency levels in three test subcomponents: reading, vocabulary, and grammar. Results are not reported to students but to the curricular team of each program months after the test is administered, and they come in the form of achievement percentages in each of the components as well as an overall CEFR proficiency level. Both achievement percentages and CEFR proficiency levels are based on and reflect students’ test scores. Access to these results was requested by the researchers, but it is not clear to the researchers how the test validation team established the cut-off scores for each CEFR level.
The English language class
A first-semester English language course was the focus of this study. This class is part of a ninesemester series of courses aimed at developing students’ English proficiency level in the four communicative skills (listening, reading, speaking, and writing) as well as providing a linguistic knowledge basis (e.g., vocabulary, grammar, sound system). The overarching course methodology is communicative, drawing from methodological approaches such as Task-Based Instruction and Content and Language Integrated Learning. Most lessons and class materials are designed by teacher educators. The course in semester 1 runs for 16 weeks, from March to July, with two consecutive 80-minute lessons every day from Monday to Friday.
The interview pocedure
Two English language teachers were interviewed after the end of semester 1. Both taught all 87 students and had been working with first year cohorts for the past few years. The interviews were semistructured, with a common set of questions at the beginning followed by a broad list of tentative themes (e.g., course design, course teaching, studentdirected learning, study habits, student attitude, and motivation) to guide the entire procedure. Examples of these are “in your opinion, what variables have an impact on students’ academic performance in the English language class?”, “to what extent is attendance important for students’ academic performance in the English language class?”, and “to what extent is student motivation/attitude important for students’ academic performance in the English language class?”. The interviews were conducted by a trained research assistant who had years of experience working with first-year English pedagogy students as part of the tutorship program at the university (Alonso and Ramos-Gálvez, 2018).
Each interview lasted approximately 70 minutes and was audio-recorded, transcribed by the research assistant, and double-checked for accuracy by the research team.
Quantitative and qualitative data analysis
Two methods were used for data analysis. Firstly, six measures of quantitative data were analyzed using SPSS Statistics, version 26. The six different measures correspond to students’ overall test results, scores for each of the three test subcomponents (grammar, vocabulary, and reading), and two course-related measures (overall course grade and percentage of course attendance). Relationships between the variables were established by means of correlation and regression analyses (this is further discussed below).
Secondly, interview transcripts were coded by the two researchers. Given the exploratory nature of the study, no a priori framework was imposed on the data, thus allowing for codes and themes to emerge inductively in a bottom-up approach ( Dörnyei, 2007 ).
Descriptive statistics corresponding to students’ (n = 87) test scores (as well as their CEFR equivalent) and language class performance and attendance are summarized in Table 1. Frequencies are expressed according to the maximum possible test score of 50; the four CEFR levels based on the scores covered by the test (A1, A2, A2+, and B1); final language class grades, which can range from 0 to 70; and the percentage of class attendance. In general, students performed at the upper end of test scores and CEFR levels targeted by the test (A2+ and B1) and exhibited what would be considered expected language class performance and attendance.
An initial Spearman’s Rho correlation test between the students’ test results and language class performance shows that there was a statistically significant and weak correlation between these two variables (r(85) = 0.37, p < 0.01).
Then, the test scores and class performance were explored in more detail. Table 2 presents the descriptive statistics for the percentage of achievement that the students obtained in each of the three English entry test subcomponents (grammar, vocabulary, and reading).
The students performed relatively well on all three subcomponents, with grammar being the lowest and having the most spread-out scores, followed by reading, and vocabulary, which showed the highest performance. The TEFL students’ high performance in the English entry test (designed and administered at the university level) is hardly surprising. As these students have chosen to study a degree in English teaching, they are likely to have a positive disposition and a potentially higher aptitude towards language learning.
Table 3 summarizes a multiple correlation analysis between language class performance (representing students’ overall course grade and used as the dependent variable), each of the three test subcomponents, and class attendance, with the last two being the independent variables.
Test score CEFR equivalent
A2 A2+ B1
11 36 30
% Grammar subcomponent
% Vocabulary subcomponent
% Reading subcomponent
M SD n
M SD n
67.42 22.44 87
71.72 14.70 87
*** Correlation is significant at the 001 level unilateral
Language class performance
*** Correlation is significant at the 001 level unilateral
The Pearson correlation coefficients show statistically significant and positive relationships between variables. For instance, class attendance shows a strong association with class performance (r = 0.78, p < 0.01), while correlations between the grammar subcomponent and class performance (r = 0.31, p < 0.01), the vocabulary subcomponent and class performance (r = 0.31, p < 0.01), and the reading subcomponent and class performance (r = 0.26, p < 0.01) are all positive but weak. The lack of a stronger relationship between the English
entry test subcomponents and class performance is not surprising, especially considering that previous empirical studies have shown inconclusive associations between English proficiency measures and academic performance (Cotton and Conrow, 1998. Graham, 1987. Ingram and Bayliss, 2007). However, the strong relationship seen for class performance and class attendance might support that attending classes is essential when it comes to developing communicative competence in a foreign language.
A multiple regression analysis for class performance (dependent variable), test subcomponents, and class attendance (independent variables) found that test subcomponents and class attendance could help to explain around 69% of the variation in course performance (F(4, 82) = 49.17, p = 0.000, R. = 0.706, R. Adj = 0.69). It was found that the scores of the reading (β = 0.41, p = 0.60) and vocabulary (β = 0.11, p = 0.19) subcomponents do not help to explain class performance. However, the scores of the grammar subcomponent (β = 0.18, t(82) = 2.09, p = 0.03) and class attendance (β = 0.75, t(82) = 12.48, p < 0.01) demonstrate statistical significance and might help to explain the results regarding class performance. More specifically, the β coefficients of this model show that course attendance explains around 75% of course performance, whereas results in the grammar test subcomponent account for around 18%.
To summarize, the researchers see that all independent variables (results for the grammar, vocabulary, and reading test subcomponents and course attendance) have a positive correlation with course performance. However, only the grammar test results and course attendance help to explain and predict course performance. Out of these two, course attendance shows a potential to explain course performance, with a 0.75 β coefficient (75%).
The qualitative data obtained from interviews were initially coded around broad tentative themes (e.g., course design, teaching, studentdirected learning, and study habits), while more specific themes emerged through subsequent rounds of analysis. The following section presents and discusses the themes that were given more importance by the teacher educators, namely those related to student attitude and motivation and course design and teaching.
Attitude and motivation
Both teachers highlighted the importance of student motivation, self-directed work, and study habits in their learning. Aspects such as attendance and class participation, as well as having a positive attitude towards L2 learning and learning determination, are perceived to be particularly relevant for academic success.
When it comes to class attendance, what the teacher educators said was consistent with the positive correlation between class attendance and students’ academic performance registered in the quantitative analysis. For instance, teacher educator 1 said “…I think it is clear that there is a relation between students’ attendance to class and their academic performance…”, while teacher educator 2 reported the following: “Attendance has a great impact on students’ development. For example, when they are asked to perform in an integrated task, you can see that the ones who miss classes are clearly more disadvantaged”.
Both teacher educators also agreed that a positive attitude leads to better learning. In the same vein, it is also interesting to note that teacher educator 2 associated an immature or childish attitude with poor academic performance: “We had a very loud and immature group of students, they would laugh, shout and argue in class, really very childish, and their performance was very poor. They struggled because they couldn’t concentrate and would always interrupt”. An additional illustrative quote is provided by teacher educator 1:
I remember one student. Her English was very, very weak, but she was so disciplined and positive, always showing a great attitude [….] Regardless of how much I tried, she would never understand what I was saying in English, but she kept a positive attitude, she was very, very persistent, and she managed to pass the class. If you saw her today, you wouldn’t believe how much she struggled. Now she is not the best, but she can communicate in English!
Similarly, student participation was also linked to better academic performance by both teacher educators, with teacher educator 1 making an explicit connection between participation and effective communication: “The most participative students, yes… those who always participate actively, yes, that has helped them a lot in their communicative performance”. In turn, teacher educator 2 reported: “…these students have a good performance, eh… but, yes, of course, they would never miss a class and always participated quite actively”.
Finally, student determination and motivation were also reported to be essential for class performance, this was especially emphasized by teacher educator 2, as seen in the following excerpts: “…there are two students whose English level was really, really elementary, but that completed their first and second semesters. I think it’s mostly a matter of persistence; sustained work is essential in the first semesters of study” and “...this group of students is really cooperative; they are always actively engaged in class and willing to do whatever it takes to improve their English”.
In general, the teachers perceive that students’ attitude and motivation play an important role in their learning. This is hardly surprising, especially given the communicative nature of an English class.
Course design and teaching
One relevant aspect mentioned by both interviewed teachers is related to the appropriateness of the material selected for teaching purposes. It is perceived that students value the relevance of the studied content. For instance, teacher educator 1 said that “using simpler texts at the beginning facilitates students’ learning. We [teachers] cannot expect that they understand every word in a text, but they start processing information little by little...”. Similarly, teacher educator 2 sees the selection of material, specifically chosen for students’ level to scaffold their learning process, as a key element to motivate students and have them involved in meaningful learning experiences:
...texts chosen were appropriate and made sense to students. Most of them mentioned that they liked the topics addressed and that they felt motivated to read content-related material. The selection of texts is key, and it’s very important that the teacher knows how to use that material in class.
Regarding course design and the teaching practice, both interviewees stated that the teacherstudent relationship, teaching team organization/ collaboration, and exposing students to different teaching styles all have a positive impact on students. For example, teacher educator 1 claims that teachers’ “…attitude towards the class and what we do there have a positive impact on students’ learning” and that “there is a good relationship between students and teachers, and among teachers too. We are open to consider the feedback we receive from our students”. Exposing students to different teaching styles by having the team of teachers see all studentgroups is also seen as positive by teacher educator 2:
I have never worked in a program in which teachers rotated [sic]. I feel like the fact that each of us had a different specialty also gave students a special gift. Students could see that teachers had different perspectives and opinions, and that those are not necessarily incompatible or inconsistent. We are all constantly learning.
To summarize, the informants reported that aspects such as the students’ own work and attitude influence their academic performance, along with other factors such as their attendance to class, participation, and commitment to learning. On the other hand, aspects such as course design, teacherstudent and teacher-teacher relationship and distribution of teachers per class seem to facilitate students’ involvement with the class, which results in better learning outcomes.
The purpose of this study was to explore a set of variables involved in students’ academic success during their first semester of studies. The first part of the research addressed the relationship between students’ entry English proficiency level and class attendance and performance. Then, by means of teacher interviews, this study explored how aspects such as motivation, attitude, course design, and teaching itself relate to students’ academic success.
Correlations between test scores and student grades were statistically significant but weak, at r = 0.37 for overall test scores and course grades and r = 0.26, r = 0.31, and r = 0.37 for the reading, vocabulary, and grammar test subcomponents and course grades, respectively. It is worth noting that the relatively small sample size and the homogeneity of the group of students could have contributed to these results (out of the total 87 students, 10 were categorized as A1, 11 as A2, 36 as A2+, and 30 as B1). Another possibility is that, in spite of the English entry test subcomponent labels, the constructs operationalized by the test are different from those operationalized in course assessments, even in those assessments that are more focused on fine-grained language components rather than communicative skills.
It is well known that cognitive and contextual factors play a significant role in test performance ( Weir, 2005 ), neither of which have been considered in this study. Similarly, performance on the entrance test and on course assessments present students with different consequences. The former is administered as an obligatory but low to no-stakes examination, whereas the latter determines whether students move on to the next semester of studies, arguably leading to high-stakes consequences. The researchers believe that this relationship between the stakes and the quality of the assessments also deserves more attention. For example, it has been argued that the low stakes of diagnostic assessments would not negatively affect student performance to the extent that other high-stakes assessments would (Alderson, Haapakangas et al., 2015).
Before discussing qualitative results, it is worth considering that Pollitt (1988) suggests that correlations in the range of 0.25-0.35 are regarded as typical and informative for test performance and later real-life measurements. This means that the correlations reported in this study are not only typical, but also potentially relevant for this research context. Where no other diagnostic measures exist, such as in the studied program, using results from the entry test to inform teaching and provide student support can be a practical and beneficial approach, at least until more diagnostically relevant procedures come into existence. This does not mean that such practices should replace diagnostic ones that better align to hypothetical features of diagnostic tests such as those proposed in the literature ( Alderson, 2005. Alderson and Huhta, 2011. Alderson, Brunfaut, and Harding, 2015).
The results from the two teacher interviews revealed that students’ motivation, participation, attendance, and grit are perceived to be equally, if not more important than their entry English proficiency level. This was also to some extent signaled by this study’s quantitative results, which showed a strong correlation between course attendance and course performance (r = 0.78). Both teachers agree on and emphasize the importance of students’ willingness to attend and participate in class, overcome frustration, persevere, and commit to long-term goals in their success in the English language course. This, in spite of their difficulties with the target language. This would suggest the need to not only better understand students’ entry profiles in terms of motivation and grit, but also to capitalize on these factors in order to ensure better learning experiences. Given that diagnostic practices in English as a second and a foreign language have been conceptualized in terms of language abilities ( Alderson, 2005. Alderson and Huhta, 2011. Alderson, Brunfaut, and Harding, 2015. Harding et al ., 2015), it might be beneficial to complement language-focused diagnoses with measures of motivational variables that also play a crucial role when it comes to learning the target language.
Teachers also reported the importance of some logistical, curricular, and pedagogical decisions (e.g., having multiple teachers, working collaboratively, promoting a good classroom atmosphere, and a close teacher-student relationship) for students’ learning attitude. Although these are typically not easily manageable variables and therefore lack real diagnostic potential, they illustrate the multifactorial nature of academic success and the importance of such considerations, even in higher education.
This study, exploratory and small in scale, represents a first attempt to better understand how English entry proficiency level, class attendance, and other variables perceived to be important by teacher educators relate to students’ academic performance in their English language class during the first semester of university studies at Chilean universities. Our findings indicate a weak relationship between English entry proficiency levels and course performance, while percentages of attendance have greater potential in explaining students’ course performance. As reported by the interviewees, aspects such as students’ attitude, commitment, and class participation can positively influence academic success.
Originally, this study intended to explore variables affecting student motivation by means of focus group interviews. Unfortunately, the researchers were unable to collect these data, as the students went on a national indefinite strike. However, some of the qualitative data suggest that teachers perceive motivation to be relevant for student language academic success.
It has been noted that the importance of entry profiles and diagnostic practices has recently increased, not only in the field of language testing and assessment, but more specifically in Chilean university-level pedagogy programs. The researchers present results with modesty and caution, and they encourage others to further explore ways to better understand students’ university entry profiles in order to address their strengths and weaknesses and to promote better learning trajectories. The results of this study emphasize the importance of students’ entry English proficiency level, as well as their motivation and grit, and are consistent with other similar exploratory findings obtained at the studied university (Alonso and Ramos-Gálvez, 2018). As long as there are no real diagnostic tests, it might be beneficial for universities to profile students’ needs by resorting to teachers’ views and other available instruments such as placement tests or personality questionnaires. This is particularly crucial when attrition rates or consequences of failing a course are high. Finally, students themselves could become active informants by shedding light on the various difficulties they face as they progress through their academic trajectories by participating in focus groups or other forms of research participation procedures.
Copyright (c) 2022 María Ester López
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
The journal allow the author(s) to hold the copyright without restrictions. Also, The Colombian Apllied Linguistics Journal will allow the author(s) to retain publishing rights without restrictions.