Reading against all odds: creative coping strategies of international students in HE

ABSTRACT

Many overseas students in Higher Education in the UK struggle to understand the compulsory texts for their course, and obtain lower scores for their modules than their monolingual peers. While the existence of this achievement gap is well established in the literature, little is known about the ways in which overseas students in HE with limited English language and literacy skills meet the challenges posed by the compulsory reading on their course. The current study is novel because we focus on a postgraduate Law module and use a mixed methods approach to study the relationship between students’ language and literacy profiles and their academic achievement. The results show that students’ vocabulary sizes did not provide enough lexical coverage for them to understand the set texts and that the language and literacy profiles of our students did not correlate with their module scores. In-depth interviews revealed that students with IELTS scores of 6.0 or lower used ‘creative reading’ strategies such as machine translation, to pass assignments, while staff used ‘compassionate marking’ strategies in the case of struggling writers, which means module scores for some students were likely inflated. Implications for admissions tutors and teachers in HE are formulated at the end.

KEYWORDS:

• Academic achievement
• vocabulary size
• reading comprehension
• law
• IELTS
• assessment

1. Introduction

Being able to read an academic text and understanding it is an essential skill for students in Higher Education. While mastering this skill is already challenging for students who study through the medium of their first language (L1), this is even more the case for students who have grown up with another L1, and do not have the same extensive experience with the language of instruction as their monolingual peers. Since reading is the main way that L2 students learn outside the classroom (Schmitt, Jiang, and Grabe 2011), the ability to read effectively is crucial for student learning. If students are under-prepared for this challenge, their levels of academic achievement may be compromised (Trenkic and Warmington 2019).

While HE graduate attribute descriptors typically include the expectation that students should become masters of their discipline, this is difficult for L2 students with weak language and literacy skills. Indeed, in Trenkic and Warmington’s study, a composite score of receptive and expressive vocabulary explained the largest proportion of variance in academic outcomes of Chinese students’ degree programmes (16.81 percent of unique variance), and a composite score of reading and writing measures was the second best predictor (9.55 percent of unique variance). There was no strong impact of vocabulary or literacy on academic achievement among a group of British undergraduate students who had similar non-verbal reasoning skills and took part in the same study. Thus, the relationship between language proficiency and academic achievement only holds up to a certain threshold of language proficiency: beyond that threshold the impact of language proficiency becomes negligible and other variables become more important for explaining the variance in academic achievement.

That vocabulary turned out to be the best predictor of study success among international students is not surprising because bilinguals tend to have smaller vocabularies than monolinguals (Bialystok and Luk 2012). Thus, if international students in HE struggle with the academic texts, a key reason for this is likely to be that they do not have the vocabulary necessary to read those texts. Trenkic and Warmington (2019) found that L2 PG students have a mean vocabulary size¹ of less than 8000 word families while 1st year L1 UG students have a mean vocabulary size of over 15,000 word families. Similar scores were obtained by Larson (2017) in an online study of the vocabulary sizes of monolingual speakers of English and L2 learners of nine different nationalities.

The fact that L2 learners have smaller vocabularies has important implications for reading. There is an extensive literature which focuses on the number of words that readers need to know in order to be able to understand the text (i.e. the lexical coverage). It is often assumed that if a reader can at least match 95–98 percent of the running words in a written text to their meaning, they may have a level of text comprehension which is adequate for learning through their reading (Hu and Nation 2000; Laufer and Ravenhorst-Kalovski 2010). However, 95 percent running-word understanding means that a reader can expect to encounter one unknown word in 20, which is reached at 5000 word families (Nation 2013). Understanding 98 percent of the words implies that readers encounter only one unknown word in 50, and they would need an estimated word family size of 8–9000 (Nation 2013). This means words from the mid-frequency word band (3k–9k) are essential for independent reading (Nation 2013; Schmitt and Schmitt 2014). However, for optimal comprehension even 98 percent coverage may not be enough. Even L2 learners who know all the words in a text do not necessarily understand it fully, because higher level inferencing is also needed to develop a mental model and text level comprehension (Khalifa and Weir 2009). Furthermore, the depth of meaning of a whole text fully emerges only when considered in relation to the wider context of the academic discipline to which the texts belong. Such texts follow the implicit conventions of their particular academic discourse community, a complex writing style to which L2 students may be unaccustomed.

This is particularly the case for the academic register of legal scholarship texts, also sometimes referred to as legalese, which is profoundly different from everyday conversational style in that it contains nominalizations, embeddings, subordinations, passive constructions, archaisms, and complex lexical phrases, which are not used in everyday language and are difficult for laypersons. An additional difficulty for L2 learners in the field of Law is that legal systems differ between countries. Therefore, second language (L2) scholars/learners and translators can only fully understand legal terms if they are familiar with the legal systems of the source and the target languages (Giampieri 2016). The most common multiword units within Law-specific corpora relate to specific aspects of their disciplinary content (Breeze 2013). These technical terms can be single multimorphemic words, such as arbitration, but they are often multiword units, which can be compounds, consisting of two lexemes, such as vertical agreement² or involve longer expressions (see the supplemental files). The meanings of these expressions can often not be derived from their constituent parts.

Due to space limitations, we cannot provide a detailed review of the difficulties that international students may experience with reading, but there is extensive evidence that they have problems with unknown multiword units in general (Chung and Nation 2003). This may be due to the higher cognitive processing burden of breaking them down into their potentially polysemic constituent parts prior to hypothesising their meaning as a single unit (Siyanova-Chanturia, Conklin, and van Heuven 2011). Furthermore, accessing multiword units is difficult for L2 learners because they have often not had sufficient exposure to the L2 to ensure that new words and expressions are well entrenched in memory (Conklin and Schmitt 2008).

While we have emphasised the difficulties involved in understanding the vocabulary of the reading materials, we are very aware that vocabulary is not the only variable that affects reading comprehension: for example, their knowledge of L2 grammar, their ability to read in their L1, L2 phonological awareness, working memory, background knowledge and metacognition (see Jeon and Yamashita 2014, for details). Establishing which of these is most important is difficult, because the strength of the correlations depends to a large extent on how these variables are measured (see Zhang and Zhang's (2022) meta-analysis for further details). Importantly, Zhang and Zhang concluded that the ‘true’ correlation between vocabulary knowledge and reading comprehension/listening comprehension may fall within the range between .56 and .67, and that vocabulary accounts for 31–45 percent of the variance in L2 comprehension.

We need to bear in mind that not all international students obtain low scores for their degree programmes. Thorpe et al. (2017) point out that the academic performance as measured with GPA of UK-based international students from America, Europe and South-East Asia (e.g. Malaysia and Singapore) is similar to that of students who were assumed to be native speakers³, possibly because in some of these countries English is widely spoken as an L2. Students from East Asia, West Asia and Africa, by contrast, may be at a serious disadvantage with respect to their monolingual English-speaking peers because of the larger linguistic distance between English and the students’ L1, which means that it is more challenging for these overseas students to learn English, both prior to arrival and during the international students’ study, thus further disadvantaging them systematically.

While we cannot discuss all studies on the academic achievement of international students in detail here, it is clear that there is now a growing body of evidence showing that many international students in the UK struggle with academic reading in English and do not achieve the same results for their degree course as monolingual speakers of English enrolled at the same university (Thorpe et al. 2017; Trenkic and Warmington 2019). It is also clear that different vocabulary measures or C-tests (Daller and Wang 2017; Milton and Treffers-Daller 2013) can predict a significant proportion of the variance in degree outcomes of international students. Importantly, Thorpe et al. (2017) note that postgraduate students find themselves in a particularly difficult position because they have less time to adapt to the UK HE environment, while the course requirements for independent scholarship are higher for postgraduates than for undergraduates.

While many studies into the predictive validity of language proficiency tests such as IELTS take a macro perspective in that they use aggregate scores of Grade Points Average across different modules and subject areas as the dependent variable (Pearson 2021), we have chosen to focus on one discipline which has hitherto been relatively neglected in this field, namely Law, although there is considerable evidence that legal language is particularly complex for L2 learners (Giampieri 2016). Because most studies focus on undergraduates, we hope to extend the available literature by concentrating on postgraduates, and by studying the ways in which students with small vocabulary sizes cope with reading set texts in their discipline.

2. The current study

This study aims to fill a gap in the available research by (a) analysing the ways in which HE students’ vocabulary sizes and general language proficiency impact on their academic achievement on a PG Law course, as expressed in grades for one of their modules; (b) providing new insights into difficulty with understanding lexical items used within their related legal written context in this module; (c) analysing students’ awareness of their reading behaviour and the strategies they use to improve their performance; and (d) investigating staff awareness of the issues and the strategies employed by students to obtain better grades.

The research questions we aim to answer in this study are as follows:

1. To what extent is there a relationship between L2 students’ vocabulary sizes, their reading scores and their final module grades on the PG Law?

2. What can analyses of the lexis in the texts reveal about the level of lexical challenge that legal scholarship texts present?

3. To what extent do students experience difficulties during reading of the compulsory texts, and which strategies do they employ to solve any reading problems?

4. To what extent are staff aware of the language and literacy difficulties students experience and the strategies they use?

3. Methods

3.1. Research design, participants and instruments

A mixed method design was chosen for the current study, as this was most appropriate for a study wishing to complement the insights obtained from strictly quantitative approaches to the relationship between the three key variables (vocabulary, reading and academic achievement). More specifically, the study is based on a sequential explanatory design, whereby in the first instance quantitative data are collected, and this stage is followed by an interpretative, qualitative stage, which allows for a deeper understanding of the quantitative results, and in particular the correlations between the three key variables, through insights obtained from interviews with students and staff that quantitative analyses cannot reveal.

Thirty-four new L2 entrants to a ten-week LL.M International Commercial Law module volunteered to provide biodata and participate in the quantitative part of the study (Group 1). Their average age at the time of testing was 27.2 years. Their countries of origin were as follows: China (n = 6), Russia (n = 4), Indonesia (n = 3), India (n = 2), Bulgaria, Egypt, Greece, Italy, France, Libya, Japan, Jordan, Libya, Nigeria, Oman, Pakistan, Portugal, Saudi Arabia, Serbia, Tanzania, Thailand, Turkmenistan, United Arab Emirates (n = 1). They were all enrolled on a compulsory module entitled Advanced International Commercial Law Issues (AICLI), and they all gave us permission to view their summative assignment scores. Performance was measured on a continuous scale. As is common in the UK, a score of 50 percent constitutes a pass mark at postgraduate level. The marking criteria for the module reveal that language was not an area of focus for awarding grades.

Four of Group 1’s participants had not been required to take an English language proficiency qualification as a condition of their offer of a place because they had already studied for their undergraduate degree at a UK university. As their IELTS scores were not available, and their experience with English was very different from those of the remaining cohort, these four students were not included in the quantitative part of the analyses. However, all 34 students who formed Group 1 were invited to participate in the qualitative interview stage (Group 2). All fourteen who applied for this were accepted; they attended two or three semi-structured interviews each, bringing a reading record with them to promote discussion concerning the reading that they had done for Week 1 of their course (Law and Economics) and Week 4 (Competition Law). Their final interview concerned reading preparation for their summative assignment for the AICLI module. After completion of the analyses of the qualitative interviews, the results were discussed with Staff in the Law School who had taught this module and a higher authority within the School of Law.

The results from a pilot study revealed that a few participating students knew more than 14,000 word families. This meant that Nation and Beglar’s (2007) test was less suitable for these learners as words of frequency layers lower than 14,000 would not be included. The 20,000 word version of the VST (Coxhead, Nation, and Sim 2015) was therefore selected as the vocabulary size measure, although we are aware that the test possibly overestimates individuals’ vocabulary sizes because it is based on a multiple choice format, which allows for guessing, and there are questions about the sampling of words which represent each frequency layer (see Gyllstad, Vilkaitė, and Schmitt 2015; Stoeckel, McLean, and Nation 2021 for a full discussion). However, using this format makes it easier to compare our results against those of other studies in the same field which have used the VST. In addition, the test can reveal to what extent the required lexical coverage of 98 percent of the words in the text can be achieved with the participants’ total VST scores. We also obtained AICLI module summative assignment grades, moderated by the end of the academic year 2019–2020, as well as their IELTS scores (overall scores and separate scores for reading).

We used Vocabprofile from the Compleat Lextutor (Cobb 2020) (https://www.lextutor.ca/) to analyse the frequency layers to which the words in the texts for Week 1 and Week 4 belonged. Compulsory reading texts for Weeks 1 and 4 were uploaded separately to Sketch Engine (Kilgarriff 2020) (https://www.sketchengine.eu/) to enable analysis of the most common key words, collocations, two-word sequences and other multiword units for any particular text.

Ethical approval was obtained from the Ethics Committee at the university where the study was carried out, prior to the start of data collection (20 April 2018).

3.2. Procedure

Group 1 participants completed the vocabulary task in September 2018 in Week 0 of the Autumn. Group 2 participants engaged in the qualitative phase of the study later that term. Topics selected for discussion in interviews were designed to allow participants to express their insider perspectives as freely and accurately as possible. These were: reading behaviour as displayed in their reading record; challenges/areas of difficulty; learning achieved (for example, new words learned through reading); new concepts understood; close reading and discussion of a self-selected text extract of particular interest. Students’ final interview was devoted to gathering their experiences of reading for their upcoming assignment for this particular module. Group 1 participants were offered a small thank-you token for their participation, and those from Group 2 were in addition offered a £55.00 book token, as an incentive to mitigate the risk of attrition. This was funded by the HEI Teaching and Learning Programme Fund.

Awareness of the potential for the first author’s position as a tutor in HE to affect the outcomes of the qualitative part of the study also needed to be considered. It could be anticipated that due to the likely high power distance of the teacher/student dimension, students would be unwilling to reveal their innermost thoughts to a teacher to whom they felt that they should show respect and indeed may have felt the need to try to impress. It was decided that it would lower the power distance dimension if the researcher were viewed more as a peer rather than as a teacher, and it would be helpful if she too could understand more of the student perspective and have at least a minimal background context for students’ reading requirements. Having gained the module convenor’s permission to attend the module lectures, the first author therefore undertook to do this throughout the term.

3.3. Data analysis

An analysis of the quantitative data with the Shapiro Wilk test revealed that the VST results (W(30) = .973, p = .616) and the summative assignment scores (W(30) = .953, p = .203) were normally distributed, but the overall IELTS score (OIS) score (W(30) = .856, p < .001) and the reading subscores (W(30) = .906, p = .012) were not. Because the sample was small and not all variables were normally distributed, we used Spearman’s rho for computing correlations between variables.

The interviews with students and staff were voice-recorded, transcribed and coded using N-Vivo. The data reduction process was iterative and fully discussed with an additional rater. Descriptive labels were gradually reduced to specific codes through returning to the literature throughout the data analysis process. The interview coding system for each group that was finally developed from this (see the final coding summary, and anonymised extracts from the interviews in the supplemental files).

4. Results

4.1. Vocabulary sizes, reading and academic achievement on the law module

The mean OIS for the 30 students in Group 1 was 6.63 (SD .71) and scores ranged from 5.5 to 8.0. Ten students had an OIS of 6.0 or below, 11 an OIS of 6.5 and nine an OIS of 7.0 or above. The mean IELTS reading sub-score was also 6.63 (SD 1.07), and scores ranged from 5.0 to 9.0. There were 15 students with reading scores of 6.0 or below, four with a score of 6.5 and 11 with a reading score of between seven and nine. The students’ vocabulary sizes as measured with the VST ranged from 2600 to 16,800. The mean score was 10,026.67 (SD 3299.21). Students’ mean final module assignment grade was 64 (SD 6.83). Grades ranged from 50 to 75.

An overview of the correlations between variables is given in Table 1, which shows there was a medium strength correlation between the OIS and VST scores (r_s = 0.545, p = 0.002), and slightly stronger correlation between the reading sub-score and the VST (r_s = 0.594, p = .002) than between the OIS and VST. Interestingly, the module score did not correlate with either the VST or the OIS or the reading subscore, which calls for an explanation (see the discussion section).

Table 1. Spearman’s correlations between OIS, IELTS reading subscore, the VST and the module score.

	OIS	IELTS reading sub-score	VST	module score
OIS	1.000	.725**	.545**	0.299
p-value		0.000	0.002	0.108
IELTS reading subscore		1.000	.594**	0.110
p-value			0.001	0.564
VST			1.000	0.070
p-value				0.696
module score				1.000

**Correlation is significant at the 0.01 level (2-tailed).

Table 2 provides further information about the relationship between vocabulary and OIS scores and shows that mean vocabulary sizes tended to increase in line with the corresponding increase in OIS.

Table 2. Correspondence between Group 1 vocabulary size scores and OIS at 0.5 band intervals.

	Sample size	Lowest VST score	Highest VST score	Range in VST scores	Mean VST score
5.5 OIS	1	8800	8800	0	8800
6.0 OIS	9	2600	12,000	9400	8155
6.5 OIS	11	6200	13,800	7600	9455
7.0 OIS	3	9000	12,800	3800	11,130
7.5 OIS	2	11,600	13,400	1800	12,500
8.0 OIS	4	11,400	16,800	5400	14,050

4.2. The lexical challenges presented by legal scholarship texts

The vocabulary profiles of the compulsory texts for Week 1 shown in the supplemental files reveal that for three of the six texts, vocabularies of more than 8000 word families are needed to achieve a lexical coverage of 98 percent, which means that eight students in our sample, who have vocabulary sizes smaller than 8000 word families, would struggle with these texts. For week 4 (see the supplemental files), only two of the texts are accessible for students with smaller vocabularies, and in both weeks, there is one text for which a vocabulary of 20,000 word families would be needed to achieve a coverage of 98 percent (see the supplemental files for details).

Tables 3 and 4 provide further information about the vocabulary sizes needed to read the set pieces, and the readability indices for each text. The results clearly show that vocabulary from the low mid frequency bands (6k–9k) is often needed in addition to those from the higher mid frequency bands (3k–5k) to understand texts. In terms of whole texts, a 98 percent text coverage of Coase can be achieved with 6–7000 word families, whereas for Petit and Henry a vocabulary size of 9–10,000 word families is required for 98 percent coverage (see the supplemental files). Further information about the difficulty of the vocabulary in the texts can be found in analyses of the keywords from these two sample texts. Key words were extracted with the Keywords Terminology Extraction function under Sketch Engine. These analyses show that 96.6 percent of the key words in Coase and 87.1 of the key words in Petit and Henry are to be found in the word family bands of up to 9000 (Tables 3 and 4). Indeed, even low frequency word family bands are needed if 98% coverage of Petit and Henry is required. As understanding such keywords is essential for text comprehension, it is clear that the compulsory texts are challenging for students with smaller vocabularies.

Table 3. Key words as word family band percentages (Coase 1937).

Word family band	Frequency level	Score (%)
1–3000	High frequency	62.1
3,001–5000	High mid-frequency	17.3
6,001–9000	Low mid-frequency	17.2
9001 +	Low frequency	3.4

Table 4. Key words as word family band percentages in Petit and Henry (2010).

Word family band	Frequency level	Score (%)
1–3000	High frequency	66.6
3001–5000	High mid-frequency	7.7
6001–9000	Low mid-frequency	12.8
9001+	Low frequency	12.9

In addition, the samples of formulaic language shown in the supplemental files show that the texts contain many technical terms that cannot easily be matched to meaning without a background knowledge of the legal concepts whose meanings are buried within the polysemic nature of the formulaic language used to convey these key terms. Key words such as those shown in the supplemental files may also be challenging to decipher, although many of them are also used in general language. Finally, as the supplemental materials show, the Flesch Reading Ease (FRE) scores confirm that all the texts in Weeks 1 and 4 are difficult or very difficult, although the Flesch-Kincaid Grades (FKGs) appear to suggest that they are mostly appropriate for L1 UG students (see Crossley, Alan, and Macnamara 2011 for a critique).

4.3. Students’ experiences with reading legal texts

The aim of this part of the study was to complement the quantitative results with qualitative analyses of the students’ reading behaviour, as it emerged from discussions with the first author of the paper. These are likely to give more in-depth insights into the problems students face during reading and provide information about the strategies they adopt to complete the task.

The analysis will focus on two students (Fang and Kosta) who were selected because they came to interviews with the same PowerPoint slide from one of the lectures, but very different vocabulary sizes. The fact that they wanted to discuss the same text made it possible to reveal the differences in the ways in which they approached the reading task. Fang studied Law in China but had no background in Economics. Her IELTS scores were among the lowest in the cohort (6.0 for OIS and a reading sub-score of 5.5), but she was awarded 63 percent for her assignment. The interview in the supplemental materials reveals that she struggled to understand the text on the PPT slide, even at the word recognition level despite her vocabulary size of 8400. To help her understand words in the text, Fang used a bilingual electronic resource which gives discipline-specific and more general definitions. She translated unknown lexical items and wrote them directly on the text. The marks that she has made on the slide (Figure 1) shows that she had translated the title as well as five keywords from the text of 88 words into Chinese, that is approximately 10 percent of the text. Thus, she was not even close to reaching the 98 percent coverage that would be required to understand the text. The interview in the supplemental files reveals that she was clearly challenged by the need to interpret formulaic language. Although she could match the lemmas black and box to her mental lexicon as separate lexical items, without relevant L2 cultural knowledge, she could not infer the meaning of the compound black box, and so was prevented from building the required mental model of the entire text. Despite her Chinese translations, she had difficulty in summarising the final sentence of the slide.

Figure 1. Lecture slides annotated in Chinese by a student.

This Figure shows a screenshot of a pdf of a lecture on Law and Economics, consisting of a title and three short paragraphs. The screenshot also shows annotations in Chinese made by a the student.

Kosta’s reading behaviour contrasted strongly with that of Fang. She was articulate and confident, but despite her VST score of 15,800, she still found it challenging to read some of the set texts for Week 1 due to her lack of Economics background knowledge. She was not required to take an IELTS test as her undergraduate degree had been completed at a UK HEI.

Although she stated finding one of the set reading texts difficult (see the supplemental files), Kosta’s notes revealed that she was able to read and understand the slides from the lecture. She also mentioned that she planned to use the Reading Week to try and read texts that she had not yet understood and to ask further questions of the lecturer. She used an online Economics dictionary that a friend recommended to her and found it helpful to search using Google, a tool which she could use with discretion in that she could identify which of the polysemous meanings suggested was most likely to match its context of use. She was careful to store notes from her reading to prepare for the upcoming lectures together with the notes that she took during lectures. If she read online, she always took notes on paper. She had developed a colour-coded system to help her remember the information that she read. Her colour-coded notes were stored in a ring-binder at home. At interview, Kosta said she felt able to learn new words from the texts and confidently used technical terms from her reading, including compounds such as vertical agreement.

In summary, it is clear that students such as Fang struggle to decode the compulsory texts, far more than students with vocabulary sizes similar to those of monolingual British students, such as Kosta. It is therefore not surprising that some students use machine translation websites such as Youdoao (http://translate.youdao.com/) or Multitran (https://www.multitran.com/) to translate whole texts from L2 into L1 or vice-versa using machine software. Interestingly, Hannah, the student with the lowest VST score (2600) achieved the joint-highest module assignment grade of 75 percent. During the interview she admitted that she uses Google Translate when she gets very tired towards the latter stages of her essay writing. She also uses a proof-reader who has a background in Law and can improve the quality of her assignment. Another student, Ying, who had a VST score of 3000 and an IELTS score of 6.0, admitted using Google Translate regularly for her reading and writing only when meeting with me informally on campus after the interviews were over. The strategies used by the students will be discussed further in the discussion section.

4.4. Staff awareness of students’ language and literacy problems

Staff at the Law Department were provided with a summary of the findings of the study. They were surprised that students who arrived with an IELTS score that was acceptable according to the University’s admission criteria were experiencing so many problems with reading. They were aware, however, that students with low IELTS scores were using what they called ‘creative coping strategies’ to help them understand texts and write essays. In addition, staff mentioned ‘compassionate marking’, meaning that markers were lenient in marking essays from students whose language and literacy skills were unsatisfactory, but had understood the essay question and appeared to understand the basic concepts. They also mentioned that they had noted increasing evidence of academic malpractice. However, staff suggested that it was not easy to detect the difference between assignments written by an L2 student with weak English and assignments that have been automatically translated by machine. This requires further consideration.

Staff felt it was important for the results and findings from this study to be made more widely known, particularly to those who set entry-level criteria, but there was the expectation that financial considerations are paramount in this regard, and that entry-level criteria will remain the same. Genuine concern was raised at the finding that the bottom cohort of students struggle so unduly and that this is at odds with the stated university aim that students should become masters of their discipline. Such students’ reading-to-learn needs must be better understood, and staff would like to engage further with how to support those who can only pass this module by adopting coping strategies that are in fact ultimately detrimental to their ability to gain disciplinary mastery through the medium of their L2.

5. Discussion

Our first research question asked about the relationship between L2 students’ PG law vocabulary sizes, IELTS scores and final module grades. Students’ mean vocabulary sizes (around 10,000 word families) were somewhat higher than those reported for the Chinese students in Trenkic and Warmington, who were found to know around 8000 word families. There may be various reasons for these differences. First of all, our students came from a wide range of L1 backgrounds, including European languages, while the international students in Trenkic and Warmington were all Chinese. For European students, learning English may be easier due to the larger number of cognates that exist between English and other Indo-European languages. Second, our students were on average around four years older, may therefore have learned English for a longer time, and possess larger vocabularies for this reason. Third, we used the 20k version of the VST, while Trenkic and Warmington used the 14k version of the test, and results from these two tests may not be entirely comparable. It is possible that the VST overestimates students’ vocabulary sizes because of its multiple choice format, and because each 1000 word frequency layer in the test is represented by only five words. If these happen to be relatively easy, this means the score for that frequency layer is inflated (see Stoeckel, McLean, and Nation 2021 for further discussion). The fact that students’ VST scores were higher while their mean IELTS scores (6.63) were lower than those in Trenkic and Warmington (6.92) lends support to the interpretation that the VST scores may have overestimated students’ vocabulary sizes. According to the VST results, in our sample there were three students with vocabularies smaller than 5000 words, and the lowest score on the VST was 2600 words. If these scores are inflated, the vocabulary sizes of some of the students in the sample were indeed extremely low. Given the required minimum of 8000–9000 words for reading (Nation 2013), it is clear that the ones with such small vocabularies must have found it very difficult to read at all and struggled enormously with the set texts for the LL.M, several of which required knowledge of more than 9000 word families. However, students speaking L1s that are similar to English, or those that had completed undergraduate degrees in the UK, had vocabulary sizes similar to those reported for monolingual British students in Trenkic and Warmington (2019). These students were much more comfortable reading the postgraduate set texts. Thus, our study confirms the findings from Thorpe et al. (2017) regarding the diversity of the international student population and the higher expectations regarding independent scholarship that apply to postgraduate students by comparison with undergraduate students.

Despite the potential threats to the validity of the VST scores, there were robust correlations between the VST and the IELTs scores in our sample. Recall that we used spearman’s rho, which is computed on the basis of ranked data, for the analysis, which means the VST scores are now no longer treated as continuous variables but as ordinal variables: the exact value of a score is therefore no longer relevant, because only its position in the rank order is used for the computation of spearman’s rho. The result showed there was a correlation of .594 between the VST and the IELTS reading subscore, which means the correlation coefficient fell exactly within the range of values (.56–.67), which Zhang and Zhang (2022) found in their meta-analysis of the relationship between vocabulary knowledge and L2 reading/listening.⁴ The fact that this correlation was of a magnitude that might be expected on the basis of Zhang and Zhang’s meta-analysis, lends support to the validity of the findings of this study, namely that students with small vocabularies had lower IELTS reading subscores than their peers with larger vocabularies⁵.

Perhaps the most surprising finding was that there was no correlation between students’ OIS at entry and their final assignment score, nor between their VST sizes and the scores for this assignment. While it is the case that the more proficient readers do not necessarily score highly in assignments (Trenkic and Warmington 2019), correlations between language and literacy skills on the one hand and module grades or overall GPA on the other hand have frequently been found in other studies (see e.g. Ingram and Bayliss 2007; Milton and Treffers-Daller 2013). However, these correlations were often not very strong and in some studies no correlations were found at all. The absence of such correlations could be due to the fact that a myriad of contextual and individual student factors impact on students’ module scores and degree outcomes (see Pearson 2021 for an overview), and we cannot control for all of these in this small-scale study. As one reviewer points out, the fact that samples are generally truncated (in that students who do not obtain sufficiently high IELTS scores are not accepted onto the programme) could also have depressed correlations (see also Daller and Phelan 2013). Further insights into the reasons for the lack of correlations were obtained through in-depth interviews carried out with a subsample of students in Stage 2 (see below for a discussion of research question 3).

Our second research question focused on the lexical challenges posed by the Law texts. First of all, these analyses revealed that students may need to know words beyond the mid frequency level, which has been defined as words between the 3000 and 9000 word families (Schmitt and Schmitt 2014) in order to obtain a lexical coverage of at least 98 percent, which might give them a chance to understand the texts. Clearly, in our study this was not the case for students at the lower end of the vocabulary size spectrum. Second, the technical expressions often have a specific meaning that cannot always be directly inferred from its component parts, constitute an additional hurdle for students (Chung and Nation 2003). It is clear from the excerpt from the PPT slides from one of the lectures, and the overview of technical terms in the supplemental files, that the reading texts the students were facing contained many such technical expressions. In particular, when such terms consist of compounds, reading indices such as the Flesch Kincaid Grade may underestimate the difficulty of academic texts, if such reading indices do not take into account fixed expressions.

Our third research question addressed students’ reading strategies as they emerged from the interviews held in Stage 2 of the project. These interviews revealed not only that students at the lower end of the language and literacy spectrum were virtually unable to read the academic texts, but also that some students resorted to strategies, such as using proofreaders or machine translation devices, to cope with the difficult task of reading the set texts and writing essays for the assignment. It is likely that the use of these strategies was at least in part responsible for the high scores that some students with low language and literacy skills obtained for their final assignment. While these strategies may be helpful in terms of gaining a degree, they do not necessarily lead to the disciplinary mastery to which universities aspire, nor to the development of English language skills. Such strategies are likely to have been responsible, at least in part, for the lack of correlations between, on the one hand students’ vocabulary sizes and their IELTS scores, and, on the other hand, their assignment scores. These results are particularly relevant in the context of the recent emergence of Artificial Intelligence tools, such as ChatGPT, which enable students to rely on even more sophisticated tools to support their reading and writing skills (Sullivan, Kelly, and McLaughlan 2023).

Our fourth question focused on the staff perspective on the issues raised above. This revealed that staff were concerned about the university’s apparent willingness to offer places to incoming L2 postgraduate students with an OIS of 6.0 or lower. They also revealed that compassionate marking practices may have been the second major factor leading to assignment scores that were higher than might have been expected on the basis of the language and literacy skills displayed in essays. These marking practices may well have negatively affected the correlation between students’ assignment scores and their language and literacy scores.

6. Recommendations for practice

As the results from our study have shown that there is much room for improvement in the ways in which L2 students are supported in HE in the UK, we would like to offer some recommendations for practice that may help to improve the situation for staff as well as students.

Students need to learn to read in their L2 in order to develop disciplinary mastery in a UK HE PG environment. Suggested changes to the AICLI module delivery involve the potential for employing a readily available law-trained reading assistant within the School of Law and/or an English for Specific Academic Purposes (ESAP) induction programme of potentially 8 weeks for those who have an OIS of 6.5 and below (but see Trenkic and Hu 2021 for a critical analysis of the effectiveness of such programmes). Only students with this OIS score would be invited to attend, and a fee would be payable.

It would be useful for staff to openly discuss ethical issues relating to this module, for example: the practice of compassionate marking; the dependence of some students on the use of machine translation or proof-readers who are qualified to intervene on content; and the use of L2 translations of L1 resources. It would be helpful to set up a student/staff forum to openly discuss some of these issues in a way which is designed to support and better understand how these ethical issues present to students themselves.

Of course, admitting students with the minimum acceptable IELTS score of 6.0 is likely to lead to problems of the kind we have discussed in this paper. If universities do admit applicants with such low IELTS scores, one might expect admission staff to inform these potential students that it is unlikely that they will become masters of their discipline through the use of their L2, as intended.

7. Conclusion

Here we have reported on a study of the relationship between vocabulary, reading and academic achievement among international postgraduate Law students at a university in the UK. The study was based on a mixed methods approach, which aimed at complementing the quantitative findings from earlier studies into the relationship between these variables. The results show (a) that vocabulary sizes of around 8000 words were required for reading many Law texts, and some texts could only be read with a vocabulary of 20,000 words; in addition, a key issue was the widespread use of multiword units representing technical expressions needed for understanding the set texts; (b) that international postgraduate students with small vocabulary sizes and IELTS scores of 6.0 or lower do not have the language and literacy skills needed to understand the set texts for their course; (c) that there were no correlations between the final assignment for the Law module on the one hand, and the Vocabulary Size Test and the overall IELTS score and the IELTS reading subscore on the other hand. While our study confirmed the results of previous studies which showed that many international students in HE in the UK underperform by comparison with home students because of their limited language and literacy skills, the qualitative part of the study provided new insights into the reasons for the quantitative findings. Students’ ‘creative coping strategies’, which included the use of Google translate and similar online translation tools, in combination with ‘compassionate marking strategies’ from staff, were responsible for the lack of correlations with the final assignments. In other words, the marks given did not provide an accurate reflection of students’ understanding of the module content. These lead to important questions about academic integrity that are increasingly relevant in the context of the emergence of Artificial Intelligence tools enabling students to submit assignments entirely written by such tools. Further research is therefore urgently needed into the ways in which international students can be better prepared for the language and literacy challenges that they face in their studies. This will clearly need to involve raising awareness of the importance of establishing individual lexical profiles and improving understanding of technical expressions through reading, as well as increasing familiarity with relevant disciplinary concepts through encouraging students to engage more fully with their Community of Practice. In this way, international students may have the same chances as home students to become Masters of their discipline.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Anne Vicary

Anne Vicary is an Honorary Fellow at the University of Reading. She has in-depth knowledge and experience in EAP and academic writing and reading. Her research-based approach to teaching has already led to several publications. Her most recent research has enabled her to gain insight into international student reading in terms of student expectations, comprehension levels and self-perceptions as they read for their studies.

Jeanine Treffers-Daller

Jeanine Treffers-Daller is a Professor Emerita in the department of English Language and applied linguistics at the University of Reading. She has published widely about multilingualism, with a specific focus on codeswitching, language dominance and the measurement of lexical diversity in oral and written language. She is the Editor-in-Chief of Languages (MDPI) and a member of the Editorial Board of Bilingualism, Language and Cognition, as well as the International Journal of Bilingualism.

Notes

1 Counts of the number of words a person knows (that is their vocabulary size) are generally based on word families. These means that inflected forms of a word (mended, mends, mending) are counted as different forms of the same word, in this case mend. Similarly, derived forms (government, misgovern, ungovernable, etc.) are counted as different forms of the same word, namely govern.

2 A vertical agreement is defined as an agreement among economic competitors on different levels of production or distribution that affects competition (Merriam Webster online, accessed, 20th November 2022).

3 Thorpe et al. (2017) note that the student record system did not contain information about whether or not a student was a native speaker of English. For the purposes of their study, students from countries accepted by the UK General Medical Council as ones where the first and native language is English were assumed to be native English speakers.

4 The theory of moving equilibrium explains that ‘the emergence of alternative outcomes is not the result of alternative theories, rather the necessary consequence of different conditions (transaction costs) affecting the same phenomenon’ (Nicita 2014, 96)

5 As one reviewer points out, IELTS scores are valid for two years, which means that some students could have obtained their IELTS scores two years before the start of the current study. However, the potential delay between the collection of the IELTS scores and the VST data does not appear to have affected the correlation between both variables in the current study because the correlations are exactly as predicted by Zhang and Zhang’s (2022) systematic review.