What Do the Measures of Utterance Fluency Employed in Automatic Speech Evaluation (ASE) Tell Us About Oral Proficiency?

ABSTRACT

This paper explores what the measures of utterance fluency typically employed in Automatic Speech Evaluation (ASE), i.e. automated speaking assessments, tell us about oral proficiency. 60 Chinese learners of English completed the second part of the speaking section of IELTS and six tasks designed to measure the linguistic knowledge and processing assumed to underpin second language speech production. A sample of eight native speakers rated the learners’ oral productions for functional adequacy. Analyses of the data confirm: (1) articulation rate, mid-clause pause frequency, and repetition frequency predict functional adequacy, (2) breadth of lexical knowledge is the main predictor of articulation rate as well as functional adequacy, (3) speed of syntactic processing predicts end-clause pause duration and speed of lexical processing predicts mid-clause pause duration, and (4) measures of utterance fluency together account for 60% of the variation in functional adequacy scores. These findings suggest that articulation rate best reflects overall functional adequacy. Moreover, other measures of utterance fluency reflect different areas of underlying knowledge and processing, opening up the possibility of automating diagnostic speaking tests.

Acknowledgments

This study was funded by the British Council and the Department of Education, University of York. The authors would like to acknowledge Dr Ping (Abby) Wang, Dr Maha Alghasab and Dr Xiaoyin Yang for their support with data collection. Acknowledgement should also be given to the following RAs for their support with coding: Dr Khalid Alahmed, Rachel Brown, Dr Stewart Cooper, Dr Sara Ebrahimi, and Emily Severn, and to Dr Sible Andringa for support with statistics funded by a British Academy Skills Acquisition Award.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ Independent clauses, sub-clausal units, and subordinate clauses were all considered to be clauses and defined following Foster et al. (2000).

Additional information

Funding

This work was supported by the British Academy [Skills Acquisition Award]; British Council [ELTRA].