The role of bottom-up strategy instruction and proficiency level in L2 listening test performance: an intervention study

ABSTRACT

This study examined the effects of bottom-up instruction (BI) on a high-stakes listening test for second language (L2) listeners and compared these effects with those of strategy instruction (SI) in relation to their proficiency levels. Two intact classes (48 L2 listeners) of different L2 listening proficiency levels received five weeks of BI and two weeks of SI. BI offered explicit knowledge of spoken English combined with reproduction practice, whereas SI helped learners deploy metacognitive knowledge and test-taking strategies. The data were collected three times: before the intervention (pre-test) and after each instruction (mid- and post-test). A common-item design that permits Rasch analysis was used. The results showed that BI improved L2 ­listeners’ test performances regardless of their proficiency levels, ­highlighting the usefulness of BI in boosting listening proficiency. Conversely, SI benefited only high proficiency L2 listeners, which supports the threshold hypothesis claiming that L2 listeners require a certain level of L2 proficiency to deploy metacognitive knowledge and test-taking strategies successfully.

Abstract in another language (Japanese)

本研究は、英語リスニングのボトムアップ指導 (Bottom-up instruction, 以下BI)が影響力の大きいリスニングテスト結果に与える効果を、学習者の熟達度に応じて、方略指導 （Strategy Instruction, 以下SI）との比較を通じて検証した。英語リスニングの熟達度が異なる２クラス（計４８名）が週１回計５回のBIと週１回計２回のSIを受けた。BIでは英語の話ことばについての明示的知識を与え音読練習を課した。一方、SIでは学習者のメタ認知知識とテスト受験方略を活性化する指導を行った。そして、リスニングテストを実験前(事前テスト)、それぞれの指導直後（中間テスト、事後テスト）の計３回実施した。リスニングテストは共通項目デザインを採用し、分析にはラッシュモデルを用いた。

その結果、学習者の熟達度にかかわらず、ボトムアップ指導は学習者のリスニングテスト結果の向上に有効であった。一方、方略指導は熟達度の高い学習者には有効であったが熟達度の低い学習者には必ずしも有効ではなかった。この結果は、学習者がメタ認知知識や受験方略を効果的に活用するためには一定の熟達度が必要であるとする説 (threshold hypothesis) と一致した。

Plain language summary

This study examined the effects of bottom-up instruction (BI) on a high-stakes listening test for second language (L2) listeners and compared these effects with those of strategy instruction (SI) in relation to their proficiency levels. To date, such effects have not been rigorously examined despite the fact that listening instruction in the context of English as a Foreign Language (EFL) is most likely optimized when contextualized in coordination with high-stakes tests. High-stakes tests are tests that seriously affect the test takers’ opportunities. Two intact classes (48 L2 listeners) of different L2 listening proficiency levels received five weeks of BI and two weeks of SI. BI offered explicit knowledge about spoken English combined with reading aloud practice, whereas SI helped learners use knowledge such as planning, directed attention, and problem-solving (metacognitive knowledge) as well as test-taking strategies. The data were collected three times: before the experiment (pre-test) and after each instruction (mid- and post-tests). Some test items were used across the tests to estimate and compare the listening abilities of different participants at different times. The results showed that BI improved L2 listeners’ test performances regardless of their proficiency levels, highlighting the effectiveness of BI in boosting listening proficiency. Conversely, SI benefited only high proficiency L2 listeners, supporting the threshold hypothesis, which argues that L2 listeners require some L2 proficiency to successfully utilize metacognitive knowledge and test-taking strategies.

KEYWORDS:

• L2 listening instruction
• bottom-up instruction
• strategy instruction
• high-stakes tests
• common-item design

Acknowledgements

The author is grateful to Yo Hamada (Associate Professor, Akita University, Japan) and Yoichi Meguro (Lecturer, Nihon University, Japan) for their insightful comments on the earlier version of this paper. The author would also like to thank the two anonymous reviewers and editor for their constructive feedback.

Open Scholarship

This article has earned the Center for Open Science badge for Open Data through Open Practices Disclosure. The data are openly accessible at https://jstagedata.jst.go.jp/. To obtain the author’s disclosure form, please contact the Editor.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Notes on contributors

Kozo Yanagawa

Kozo Yanagawa is an associate professor at Hosei University, Tokyo. His research interests include second language listening pedagogy, language testing, and task-based language teaching. He received his Ph.D. from the Centre for Research in English Language Learning and Assessment (CRELLA) at the University of Bedfordshire in the UK. He is currently an editor-in-chief of the JLTA (The Japan Language Testing Association) Journal.