Effects of Interspersed versus Summary Feedback on the Quality of Students' Case Report Revisions

Abstract

This study examines whether students show greater improvement in written case analyses when given feedback that is either interspersed throughout their written case analyses or presented only as a summary, and whether the benefits of these placements vary across differing levels of student performance in the course. Results from an exploratory field experiment conducted with Canadian accounting students who revised and resubmitted case analyses indicate that the effectiveness of feedback depended on an interaction between its placement and the course performance of students to which it was provided. Lowest-performing students increased the quality of their case responses most when provided with interspersed rather than summary feedback, mid-level students improved more when given summary rather than interspersed feedback, and highest-performing students improved significantly regardless of feedback placement. The primary conclusion from this study is that feedback placement influences how well students at different levels respond, suggesting that teachers should consider students' relative course performance when determining the most appropriate placement for their feedback. We also present evidence of the factors that affect the initial quality of case analyses and which influence students' decisions to revise and resubmit their case analyses.

Key Words:

• Communication
• writing
• feedback
• cases
• rubrics
• financial accounting
• editing
• revisions

Acknowledgements

The authors thank Brandy Mackintosh for research assistance, and Don Bacon, Darius Fatemi, Sarah Guina, Barbara Phillips, Fred Pries, Regan Schmidt, participants at the Canadian Academic Accounting Association annual conference, anonymous reviewers, and the two guest editors – Elizabeth Gray and Lynn Hamilton – for comments on a prior version of this paper. The authors also gratefully acknowledge the CAAA/CMA-Canada Scholarship of Teaching and Learning funding program.

Notes

1 One student completed only a portion of the survey. This student provided data relevant to the main analyses of case writing improvement but did not provide data relevant to the factors that affected the revision decision. Reported results are based on all available data, but results do not change if data from this student are excluded.

2 In retrospect, a stronger design to promote consistency across experimental groups would have been for each teacher to provide summary feedback to one-half of the randomly allocated submissions and interspersed feedback to the other half of the submissions.

3 Because the average improvement across feedback groups did not differ at statistically significant levels, we did not ‘curve’ (i.e. adjust) students' final case scores.

4 Grade point average (GPA) is often used as an alternative measure of prior performance, but we did not collect GPA in this study.

5 Five respondents did not provide ratings of these factors, so the sample size for the analyses of revision factors is reduced from 143 to 138.

6 A multivariate ANOVA conducted using the five revision factor ratings as the dependent measures and the revision decision as the independent variable finds the following levels of statistical significance between revisers and non-revisers (all tests two-tailed): score on initial submission (p = 0.730), understand what to revise (p = 0.004), confidence in being able to revise (p = 0.064), time available (p < 0.001), and amount to be learned (p = 0.013).

7 We also ran the ANCOVA including the number of self-reported hours spent revising as a covariate, but it was not significant and did not change the reported results. We also ran an ANCOVA to explore whether the extent of rubric use might depend on initial case score, feedback placement, or time spent revising, but these variables were not associated through main or interactive effects (p > 0.170).