Assessing engagement in rehabilitation: development, validity, reliability, and responsiveness to change of the Rehabilitation Observation Measure of Engagement (ROME)

Abstract

Purpose

We describe the development of an observational video coding tool, the Rehabilitation Observation Measure of Engagement (ROME), to quantify engagement in rehabilitative settings at the person (internal state of an individual) and between-system (interaction between individuals) level.

Methods

Forty-nine children with unilateral spastic cerebral palsy (29 males; Age: M = 9.28 yrs, SD = 3.08 yrs) and their interventionists were videotaped during different activities. Construct validity was examined by correlating the ROME with the Engagement vs. Disaffection with Learning Survey and the Pediatric Rehabilitation Intervention Measure of Engagement – Observation questionnaire. Inter- and intra-rater reliability were examined using two independent raters. The ROME’s responsiveness to change was examined by comparing scores across activities.

Results

For construct validity, results showed a positive correlation for person-level engagement (r = 0.444, p = 0.003). No relationship was found between-system-level engagement. High intrarater (91.8%) and interrater (96.1%) reliability was found. The ROME’s responsiveness to change was supported by children exhibiting lower engagement scores during repetitive shaping activities.

Conclusion

These findings provide evidence that the ROME is a reliable tool to objectively examine the construct of engagement within rehabilitation and is valid for quantifying person-level engagement. It provides information that cannot be extracted from questionnaires and can help guide intervention decisions.

IMPLICATIONS FOR REHABILITATION

• Behavioral characteristics, including engagement, of the agents involved in rehabilitation are largely unstudied, although engagement is expected to benefit motor learning.

• The Rehabilitation Observation Measure of Engagement (ROME) is an observation measure that uses predefined codes and can be used universally, as it is not limited to specific language or cognitive levels.

• The ROME is a reliable tool for objectively measuring the role of the construct of behavioral engagement during rehabilitation and valid for examining person-level engagement.

• The ROME may be used as a measure of client and service provider process, of intervention quality, or as a decision guide.

Keywords:

• Engagement
• participation
• video coding
• observational measure
• validity
• reliability
• rehabilitation
• cerebral palsy

Acknowledgements

We would like to thank all the families participating in our camps over the years as well as all interventionists for their hard work.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The study was funded by the order of Malta.