Construct Validation of an Instrument That Assesses Quality of Instruction in Community College Algebra

ABSTRACT

The Evaluating the Quality of Instruction in Post-secondary Mathematics (EQIPM) is a 14-item instrument that uses videos of teaching to investigate the quality of instruction in college algebra courses taught at community colleges. We hypothesize that quality of instruction can be characterized along three distinct factors, the interactions between: (a) students and the content, (b) instructors and the content, and (c) students and the instructors. We used confirmatory factor analysis to empirically evaluate this hypothesis. Using coding of over 900 segments of video of instruction from 40 intermediate and college algebra teachers from eight community colleges, we performed a factor analysis to identify the structure of the instrument. Our findings suggest that the EQIPM instrument captures the three hypothesized underlying factors of quality of instruction with adequate fit statistics and factor loadings. The availability of this instrument opens the door for further research on aspects of quality that influence student performance and for promising programs of professional development for community college faculty teaching mathematics.

SUMMARY

The EQIPM is an instrument that is designed to be used in community college algebra classes to ensure that the instructors show adequate quality of instruction. In this article, we show that it has been used with community college instructors and has shown to be effective and that the individual items in the measure work well together as a whole. In this way, we show that this instrument can be implemented in community college algebra classes across the nation.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. This length is preferable to whole lessons or segments that are 15 minutes long, as in both cases raters miss key elements that need to be used for the coding; segments of 5 minutes tend to interrupt activity more frequently which can influence the coding; segments of 10 minutes or 7.5 minutes seem to generate similar coding among raters. We chose the 7.5 min long segments to be consistent with the coding used in MQI and QIPA.

2. We excluded 150 segments where no instruction was visible, either because students were taking an examination or were silently working at their desks, or because the recording had started earlier than the beginning of the class or was not stopped immediately after the class was over.

3. Predictive validity is a form of criterion-related validity that assesses “how accurately scores on the test predict some criterion measure” (Cohen & Swedlik, 2018, p. 184). This type of validity can’t be established at this time and would require follow up analysis to be performed.

4. The acceptable values for Coefficient H are similar to Cronbach’s alpha and thus, similar standards were used to evaluate the values obtained. An acceptable value of reliability for Cronbach’s alpha is “0.7 or above” (Kline, 2000, p. 13).

Additional information

Funding

The work was supported by the NSF [EHR #1561436].