A Mixed-Methods Study of Early Intervention System Policy, Supervisory Review Practices, and Effectiveness

Abstract

Early Intervention (EI) systems are police accountability tools widely used to identify and address at-risk officers. Studies have yet to incorporate supervisory review practices into EI evaluations. The current study helps fill this gap by examining the operation and effectiveness of an EI system through the integration of policy and supervisory review practices using a mixed methods design. EI alert data, supervisor response memos, and semi-structured supervisor interviews were collected from a large, metropolitan police agency in the southwestern United States. Results indicated that several officer, supervisor, and EI case characteristics were associated with both supervisors’ EI policy adherence and the likelihood of subsequent EI alerts occurring. These results were supplemented with supervisors’ perceptions of EI processes and reasons for EI system effectiveness and ineffectiveness. Collectively, EI system design and implementation, supervisory review processes, and appropriate oversight of these processes were perceived to be key to an EI system’s success.

Keywords:

• early intervention
• EI systems
• at-risk officers
• police misconduct
• mixed methods
• effectiveness
• supervision
• accountability

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Although the use of EIS can also lead to negative effects, such as reduced activity or de-policing (see Archbold, 2021, Worden et al., 2013).

2 The 579 alerts were received by 256 officers (∼39% of total officers in the department) with a range of 1 to 12 alerts per officer.

3 During the coding process, it was found that supervisors decided that a formal action was necessary in only 16 of the 553 alerts (2.89%). As such, we do not include this measure of the EI policy in the study.

4 This measure therefore captures the portion of the officer’s tenure with the agency prior to their first EI alert during this study period, understanding this may not be the first EI alert ever received for those hired prior to the beginning of the study period.

5 An additional OLS model regressing the three EI policy component measures together as an additive variable on the independent variables was also conducted to see if EI case characteristics or officer and supervisor demographic characteristics influenced the total number of policy components adhered to for each alert. The results were consistent with the individual policy component models presented in Table 3 for the division assignment and supervisor race/ethnicity and tenure measures and so are not presented here for parsimony.

6 Models that included the three EI policy components together as an additive variable were also conducted to see if the combination of these factors or lack thereof would cumulatively affect the likelihood of a subsequent EI alert. This measure was also nonsignificant, indicating that number of policy components adhered to during a supervisory review of an EI alert also did not impact the likelihood of a subsequent EI alert (either for any or the same behavior) from occurring.

7 Collectively, the results raised the question as to whether the effects of EI policy components on receiving a subsequent EI alert were relative to specific performance indicators and/or EI policy change periods rather than absolute across all indicators or periods. Separate logistic regression models were conducted to test this. Model comparison tests between all sub-models and their comparable full-sample models revealed no significant differences. This indicates that in the current sample, EI policy component effects were not relative to specific performance indicator types or for periods of the study coinciding with substantive EI policy changes.

8 In the cases where the third most frequent theme was tied with another, results of four subthemes are presented.

9 All twenty-one supervisors, however, are represented throughout the interview examples presented in the qualitative findings.

10 For example, Ouellet et al. (2019) examined officer misconduct using network analysis and found officers in networks with a greater proportion of fellow officers previously named in force complaints were at an increased risk of being named in future force complaints.