Managerial Behavioral Training For Functional Leadership: A Randomized Controlled Trial

ABSTRACT

Using behavioral principles to improve organizational behaviors is a cornerstone of organizational behavior management. In this study, a manual for managerial behavioral training (MBT) based on behavioral principles was developed and tested. A randomized controlled trial compared pre- and post-training employee questionnaire data from experimental-group managers (n = 25) versus waitlist control-group managers (n = 24). Multilevel modeling was used for data analysis. MBT was found to positively affect the functional leadership behaviors of goal setting (d = .20; p = .039), performance feedback (d = .20; p = .073), value-based performance feedback (d = .22; p = .015), and consequential listening (d = .22; p = .050). In addition, MBT was found to positively affect leadership performance in terms of leader effectiveness (d = .21; p = .038) and employee engagement (d = .27; p = .024). This study describes how managerial leadership training based on behavioral principles can be used to improve managerial leadership behaviors and performance.

KEYWORDS:

• Leadership
• training
• goal setting
• performance feedback
• consequential listening

Functional managerial leadership can be an important foundation of a healthy and effective work environment (Choi & Johnson, 2022; Gravina et al., 2017, 2021; Hantula, 2015; Komaki, 1986, 2015; Komaki & Minnich, 2016; Komaki et al., 2011; Sleiman et al., 2020; Theorell et al., 2012; Yukl et al., 2019). Here, we define managerial leadership as the management of reinforcement contingencies in a work setting (Podsakoff et al., 2006) and functional managerial leadership as leadership behaviors that generate desirable long-term work outputs and business results (Binder, 2016). Behavioral principles highlight the importance of incorporating both antecedent and consequential leadership behaviors in order to achieve functional leadership (Gravina et al., 2021; Grill, 2018; Komaki, 2015). Antecedent leadership behaviors are antecedent stimuli that occur prior to and activate employee behaviors, while consequential leadership behaviors are consequent stimuli that succeed and reinforce employee behaviors (Grill, 2018). Gravina et al. (2021) emphasize consequential leadership as a principal component of a healthy and effective work environment.

Functional leadership behaviors

To perform well and generate desirable work outputs and business results, managers must accomplish two things: effective leadership (i.e., maintaining or increasing employees’ performance-related behaviors as a function of the manager’s direct or indirect administration of reinforcers; Mawhinney, 2006) and employee engagement (i.e., employees’ willingness and ability to contribute to organizational success and the extent to which employees put discretionary effort into their work; Ludwig & Frazier, 2012). Three types of leadership behaviors have been identified as important for generating desirable work outputs and business results: goal setting (Choi & Johnson, 2022; Gowen, 1985), positive verbal performance feedback (herein referred to as “performance feedback”; Sleiman et al., 2020; Weatherly & Malott, 2008), and consequential listening (Arnold et al., 2000; Grill, 2018; Theorell et al., 2012).

Goal-setting research defines a goal as an overt statement about the desired level of future performance that functions as a discriminative stimulus for employee performance (Choi & Johnson, 2022; Gowen, 1985). Managerial goal-setting behaviors involve setting overarching organizational goals as well as goals for individuals and groups, and making sure that goals and corresponding work activities are coordinated (Fleishman, 1953; Gowen, 1985; Malott & Garcia, 1988).

Performance feedback is information about a behavior that allows an employee to adjust his or her behavior (Daniels, 2016; Sleiman et al., 2020; Weatherly & Malott, 2008); it is more efficient when it targets positive rather than negative employee behaviors (Kalata & Naugle, 2008; Sleiman et al., 2020; Turner & Leach, 2012). Effective performance feedback entails using logical arguments and factual evidence when describing to employees how their behaviors contribute to fulfilling goals (Brand et al., 2020; Choi et al., 2018; Johnson, 2013; Lee et al., 2020; Palmer et al., 2015; Park et al., 2019). Binder (2016) described how performance feedback should also include informing employees about how their behaviors and work output contribute to important values. Such value-based performance feedback involves using value statements and describing work tasks as meaningful and important (Binder, 2016).

Consequential listening consists of responding to employees’ ideas and suggestions by hearing employees’ views, acknowledging their contribution, and using employee input when solving problems and making decisions (Grill, 2018). Consequential listening has been found to generate wellbeing, satisfaction, and engagement among employees (Arnold et al., 2000; Grill et al., 2017; Theorell et al., 2012). For example, Grill (2018, Figure 5, p. 25) specified an ABC model of how important employee behaviors – such as providing managers with useful information – can be positively reinforced though consequential listening.

Managerial training research

The accumulation of research highlighting the importance of goal setting, performance feedback, value-based performance feedback, and consequential listening for creating a healthy and effective work environment coincides with growing research interest in how functional leadership behaviors can be effectively trained (Martin et al., 2021). Organizations spend considerable amounts of time, money, and effort on management training (Kaiser & Curphy, 2013; Lacerenza et al., 2017). However, leadership training research is still rudimentary (Lacerenza et al., 2017; Martin et al., 2021), suffering from a shortage of randomized controlled trials (RCTs) – particularly RCTs in naturalistic settings (Antonakis, 2017; Eden, 2017, 2021; Güntner et al., 2020; Podsakoff & Podsakoff, 2019).

RCTs are needed to generate knowledge of how functional leadership behaviors can be effectively trained, as well as knowledge of causal relationships between leadership behaviors and organizational performance (Antonakis, 2017; Eden, 2017, 2021; Güntner et al., 2020; Kuehnl et al., 2019; Martin et al., 2021). In organizational behavior management research, Erath et al. (2021) clarified how the procedure of randomly assigning participants to intervention or control groups in experimental designs focusing on between-subjects interactions inhibits the impact of potential confounding variables such as history, maturation, and order effects. Randomization ensures that all potential confounders are randomly – and hence evenly – distributed between the intervention and control groups; any difference found between the intervention and control groups can thus be attributed to the intervention (Erath et al., 2021). RCTs are rare and have been called for in both leadership research (Antonakis, 2017; Eden, 2017, 2021; Podsakoff & Podsakoff, 2019) and leadership training research (Kuehnl et al., 2019; Martin et al., 2021).

Managerial behavioral training

Meta-analyses show that, while leadership training can generate short-term changes in knowledge and attitudes (i.e., cognitive outcomes), more research is needed on how to achieve sustainable changes in leadership behaviors (Avolio et al., 2009; Collins & Holton, 2004; Conard et al., 2016; Powell & Yalcin, 2010; Tafvelin et al., 2019). Results reported by Avolio et al. (2009) indicate that interventions based on behavioral principles can be more effective in achieving behavioral change than interventions based on other theoretical frameworks. A focus on behavioral principles makes it possible to design more effective interventions by considering the functional relationship between behaviors and the environmental antecedents and consequences of the behaviors (Carr et al., 2013; Gravina et al., 2021; Hanley et al., 2003; Haynes & O’Brien, 1990; Shapiro & Kazemi, 2017).

One method for achieving behavioral change, based on behavioral principles and developed within organizational behavior management, is behavioral training grounded on performance analysis (PA; Brethower et al., 2022). PA is a behavior-analytic operationalization of behavioral deficits and excesses, which considers environmental antecedents and consequences and links interventions to assessment results (Austin et al., 1999). PA incorporates functional assessment of behavior; that is, the identification of important, controllable, causal functional relationships that are applicable to the targeted behaviors, which is a procedure for understanding behaviors based on operant learning (Greer, 2020; Haynes & O’Brien, 1990; Skinner, 1953, 1963). PA has previously been successfully applied by Amigo et al. (2008), Carr et al. (2013), Cruz et al. (2019), Rodriguez et al. (2006), and Therrien et al. (2005), and Vance et al. (2022), and Wilder et al. (2019).

PA typically consists of indirect informant assessment – that is, asking relevant parties structured questions related to behavioral topography, antecedent stimuli, and consequent stimuli (Austin et al., 1999; Wilder et al., 2018). One relevant party that can provide information on managerial behaviors is the managers’ employees, who observe their managers on a daily basis and have first-hand information on the behavioral topography of managers’ leadership behaviors. Also, it is only the leadership behaviors that employees notice that can function as discriminative and reinforcing stimuli for employee behavior.

The most frequently used methods for informant assessment are rating scales, questionnaires and behavioral interviews (Austin et al., 1999). Questionnaires are valid measurement instruments used to collect topographic information on managers’ leadership behavior from the managers’ employees (Avolio & Bass, 2004; Bass & Bass, 2008; Yukl et al., 2019). Kirkpatrick (1979) recommends using before-and-after questionnaires from (at least five) subordinates to evaluate the effect of leadership training on changes in on-the-job behavior. Kirkpatrick also advocate questionnaires to employees to measure the effect of leadership training on the human relation results of feelings and attitudes (Kirkpatrick, 1979; e.g., employee engagement).

However, for the assessment of antecedent and consequent stimuli of managers’ leadership behaviors and the function of such stimuli on the behaviors, more elaborate behavioral interviews by experts in behavioral principles are called for (Austin et al., 1999). A relevant party to ask structured questions about antecedent and consequent stimuli is the individual performing the behavior – that is, the manager (Gravina et al., 2021; Wilder et al., 2011). Also, assessing the antecedent and consequent stimuli that maintain one’s behaviors can itself be an effective intervention for behavioral change (Cohen et al., 2013; R. Ferguson & Rivera, 2022; Gravina et al., 2017; Houmanfar et al., 2008; Humphreys et al., 2008; Korotitsch & Nelson-Gray, 1999; Maas et al., 2013).

Critical components in behavioral training

An end goal of PA is to link assessment results to training interventions (Austin et al., 1999; Carr et al., 2013). State-of-the-art behavioral training consists of goal setting, instructions, practice with performance feedback delivery until the predetermined criteria are met, and homework (Choi & Johnson, 2022; Shapiro & Kazemi, 2017; Sundel & Sundel, 2017).

Ludwig and Geller (1997) assessed the effect of assigned versus participative goal setting and concluded that participative goals are more effective for response generalization; that is, when individuals participated in setting behavioral goals for themselves, they increased not only the targeted behavior but also adjacent non-targeted behaviors. Leadership behaviors are often complex; that is, involving multiple stimulus–response chains, requiring elaborate problem-solving and decision-making and affecting employee performance and work output both directly and indirectly (Binder, 2016; Gowen, 1985; Gravina et al., 2017; Grill & Nielsen, 2019; Hantula, 2015). The complexity of managerial leadership behaviors implies that response generalization – and hence participative goal setting – may be particularly important in leadership training.

Training is usually considered to be an antecedent intervention, and goal setting and instructions are primarily viewed as antecedents (Choi & Johnson, 2022). However, training also contains reinforcing consequences: Choi and Johnson (2022) describe training as reinforcing targeted behaviors by bringing performers into contact with existing natural sources of reinforcement. Also, reinforcing behaviors by providing performance feedback after a targeted behavior until the predetermined criteria are met is a standard procedure in behavioral practice (e.g., shaping; Ferguson & Christiansen, 2008; Sundel & Sundel, 2017). After shaping behaviors during behavioral rehearsal so that the trained behavior is efficient in the manager’s everyday work environment, consequent stimuli in the work environment are likely to further reinforce the behavior (Robinson, 2008; Sundel & Sundel, 2017) through instrumental (Greer, 2020) and social-positive (Beavers et al., 2013) reinforcement.

Homework (i.e., putting behaviors into practice and improving functioning in the diversity of situations at work in which the behavior is useful) ensures that the trained behaviors are adapted to the natural environment and increases the opportunities for instrumental and positive-social reinforcement to occur. In behavioral therapy, homework has been found to contribute to a considerable part (d = 0.48) of the treatment effects (Kazantzis et al., 2010).

Purpose

The primary aim of the current study was to develop and test a leadership training method that incorporates the aforementioned state-of-the-art components of behavioral training (i.e., PA, participative goal setting, instructions, practice with performance feedback delivery until the predetermined criteria are met, and homework) into a standardized manual for improving managers’ functional leadership behaviors (i.e., goal setting, performance feedback, value-based performance feedback, and consequential listening). Previous research suggests that these behaviors may be critical for leadership performance in terms of leader effectiveness and employee engagement. However, the effect of these behaviors on leadership performance has not been tested in randomized naturalistic experimental trials. The secondary aim of the current study was therefore to test whether training managers in goal setting, performance feedback, value-based performance feedback, and consequential listening would result in an increase in the managers’ leadership performance in terms of leader effectiveness and employee engagement.

Method

This study was approved by the Swedish Ethical Review Authority 1060–18/2019-00590.

Participants

Participants were recruited by a division in a Swedish municipal government responsible for recruiting municipal managers (Bergvall, 2012). Using the division’s registry to identify potential participants, 205 individuals were assessed for eligibility from February to June 2019. Individuals who had been municipal managers for at least 6 months were eligible for the study, and 59 managers agreed to participate. Figure 1 shows the enrollment, allocation, follow-up, and analysis, and Table 1 provides the characteristics of the participants (n = 49).

Figure 1. Flow Chart of the Enrollment, Allocation, Follow-up, and Analysis of Participants.

Table 1. Characteristics of the Participants.

Characteristic	Experimental group (n = 25)			Control group (n = 24)			Total (N = 49)
	M	SD	%	M	SD	%	M	SD	%
Age (years)	47.9	5.4		44.5	5.2		46.2	5.5
Managerial experience (years)	9.2	7.1		6.8	6.5		8.1	6.9
University degree			100			88			94
Gender (male)			32			42			37
Managerial level ^a
First-line manager			68			75			71
Strategic manager			32			25			29
Type of organization
Technical			28			29			29
Educational			28			13			20
Healthcare			12			25			18
Cultural			8			4			6
Other			24			29			27
Employee characteristics
Age (years)	46.5	10.2		45.3	10.7		45.8	10.5
Gender (male)			38			35			37

^aFirst-line managers have no subordinate managers while strategic managers do.

Randomization procedure

The managers were randomly allocated to the experimental or control group using the RAND function in Microsoft Excel^TM, version 16 (i.e., all managers were provided with a random decimal number between 0 and 1 and sorted based on this number; the first 30 managers were assigned to the experimental group and the last 29 to the waitlist control group). The 30 managers in the experimental group were subsequently randomly assigned to trainers for the MBT, such that each of five trainers was assigned six managers to train. The managers in the waitlist control group received the MBT after all data had been collected. Only the managers and trainers were informed of the randomization results. To blind those assessing the dependent variables (i.e., the managers’ employees), the managers were instructed not to reveal any information about the training to their employees, colleagues, or superiors. All managers were informed that their employees were to rate their leadership behaviors and performance pre- and post-intervention. All employees were informed that the research involved the evaluation of MBT and that 50% of the managers in the project would receive the training. They were also informed about the blinding: “In order for your answers not to be affected by any expectations of changes in leadership, you will not receive information about whether or not your manager is receiving training.”

Dependent variables

Online questionnaires were used to collect data on the dependent variables from the managers’ employees pre-intervention (i.e., June–August 2019) and post-intervention (i.e., December 2019–January 2020). Data were collected from the experimental and control groups simultaneously. Each manager was rated by an average of 6.4 (SD = 3.1) employees pre-intervention (i.e., M = 6.4 [SD = 2.8] in the experimental group and M = 6.5 [SD = 3.4] in the control group) and by an average of 6.1 (SD = 3.1) employees post-intervention (i.e., M = 5.9 [SD = 3.2] in the experimental group and M = 6.3 [SD = 3.1] in the control group). The response rate was 65% pre-intervention (69% in the experimental group and 62% in the control group) and 65% post-intervention (66% in the experimental group and 64% in the control group). The analyses were based on the N = 256 (n = 125 in the experimental group and n = 131 in the control group) employees who had complete pre- and post-intervention questionnaire data.

Leadership behaviors (primary outcomes)

Goal setting was measured using five items adapted from the Initiating Structure scale by Halpin (1957); Cronbach’s alpha was .91 pre-intervention and .89 post-intervention. Performance feedback was measured using an adaptation of Yukl et al.’s (2008) four-item Rational Persuasion scale; Cronbach’s alpha was .94 pre-intervention and .93 post-intervention. Value-based performance feedback was measured using an adaptation of Yukl et al.’s (2008) four-item Inspirational Appeal scale; Cronbach’s alpha was .94 both pre- and post-intervention. Consequential listening was measured using an adaptation of Grill et al.’s (2017) three-item Participative Leadership scale (based on Arnold et al.’s (2000) Participative Decision-making scale); Cronbach’s alpha was .91 pre-intervention and .92 post-intervention. All items were rated on a Likert scale ranging from 1 (never) to 7 (always) and are presented in full in Appendix A, Table A1. The convergent and discriminant validities of the primary outcome measures were tested using confirmatory factor analyses (CFAs; Byrne, 2016). The four-factor model was found to acceptably represent the data (χ² [98] = 282.136; CFI = 0.955; RMSEA = 0.086) and to be superior (Δχ² = 1033.690; p < .001) to a one-factor model (χ² [170] = 1427.967; CFI = 0.750; RMSEA = 0.171).

Leadership performance (secondary outcomes)

An adaptation of the Multi-factor Leadership Questionnaire (MLQ; Avolio & Bass, 2004) was used to measure leadership performance. Leader effectiveness was measured with four items; Cronbach’s alpha was .93 both pre- and post-intervention. Employee engagement was measured with three items; Cronbach’s alpha was .96 pre-intervention and .95 post-intervention. All items were rated on a Likert scale ranging from 1 (never) to 7 (always) and are presented in full in Appendix A, Table A2. The convergent and discriminant validities of the secondary outcome measures were tested using CFAs (Byrne, 2016). The two-factor CFA model (χ² [13] = 117.224; CFI = 0.952; RMSEA = 0.177) was found to be superior (Δχ² = 51.768; p < .001) to a one-factor model (χ² [14] = 168.992; CFI = 0.927; RMSEA = 0.211). However, the last two items of the leader effectiveness scale focus specifically on employees, while the first two items do not. Hence, in the final model, the error terms of the first two items were allowed to covary, resulting in a model that acceptably represented the data (χ² [12] = 41.410; CFI = 0.986; RMSEA = 0.099). Table 2 shows the pre-intervention intercorrelations between all dependent variables.

Table 2. Intercorrelations Between the Dependent Variables.

Dependent variable	1.	2.	3.	4.	5.
Leadership behaviors
1. Goal setting
2. Performance feedback	.75**
3. Value-based performance feedback	.73**	.85**
4. Consequential listening	.61**	.63**	.72**
Leadership performance
5. Leader effectiveness	.74**	.64**	.71**	.70**
6. Employee engagement	.75**	.70**	.76**	.72**	.88**

n = 256.

**p < .01.

Invariance testing

Wilder et al. (2019) emphasized the importance of developing valid and reliable measurement methods for performance assessment and measurement invariance is one way of evaluating the equivalence of measurement constructs across time (Putnick & Bornstein, 2016). Accordingly, time invariance was tested by comparing CFA-models in terms of the change in comparative fit index (CFI), using < −.01 as the criterion for accepting more parsimonious models in a stepwise comparison between unconstrained, metric-invariant (weak), scalar-invariant (strong), and residual-invariant (strict) models (Putnick & Bornstein, 2016). The results of the invariance tests are reported in Table B1 and show that all constructs were strict invariant.

The managerial behavioral training: independent variables

A manual for one-on-one MBT (Björnsdotter & Grill, 2021) was developed for training managers in four types of leadership behaviors: goal setting, performance feedback, value-based performance feedback, and consequential listening. The MBT manual was developed by the first author, the third author, and four other licensed organizational behavior psychologists. The length of the MBT was set to six bi-weekly 90-minute sessions, delivered over 3 months, from September to November 2019, by the first author and the four psychologists. The third author watched video recordings of the training sessions and supervised the trainers in implementing the training in accordance with the MBT manual. The training comprised PA, goal setting, instructions, practice with performance feedback delivery until the predetermined criteria were met, and homework.

PA: The PA consisted of a topographic assessment of the manager’s pre-intervention levels of goal setting, performance feedback, value-based performance feedback, and consequential listening, conducted by three parties: the manager’s employees, the manager, and the trainer. The employees did their topographic assessment via online questionnaires. The manager and trainer conducted the topographic assessment jointly in a behavioral interview during the first two training sessions. This behavioral interview also included identifying and analyzing the antecedent and consequent stimuli functionally related to the targeted leadership behaviors (i.e., functional assessments). During the PA, the managers were trained to conduct functional assessments of their own leadership behaviors and of their employees’ behaviors (Gravina et al., 2021). Through the PA, the manager and trainer together identified the manager’s most important behavioral deficits.

Goal setting: Goals were set to target the deficits identified in the PA. One to three behavioral (Brown & Latham, 2002; Latham & Seijts, 2016) and participative (i.e., agreed upon by the manager and trainer; Ludwig & Geller, 1997) goals were specified and written down (e.g., “I will implement weekly follow-up meetings with all my employees in which I will give them positive performance feedback”).

Instructions: The instructions entailed providing the managers with spoken and written information on why and how to conduct PAs, how to set behavioral and participative goals, how to give performance and value-based performance feedback to employees, and how to use listening behaviors as a reinforcing consequence (e.g., by validating employee suggestions).

Practice with performance feedback delivery until the predetermined criteria were met: Practice with performance feedback entailed in-session behavioral training and rehearsal (Segrin, 2008; Sundel & Sundel, 2017) to shape the leadership behaviors specified for each manager in his or her behavioral goals. Training exercises for a wide range of leadership behaviors were detailed in 11 elective modules, from which the manager and the trainer jointly selected those that best suited the manager’s behavioral goals. These modules were: consequential listening and validation, dealing with criticism, PA of employee behaviors, positive reinforcement through performance feedback, corrective feedback, time management, meeting management, goal setting and follow-up, problem-solving, cognitive reappraisal, and mindfulness (Björnsdotter & Grill, 2021). The managers were recorded during the exercises and given video feedback (Fukkink et al., 2011) on their behaviors.

Homework: Homework assignments were formulated and written down at the end of every session to ensure that the trained leadership behavior was shaped to fit the manager’s work environment (Robinson, 2008). The homework consisted of using the leadership behavior that had been trained during the session at work. Each part of the homework was followed-up on at the beginning of the subsequent training session, and the manager received performance feedback on all the behaviors performed in accordance with the manager’s behavioral goals (i.e., the manager was informed about how the behavior had brought the manager closer to his/her goals).

Data analysis

To evaluate the effect of MBT on leadership behaviors and performance, we conducted multilevel modeling (MLM) using the mixed-model procedure in SPSS^TM version 26. When the employees are nested in groups (related to the same manager), MLM accounts for this dependence in the data (Snijders & Bosker, 2011). The main fixed effects for measurement time (pre- and post-intervention), experimental condition (experimental and control groups), and interaction between time and experimental condition were estimated separately for each dependent variable. The models comprised three levels. Repeated observations (level 1) from the employees (level 2) were estimated using an unrestricted variance structure, allowing the variance to take different values pre- and post-intervention, and estimating a covariance between pre- and post-intervention measures. At level 3, the variation between managers was accounted for by a random intercept and a random slope. Maximum likelihood was used for the estimation. The small number of participants implied a high risk of Type II errors (i.e., failing to detect real effects; Mudge et al., 2012). To limit that risk, the alpha level was set to .1, thus accepting a 10% risk of Type I errors (i.e., rejecting true null hypotheses). In accordance with Cohen (1992), the effect sizes were computed by dividing the unstandardized parameter estimate of the interaction effect by the pre-intervention standard deviation.

Results

Intervention integrity

Intervention integrity (Cymbal et al., 2022) was assessed by having the trainers complete a short questionnaire after each training session. Goal setting was assessed with the question: “The manager has set behavioral goals for his/her leadership training”; 100% (25 out of 25) of the managers had set behavioral goals. Instructions about the targeted leadership behaviors were assessed with the question: “The manager was informed about the functional leadership behaviors”; 96% (24 out of 25) of the managers had received this information. The occurrence of functional assessment of the targeted leadership behaviors was assessed after each of the first five sessions with the question: “The manager was helped to analyze his/her leadership behaviors”; 98% (122 out of 125) of the sessions contained such functional assessments. Practice with performance feedback delivery through in-session behavioral training was assessed after sessions 2–5 with the question: “The manager was supported in training new leadership behaviors”; 90% (90 out of 100) of the sessions contained behavioral training. The performance of leadership behaviors between sessions (i.e., at work through homework) was assessed after all sessions except for the first with the question: “The manager has completed the homework according to plan”; managers had done their homework after 84% (105 out of 125) of the sessions. In addition, an overall question was used after each session to assess overall adherence to the MBT manual: “Today’s session was conducted in accordance with the manual”; 88% (132 out of 150) of the sessions were conducted in accordance with the manual.

The training was designed so that the trained behaviors were shaped to fit the work environment well enough to generate positive instrumental and social reinforcement of the behavior, both during the session through positive performance feedback from the trainer and during the homework through positive reinforcement in the work environment. Therefore, the intervention integrity assessment also had the managers respond to a short questionnaire after each training session. Positive performance feedback during the session was assessed after all sessions with the question: “My trainer gave me positive performance feedback”; 95% (142 out of 150) of the sessions contained that kind of feedback. Instrumental and social positive reinforcement during the homework was assessed after all but the first session with four questions: (1) “My new leadership behaviors functioned well,” where 94% (117 out of 125) of the behaviors functioned well; (2) “My new leadership behaviors were appreciated by my employees,” where 62% (78 out of 125) of the behaviors were appreciated by employees; (3) “My new leadership behaviors were appreciated by my colleagues,” where 54% (68 out of 125) of the behaviors were appreciated by colleagues; and (4) “My new leadership behaviors were appreciated by my superior manager,” where 42% (53 out of 125) of the behaviors were appreciated by superior managers.

In addition, video procedural integrity was assessed by having two master students rate the occurrence of accurately completed homework, in session behavioral rehearsal with video feedback, and behavior specific positive performance feedback from the trainer. Seven training sessions were rated by both students and the interobserver agreement (Watkins & Pacheco, 2000) was 92%. In all, 88 training sessions were rated and accurately completed homework was observed in 74% of the sessions, behavioral rehearsal with video feedback observed in 42% of the sessions, and behavior specific positive performance feedback from the trainer observed on average 6.8 times per session (SD = 3.2).

Primary and secondary outcomes

Table 3 shows the observed pre- and post-intervention means and the principal results of the multilevel models (the full results of the multilevel models are provided in Appendix C, Tables C1 and C2). The results indicate that MBT improved the leadership behaviors of goal setting, performance feedback, value-based performance feedback, and consequential listening. The magnitude of the effects was small and ranged from d = .20 for goal setting and performance feedback to d = .22 for value-based performance feedback and consequential listening. In addition, the results showed that there were significant effects on leadership performance in terms of leader effectiveness (d = .21) and employee engagement (d = .27). Figure 2 provides graphs on the changes in all dependent variables.

Figure 2. Pre- and Post-intervention Means for the Experimental- and Control-group. The x-axes are broken and encompass values from 4 (sometimes) to 6 (almost always).

Table 3. Observed Pre- and Post-intervention Means, and Results of the Multilevel Models (i.e., Interaction Effects Between Time and Experimental condition) for all Dependent Variables.

	Observed means
	Experimental group (n = 125)				Control group(n = 131)				Multilevel models
	Pre		Post		Pre		Post		Interaction effects (time * condition)
Dependent variable	M	SD	M	SD	M	SD	M	SD	par.	SE	p	d
Leadership behaviors
Goal setting	5.01	1.25	5.22	1.03	5.01	1.31	4.97	1.26	0.25	0.12	0.039	0.20
Performance feedback	4.75	1.39	4.96	1.31	4.63	1.43	4.55	1.34	0.28	0.15	0.073	0.20
Value-based performance feedback	4.88	1.42	5.16	1.37	4.84	1.51	4.79	1.48	0.33	0.13	0.015	0.22
Consequential listening	5.42	1.25	5.63	1.06	5.17	1.41	5.08	1.51	0.29	0.14	0.050	0.22
Leadership performance
Leader effectiveness	5.60	1.00	5.71	0.93	5.45	1.25	5.33	1.21	0.23	0.11	0.038	0.21
Employee engagement	5.13	1.16	5.31	1.12	4.97	1.41	4.79	1.36	0.35	0.15	0.024	0.27

d = effect size; par. = parameter estimate.

Discussion

Organizations spend considerable amounts of time and money on management training (Kaiser & Curphy, 2013; Lacerenza et al., 2017); however, leadership training research lacks randomized naturalistic experimental trials, and hence fails to provide robust answers as to how managers are most effectively trained (Antonakis, 2017; Eden, 2017, 2021; Kuehnl et al., 2019; Martin et al., 2021; Podsakoff & Podsakoff, 2019). Therefore, the primary aim of this study was to conduct an RCT to test the effects of MBT on leadership behaviors. Four types of leadership behaviors were the focus of training: goal setting, performance feedback, value-based performance feedback, and consequential listening. The results showed that MBT improved the targeted leadership behaviors. Hence, MBT exemplifies how leadership training can be effectively designed by applying behavioral principles and the state-of-the-art behavioral training components of PA, participative goal setting, instructions, practice with performance feedback delivery until the predetermined criteria are met, and homework.

The secondary aim of the study was to test whether training managers in goal setting, performance feedback, value-based performance feedback, and consequential listening would also result in improved leadership performance. The results showed that the increase in leadership behaviors also generated an increase in leadership performance, in terms of leader effectiveness and employee engagement. This finding supports previous research on the importance of goal setting, performance feedback, value-based performance feedback, and consequential listening for leadership performance.

The MBT involved having experts in behavioral principles help managers use PA to identify, understand, and correct behavioral deficits and excesses in their leadership. The managers were also trained to apply PA in their own organizations to identify and reinforce functional behaviors among their employees. The ongoing reciprocal relationship between functionally analyzing and training in leadership behaviors was used to shape increasingly functional behaviors. Tafvelin et al. (2021) concluded that positive utility reactions are imperative in enabling leaders to make use of their trained skills in their work. Our findings corroborate this conclusion; ensuring that the leadership behaviors in which managers are trained fit the environment in which the behaviors are to be used is a likely prerequisite for the operant learning of functional leadership behaviors – that is, positive reinforcement only occurs for behaviors that are followed by positive reinforcers (Greer, 2020; Skinner, 1953, 1963).

The effect of MBT was found to be small and similar for all outcomes. However, the MBT was an individualized intervention, designed to address the behavioral deficits of individual managers. This circumstance suppressed the effect sizes of the training due to the fact that the data analysis used to test the effects of MBT on goal setting, performance feedback, value-based performance feedback, and consequential listening included all participants, regardless of their behavioral deficits or goals. The effect sizes would presumably have been considerably larger if we had assessed, for each manager, only the effect on his/her specific behavioral goal; however, insufficient statistical power precluded such differentiated analyses. Hence, the reported effects of MBT on leadership behaviors are likely to be underestimated.

The effect sizes were slightly smaller than those found in previous leadership training research (Avolio et al., 2009; Collins & Holton, 2004; Powell & Yalcin, 2010). However, most previous studies have used research designs known to augment effect sizes, such as lab studies, temporary impact studies (e.g., passive scenarios and role playing), and lower quality designs (e.g., involving quasi-experimental designs, nonrandom assignment, non-blinding of participants, and lack of a control group; Avolio et al., 2009; Podsakoff & Podsakoff, 2019). The RCT design frees the results from such overestimations. In addition, leadership training research suffers from too many studies using self-reported outcomes (Podsakoff & Podsakoff, 2019). Managers themselves are likely to report larger behavioral changes than those that are evident to their employees, inducing the risk of overestimating the effect sizes of leadership training. Inversely, in the current study, the managers’ employees were used as independent raters of the managers’ behaviors.

Mainstream leadership research lacks concepts and measures that take behavioral principles into account (Komaki, 2015). Based on the work of Yukl et al. (2008), Arnold et al. (2000), and Halpin (1957), we developed employee questionnaire scales measuring goal setting, performance feedback, value-based performance feedback, and consequential listening. The questionnaire scales were found to have adequate convergent and discriminant validity, high internal consistency, and strict time invariance. Furthermore, the criterion-related validity of the concepts was demonstrated in two ways: First, goal setting, performance feedback, value-based performance feedback, and consequential listening were found to be highly correlated with leadership performance in terms of leader effectiveness (r = .64–.74) and employee engagement (r = .70–.76). Second, training in goal setting, performance feedback, value-based performance feedback, and consequential listening was found to improve leadership performance on the same two criteria (d = .21–.27). These concepts and measures of goal setting, performance feedback, value-based performance feedback, and consequential listening are now available for use in future leadership research.

Limitations

The sample size in this study was relatively small. Nevertheless, as argued by Breitsohl (2019) and others, there are considerable practical advantages to using small samples in organizational intervention research. RCTs in organizational research usually require large and costly research projects. Using small samples makes RCTs more practical in organizational research, enabling researchers to provide practitioners with more evidence-based organizational interventions (Antonakis, 2017; Eden, 2017).

No objective performance measures were included in this study. However, indirect assessment is the most common method used for PA (Austin et al., 1999; Wilder et al., 2018). In addition, many performance problems are behavioral deficits, rather than behavioral excesses, and behavioral deficits are not particularly well suited for direct observation (Carr et al., 2013). Thus, indirect assessment methods such as questionnaires are useful. For example, Kirkpatrick (1979) outlines four steps for training evaluation and provide examples of how employee questionnaires can be used for evaluating the effect of leadership training on behavior and human relation results; however, for reactions, learning and objective results (e.g., reduction of costs, reduction of turnover and absenteeism), other measurements are warranted.

Future research

Although this study was conducted with municipal managers in Sweden, cross-cultural leadership research indicates that leadership behaviors that are effective in one cultural context have a high probability of also being effective in other cultural contexts (Den Hartog et al., 1999; Dorfman et al., 1997; Gentry et al., 2013; Grill et al., 2017; House et al., 2004). Nonetheless, the efficacy of MBT in contexts other than Swedish municipalities merits assessment in future research. The relatively small effect size found for MBT may be related to the brief duration of the training, which was 9 hours in total. Most leadership training is more time consuming; for example, the managers studied by Von Thiele Schwarz et al. (2016) were given 20 days to properly learn PA. Future studies of MBT should test whether increasing the number of sessions results in larger effects.

Changes that are attained through behavioral therapy are enduring, as individuals tend to continue to generalize functional behaviors after behavioral therapy (Beesdo-Baum et al., 2012; Kröger et al., 2015; Lopes et al., 2014; Von Brachel et al., 2019). Follow-up measures are needed to assess the extent to which MBT induces sustainable and long-lasting changes in functional leadership behaviors. To further our understanding of leadership practice and training, we encourage future research in labs and naturalistic settings on how managers activate and reinforce employees through their antecedent and consequential leadership behaviors. By assessing how operant learning influences employee behaviors, it may be possible to unravel more of the remaining challenges in leadership research and practice. Lastly, we encourage fellow researchers to replicate or expand on our study.

Conclusion

MBT provides a comprehensive behavioral method for improving managerial leadership behaviors and performance. The results may be used by leadership trainers, managers, employees, employer organizations, unions, and policymakers in efforts to advance the managerial leadership in their organizations.

Disclosure statement

We have no known conflict of interest to disclose.

Additional information

Funding

Swedish Research Council for Health, Working Life and Welfare (FORTE) Grant STYA-2018/0004 funded this work.

Appendix A:

Questionnaire Items for the Dependent Variables

Table A1. Questionnaire Items for the Leadership Behavior Scales.

Leadership behavior	Item
Goal setting	Informs me about my responsibilities and tasks
	Delegates and schedules the work to be done
	Clarifies organizational goals
	Clarifies what is expected of me
	Sees to it that the work of group members is coordinated
Performance feedback	Describes how my work is contributing to organizational objectives
	Clarifies how my work is contributing to organizational performance and effectiveness
	Provides information or evidence that demonstrates the significance of my work
	Uses facts and logic when describing the significance of my work
Value-based performance feedback	Describes my work as meaningful and important
	Describes valuable objectives obtained through my work
	Describes how my work contributes to important values and ideals
	Talks about the importance of my work
Consequential listening	Encourages me to express ideas and suggestions
	Uses my suggestions to make decisions that affect me
	Responds to my ideas even when he/she disagrees with them

Instruction: “Rate how frequently your manager performs the following behaviors: 1 = never, 2 = almost never, 3 = occasionally, 4 = sometimes, 5 = often, 6 = almost always, and 7 = always.”

Table A2. Questionnaire items for the leadership performance scales.

Leadership performance	Item
Leader effectiveness	Is effective in representing our organization
	Is effective in doing his/her job
	Is effective in making employees perform efficiently
	Is effective in leading employees
Employee engagement	Makes me exceed expectations
	Makes me want to succeed in my work
	Makes me want to perform efficiently

Instruction: “Rate how frequently each statement fits your manager: 1 = never, 2 = almost never, 3 = occasionally, 4 = sometimes, 5 = often, 6 = almost always, and 7 = always.”

Appendix B

Table B1. Time Invariance Analysis of the Dependent Variables

	Unconstrained model^a			Weak invariant model^b			Strong invariant model^c			Strict invariant model^d
Dependent variable	χ^2	df	CFI	χ^2	df	CFI	χ^2	df	CFI	χ^2	df	CFI	Invariance type
Leadership behaviors
Goal setting	7.479	5	0.998	14.587	8	0.995	21.555	10	0.991	25.263	13	0.990	Strict^e
Performance feedback	67.875	15	0.975	72.132	19	0.974	72.510	22	0.976	79.145	26	0.974	Strict^e
Value-based performance feedback	63.803	15	0.977	70.375	19	0.976	71.387	22	0.977	72.992	26	0.978	Strict^e
Consequential listening	12.218	5	0.994	13.342	8	0.996	15.410	10	0.996	19.010	13	0.995	Strict^e
Leadership performance
Leader effectiveness	144.565	15	0.940	147.205	19	0.940	147.386	22	0.942	159.170	26	0.938	Strict^e
Employee engagement	2.868	5	1.000	5.120	8	1.000	5.751	10	1.000	7.481	13	1.000	Strict^e

^aNo parameters constrained; ^bFactor loadings constrained to be equal across time; ^cFactor loadings and intercepts constrained to be equal across time; ^dFactor loadings, intercepts, and error terms constrained to be equal across time; ^eCFI was not reduced by more than .01 in any step toward increasingly parsimonious models.

Appendix C:

Parameter Estimates and Standard Errors from the Multilevel Models of all Dependent Variables by Time, Experimental Conditions, and the Interaction Between Time and Experimental Condition

Table C1. Fixed Effects.

	Fixed effects
	Intercept		Time^a		Condition^b		Time* Condition^c
Dependent variable	par.	SE	par.	SE	par.	SE	par.	SE
Leadership behaviors
Goal setting	5.01	0.15	–0.04	0.08	0.05	0.21	0.25	0.12
Performance feedback	4.63	0.16	–0.08	0.11	0.20	0.23	0.28	0.15
Value-based performance feedback	4.88	0.18	–0.05	0.09	0.08	0.26	0.33	0.13
Consequential listening	5.16	0.16	–0.09	0.10	0.29	0.23	0.29	0.14
Leadership performance
Leader effectiveness	5.41	0.15	–0.12	0.08	0.22	0.21	0.23	0.11
Employee engagement	4.97	0.16	–0.17	0.11	0.21	0.23	0.35	0.15

Number of employees in experimental group = 125; number of employees in control group = 131; par. = parameter estimates (p < .05 in bold).

^aTime = post-intervention; ^bExperimental condition = experimental group; ^cTime = post-intervention and experimental condition = experimental group.

Table C2. Random Effects and Model Fit.

	Random effects
	Intercept^a		Slope^e		Var. t1^f		Var. t2^g		Cov. t1 – t2^h		Model fit
Dependent variable	par.	SE	par.	SE	par.	SE	par.	SE	par.	SE	Deviance(log likelihood)
Leadership behaviors
Goal setting	0.27	0.10	0.01	0.02	1.28	0.13	1.09	0.10	0.80	0.10	1427.59
Performance feedback	0.27	0.10	0.03	0.03	1.63	0.16	1.45	0.14	0.95	0.12	1595.16
Value-based performance feedback	0.42	0.14	0.00	0.02	1.71	0.16	1.55	0.15	1.11	0.13	1586.54
Consequential listening	0.26	0.11	0.02	0.03	1.54	0.14	1.38	0.14	0.93	0.12	1558.02
Leadership performance
Leader effectiveness	0.31	0.10	0.02	0.01	0.95	0.09	0.85	0.08	0.63	0.08	1290.82
Employee engagement	0.32	0.11	0.04	0.03	1.31	0.13	1.16	0.11	0.77	0.10	1500.24

Number of employees in experimental group = 125; number of employees in control group = 131; par. = parameter estimates (p < .05 in bold); S.E. = standard error.

^aRandom intercept at level 3 (managers); ^eRandom slope at level 3 (managers); ^fPre-intervention variance at level 2 (employees); ^gPost-intervention variance at level 2 (employees); ^hCovariance between pre and post intervention.