Differential long-term effects of a language-based professional development program on instructional support quality

ABSTRACT

The present study focused on the differential effects of two German PD programs in the ECEC field. We examined whether PD programs are equally effective for all ECEC professionals or whether ECEC professionals benefit differently from these learning opportunities. An experimental pre-post-follow-up study was conducted in the field with random assignment of participants (N = 50) to one of two language-based PD programs. In addition, transfer-relevant factors (transfer motivation, self-efficacy, support from supervisors and colleagues) were taken into account to investigate whether possible differences in changes in the quality of instructional support were related to the transfer conditions. Using longitudinal multilevel analyses, we found that the PD program “Talking with Children”, which focuses on language promotion strategies, is equally effective for all participants. Even with minimal perceived support from colleagues and supervisors, strategies learned in the PD program can be implemented to improve the quality of instructional support. Our results suggest the importance of considering transfer conditions to better support ECEC professionals in implementing what they have learned in PD programs in the future.

Several studies have shown that high-quality instructional support (i.e., concept development, quality of feedback, language modeling) in early childhood settings is positively related to the development of children’s language skills (Howes et al., 2008; Mashburn et al., 2008). However, German and international studies have revealed that the average quality of emotional support and classroom organization is moderate to high, while the quality of instructional support is rather low to moderate (Eckhardt & Egert, 2017; Hu, et al., 2016; Peisner-Feinberg, et al., 2015). Given the importance of the quality of instructional support in ECEC settings, the question arises as to how it can be improved. Meta-analyses indicate a high potential for improving the quality of instructional support through professional development (hereafter: PD; Egert, et al., 2020). However, we still know relatively little about the differential effects of PD programs.

Professional development programs to improve instructional support quality

PD programs in early childhood education and care (ECEC) vary widely in terms of format, content, or delivery methods (Oberhuemer, 2012; Schachter, 2015). Variations in how PD is defined and operationalized make it difficult to compare different PD programs and thus determine which components are most effective (Buysse, Winton, & Rous, 2009). Efforts to define PD have been undertaken by National Professional Development Center on Inclusion (National Professional Development Center on Inclusion [NPDCI], 2008) to provide a framework for various trainings: “Professional development is facilitated teaching and learning experiences that are transactional and designed to support the acquisition of professional knowledge, skills, and dispositions as well as the application of this knowledge in practice” (p. 3). Key components of PD encompass the characteristics and context of learners and PD providers (“who”), the content (“what”), and the organization and facilitation of learning experiences (“how”) (Buysse, Winton, & Rous, 2009). By systematically examining these intersecting components, it may be possible to understand which PD program is most effective for whom, for what outcome, and under what conditions (Egert, Fukkink, & Eckhardt, 2018). This approach could help create a better match between supply (PD programs offered) and use. Research on transfer also addresses factors that are considered relevant to the application of learning to the work context. These factors include the characteristics of the trainees, the work environment, and the design of the training (Blume, Ford, Baldwin, & Huang, 2010; Holton, Bates, & Ruona, 2000; Kauffeld, Bates, Holton, & Mueller, 2008).

According to Egert’s Logit Model, it can be assumed that PD programs affect pedagogical quality through latent outcomes (awareness, orientation, knowledge, and competence) and manifest outcomes (job performance) among ECEC professionals (Egert & Kappauf, 2019). The effectiveness of PD programs in the ECEC field has been demonstrated in several meta-analyses (Egert, Dederer, & Fukkink, 2020; Lee & Sung, 2023; Markussen-Brown et al., 2017; Werner, Linting, Vermeer, & Van IJzendoorn, 2016). Despite the promising findings from meta-analyses, it is not possible to draw conclusions from the aggregate results about the differential effects on the quality of instructional support provided by ECEC professionals. Moreover, the study results included in the meta-analyses vary considerably, ranging from negative to positive, and the width of the confidence intervals for each outcome indicates that ECEC professionals benefit from PD programs to varying degrees (Egert, Fukkink, & Eckhardt, 2018; Fukkink & Lont, 2007).

Differential effects of PD programs on instructional support quality

Overall, there are only a few studies that address differential effects in PD programs (Lohse-Bossenz, Bahn, Busch, & Brandtner, 2022). Given this desideratum, it is warranted to focus on the question of for whom PD programs are effective, rather than on the content (“what”) and the training design (“how”). Offer-and-use-models suggest that individuals perceive, process, and use learning opportunities differently (Lipowsky, 2019). In particular, learners’ cognitive, motivational, and social prerequisites are considered important in this context, as learners participate in PD programs with different preexisting skills, knowledge, and dispositions (Lipowsky, 2019; Schachter, 2015).

Most (quasi-)experimental studies of PD programs in the ECEC field have used only teacher demographics to explain differential effects on outcomes. A study of the effects of on-site consultation found that teachers in the intervention condition, but not in the control condition, made larger gains in quality over time if they had more experience compared to less experience (Bryant et al., 2009). Similar findings were reported by Dickinson and Caswell (2007), who found that more experienced teachers were more likely to adopt new (language and literacy) practices. In contrast, Early, Maxwell, Ponder, and Pan (2017) reported stronger effects of a PD program for teachers with fewer years of education on two of the three quality measures. Other studies found no differential effects related to teacher education level or years of experience (Pianta, Mashburn et al., 2008; Zan & Donegan-Ritter, 2014). Overall, the evidence for the moderating effect of professional education or teacher experience on PD outcomes is inconsistent. Research on the influence of teacher education levels has indicated that factors beyond education alone are important in improving classroom quality (Early et al., 2007). Therefore, it seems to be a research concern to investigate additional intrapersonal characteristics of ECEC professionals (Sheridan, Edwards, Marvin, & Knoche, 2009), such as motivation and self-efficacy, as well as the work environment.

Motivational orientations of ECEC professionals

Motivational orientations are considered important for professional action because they are crucial for maintaining intentions, monitoring, and regulating educational practices over a long time (Baumert & Kunter, 2013). Based on a multidimensional understanding of motivation (Kunter, 2014), not only the quantity but also the quality of motivation needs to be accounted for. The multidimensionality of the motivation construct was addressed by Rzejak et al. (2014) in a cross-sectional study (N = 102) by assessing different qualities of teachers’ motivation to participate in PD programs. It was found that the degree of motivation for social interaction with colleagues in the context of PD and the motivation to develop one’s professionalism and teaching were positively correlated with the degree of perceived effectiveness of PD in terms of the usefulness and implementability of PD content. In particular, human resource development (HRD) research has focused on the motivational orientations associated with learning transfer. In this context, transfer motivation is defined as the “direction, intensity, and persistence of effort toward utilizing in a work setting skills and knowledge learned” (Holton, Bates, & Ruona, 2000, p. 344). A meta-analysis showed that transfer motivation, along with expectancy and instrumentality, had the greatest effect on transfer (Gegenfurter, 2011). Furthermore, a meta-analytic path analysis demonstrated that motivational variables can affect learning outcomes beyond cognitive ability (Colquitt, LePine, & Noe, 2000).

In particular, self-efficacy is considered important because these beliefs influence one’s goals as well as one’s effort and perseverance for action and thus performance, and vice versa, the perceived quality of the individual’s performance can change self-efficacy beliefs (Hoeltge, 2018; Tschannen-Moran, Hoy, & Hoy, 1998). Some research findings suggest that the self-efficacy of ECEC professionals is related to the instructional support quality. A nonexperimental study indicated that ECEC professionals with higher self-efficacy ratings were more likely to provide higher quality literacy instruction (r = .20), but not higher quality language instruction (Justice, Mashburn, Hamre, & Pianta, 2008). Comparable results were revealed in a German cross-sectional study of ECEC professionals (N = 120), which found a positive relationship between the quality of instructional support and self-efficacy beliefs (Wolstein, 2021). In contrast to these findings, Hu, Guan, LoCasale-Crouch, Yuan, and Guo (2022) found that the effect of a PD program was the strongest for those with low prior teacher efficacy scores, while it was the weakest for teachers with high scores of prior teacher efficacy. Other studies have failed to find correlations between self-efficacy and interaction quality (Guo, Piasta, Justice, & Kaderavek, 2010; Spear et al., 2018). Findings from HRD research confirm the positive relationship between (post-training) self-efficacy and the quality of implementation (Kauffeld, Bates, Holton, & Mueller, 2008) or transfer (Blume, Ford, Baldwin, & Huang, 2010).

Work environment: support from supervisor and colleagues

In addition to these individual characteristics of ECEC professionals that are considered important for transfer, the work environment in which the acquired competencies are to be implemented needs to be taken into account. On the one hand, the supervisor can play an essential role in the transfer process by encouraging the employee, providing feedback, and helping to identify appropriate situations in which to apply what has been learned (Elangovan & Karakowsky, 1999). Furthermore, it is assumed that supervisors can serve as role models and guide others in implementing high-quality practice (Whalley, 2011). On the other hand, colleagues can have a positive impact on one’s teaching practice and student achievement (Vescio, Ross, & Adams, 2008). Opportunities for professional learning can arise from conversations among colleagues discussing classroom experiences (Horn & Little, 2010) or from observing each other teaching (Supovitz, Sirinides, & May, 2010). Resa, Groeneveld, Turani, and Anders (2017) have shown in their quasi-experimental study that professional exchange within the team contributes significantly to the development of language-related process quality. The importance of the work environment was demonstrated in a study of the transfer process of an in-service PD program for teachers (N = 129). It was found that particularly good transfer conditions, including sufficient time as well as support from colleagues and supervisors, could buffer the negative effects of low motivation on transfer (Dreer, Dietrich, & Kracke, 2017). These findings are consistent with HRD research results. The meta-analysis by Blume, Ford, Baldwin, and Huang (2010) revealed a positive correlation between transfer and social support. Findings from individual survey studies contrast the supportive influence of supervisors and colleagues. They suggest that a perceived lack of collegial unity and cooperation (Stes, Clement, & Van Petegem, 2007) and low supervisor openness to new ideas (Piekarek, 2006) may be potential barriers.

Current study

Given the inconsistent findings on the differential effects of PD programs in ECEC, there is a need for high-quality studies to identify the conditions that lead to sustainable effects. Moreover, empirical knowledge on both facilitating and hindering transfer factors in the ECEC field is rather sparse. Therefore, a promising strategy is to build on findings from HRD research and transfer them to the ECEC field. Based on theoretical approaches in teacher development that refer to the same factors as in HRD research (such as the offer-and-use model by Lipowsky, (2019), which is widely accepted in Germany), it can be assumed that the mechanisms of transfer in the field of education are comparable (Dreer, Dietrich, & Kracke, 2017). This suggests that factors such as transfer motivation, self-efficacy, and support from colleagues and supervisors may predict differences in PD effects.

This study examines the differential long-term effects of the PD program “Talking with Children” (TwC; German: “Mit Kindern im Gespräch,” Kammermeyer et al., 2017) on the instructional support quality of ECEC professionals. The first hypothesis states that the development of the instructional support quality differs between the treatment conditions from pretest to posttest, as well as from pretest to follow-up test. It is assumed that there is a more favorable change over time in the intervention condition than in the comparison condition. The second hypothesis is that the changes from pretest to posttest and from pretest to follow-up test in the quality of instructional support through participation in the PD program differ depending on the support from colleagues and supervisor. In the third hypothesis, we assume that the changes from pretest to posttest and from pretest to follow-up test in the quality of instructional support through participation in the PD program differ depending on the extent of transfer motivation and self-efficacy.

Method

Design

A pre-post-follow-up field study was conducted in which ECEC professionals (N = 50) were randomly assigned to the intervention condition or the comparison condition1. In both conditions, nine days of training were provided between pretest and posttest (September 2014 to July 2015). Training sessions took place approximately once a month. In addition, both conditions received four half-day refresher sessions in the post-training year.

Intervention condition

The language-based PD approach “Talking with Children” (TwC; Kammermeyer et al., 2017) focuses on language promotion strategies to improve the quality of interactions in day care centers. The approach was developed in 2008 and has been the curriculum of the PD program for language promotion specialists in Rhineland-Palatinate (a federal state in Germany) since 2017. The PD approach consists of nine modules and 60 hours in which phases of input, practice, and reflection alternate. The aim is to support ECEC professionals in the acquisition of language promotion strategies and their application in everyday situations. Didactically, this PD approach is based on the concept of “situated learning.” This concept combines various individual didactic features (e.g., authentic learning situations, cooperative learning, reflection, and articulation) that contribute to the generation of flexible and transferable knowledge (Foelling-Albers, Hartinger, & Moertl-Hafizovic, 2004). This includes hands-on activities that allow participants to try out strategies in their pedagogical practice and to share ideas with other professionals. In addition to the training sessions, two one-hour individual coaching sessions were conducted based on videos of language promotion situations submitted by ECEC professionals. During these coaching sessions, the focus was on analyzing one’s behavior in the pedagogical setting, especially the use of strategies. These aspects were discussed together with one of the trainers and alternative courses of action were developed. This was intended to facilitate the transfer to one’s practice. Two research assistants involved in the development of the PD approach conducted all training sessions. However, the data was analyzed by the first author, who was not involved in the development of the PD approach.

Comparison condition

The comparison group received a PD approach based on a curriculum that has been mandatory in Rhineland-Palatinate since 2008 to qualify as a language promotion specialist (“Sprache – Schlüssel zur Welt;” Baltrusch et al., 2009). Until 2017, both PD programs (TwC and the alternative approach) were available. The alternative approach was broader in terms of content. The focus was on the acquisition of content knowledge, pedagogical content knowledge, and professional action competence. The training sessions were conducted by two trainers with many years of experience in this PD approach. Conventional adult education methods such as lectures and discussions were used. The contents of the two PD programs are compared in Table 1.

Table 1. Content of the PD approaches.

Alternative PD program (Baltrusch et al., 2009)

TwC (Kammermeyer et al., 2017)

Guiding principles Framework and cross-sectional tasks of early childhood education for language promotion in day care centers Perception and description of child language development processes Observation and documentation of language development Methodological-didactical knowledge for language promotion Design and implementation of language promotion sessions Documentation, evaluation, and further development of language promotion sessions Children under three years of age Reflection and presentation

Language promotion strategies: Language modeling (e.g., open-ended questions, self and parallel talk); concept development (e.g., building on children’s ideas and concepts, encouraging comparisons); feedback strategies (e.g., expansion, scaffolding) Application in key situations: Reading situations, routine situations, targeted activities, symbol and role play, spontaneous conversational opportunities Reflection on personal videos

Participants

The sample was recruited from ECEC professionals who were enrolled in the regular PD program for language promotion specialists at the largest regional training providers in Rhineland-Palatinate. All ECEC professionals agreed in advance to participate in either the regular or the alternative PD program leading to the same certificate. They were then randomly assigned to one of the two intervention groups or to the comparison group. There were 18 people in the comparison group and 16 people in each of the two intervention groups. All participants were female, except for one male participant in the intervention condition. Participants had completed training as educators or assistants, except for two individuals with academic degrees and one with missing information. Additional sociodemographic data are presented in Table 2.

Table 2. Sociodemographic data.

	Alternative PD program	Talking with Children
	Group A (n = 18)	Group B1 (n = 16)	Group B2 (n = 16)
Age	41 (11.29)	38.69 (9.9)	38.75 (9.42)
Pedagogical experience	14.72 (9.77)	12.69 (8.64)	11 (9.11)
Female/male	18/0	16/0	15/1
Multilingual yes/no	17/1	16/0	13/3
Training as state-registered educator/ academic degree/ assistant/ missing^1	14/ 2/ 1/ 1	15/ 0/ 1/ 0	14/ 0/ 2/ 0
Child-educator ratio	6.07 (1.38)	5.95 (1.34)	5.38 (0.98)
Proportion of children with a migrant background in the setting	31.59 (29.64)	26.63 (21.17)	35.46 (29.93)

Note. Academically trained professionals are still rare in Germany (about 5.3% in 2016; Autorengruppe Fachkräftebarometer, 2017). In most cases, training to become a state-registered educator consists of three years of post-secondary education at a vocational school specializing in social pedagogy. This often includes two years of study and one year of supervised internship (there are differences between federal states in Germany). Training as an assistant comprises two or three years at upper upper-secondary level vocational school (depending on previous education; see Oberhuemer, Schreyer, & Neuman, 2010).

No demographic differences were found between the two intervention groups using Fisher’s exact test and Wilcoxon test. Since these groups (B1 and B2) received the same PD program with the same materials and the same trainers, and since the two groups were comparable in terms of structural conditions (group composition within the training and the premises), these two groups were combined as the intervention condition for further calculations. The models were made more parsimonious by including only the condition variable (binary).

Measures

The Classroom Assessment Scoring System Pre-K (CLASS; Pianta, La Paro, & Hamre, 2008) was used as an observational measure of teacher-child interactions. In this study, CLASS served as the outcome assessment. The CLASS measure consists of 10 dimensions that are rated by certified observers using a 7-point Likert scale (low range: 1, 2; middle range: 3, 4, 5; high range: 6, 7) with three subscales: Emotional Support, Classroom Organization, and Instructional Support. Two certified CLASS raters viewed and coded 20-minute videotapes. The rating process was blinded, meaning that the two raters did not know whether the videos were from the pre-, post-, or follow-up data, nor whether they were from the intervention or comparison condition. 20% of the project videos were double-coded using a calibration procedure to prevent observer drift. Overall, there was 95.3% agreement in the ratings of Instructional Support. Intraclass correlations (ICCs) were calculated for the dimensions Concept Development, Language Modeling, and Quality of Feedback and ranged from 0.74–0.87, indicating high inter-rater agreement.

The German Learning Transfer System Inventory (GLTSI, Kauffeld, Bates, Holton, & Mueller, 2008) is the German version of the Learning Transfer System Inventory (LTSI) by Holton, Bates, and Ruona (2000). The GLTSI was used in this study to identify transfer-relevant factors that may be catalysts or barriers to the transfer of learning from training to the work context. The GLTSI is based on self-assessments by training participants. Ratings are given on a five-point Likert scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). A theory-driven selection of items from the GLTSI was conducted. A total of 13 items were selected from the 51 items related to supervisor support, colleague support, transfer motivation, and self-efficacy. An exploratory factor analysis (EFA) with Promax rotation was conducted to examine the dimensionality of the transfer-related factors. According to Holton, Bates, and Ruona (2000), supervisor and colleague support refers to the extent to which supervisors or colleagues “support and reinforce use of training on the job” (pp. 344–345). Motivation to transfer learning is understood as “[t]he direction, intensity, and persistence of effort toward utilizing in a work setting skills and knowledge learned” (p. 344) and self-efficacy as “[a]n individual’s general belief that he is able to change his performance when he wants to” (p. 346).

Data collection procedures

From September to November 2014, prior to the intervention, language promotion interactions were videotaped for each participant (baseline). At the end of the intervention (May to July 2015), videos were videotaped again (posttest). During this period, participants completed the questionnaire. Follow-up data (videotapes) were collected one year later (May to June 2016). In the video recordings, the ECEC professionals were free to choose a language activity with an average of five children. The considerations regarding the number of children are related to the structural conditions in Germany, where additional language promotion was often provided in small groups at that time.

Data analysis procedures

Longitudinal multilevel analyses were conducted to test the hypotheses. First, a null model was specified to examine the proportion of the variance of instructional support quality at the between-person level (Model 0). Second, we included the predictor time to see if there were differential changes in the instructional support quality (Model 1). Third, the additional predictor intervention was included (Model 2) to test the cross-level interaction between intervention and time in the next step (Model 3). In the fourth step, we included work environment factors (support from colleagues and supervisor) to test the cross-level interaction between intervention and time and work environment (Model 4). In the last step, characteristics of the ECEC professionals (self-efficacy and transfer motivation) were included as predictors to test the cross-level interaction between intervention and time and characteristics of the ECEC professionals (Model 5).

All models were analyzed in RStudio (version 2022.12.0.353; Posit team, 2022) with the nmle package (Pinheiro et al., 2023) using restricted maximum likelihood estimation (REML). REML provides less biased estimates, especially when the number of clusters is small (Hox, Moerbeek, & van de Schoot, 2018).

Results

Descriptive statistics

A total of seven ECEC professionals (14% of the original sample) left the study between pretest and follow-up test. The reasons for dropping out were changes in employment, pregnancy, and illness. Of these ECEC professionals, one was assigned to TwC and six were assigned to the comparison condition. Pretest scores on the CLASS domain Instructional Support for the seven ECEC professionals were compared with pretest scores for the 43 ECEC professionals for whom pretest, posttest and follow-up scores were available. No significant differences in pretest scores were found between participants who dropped out and those who remained in the study. These seven ECEC professionals were excluded from all further analyses.

Table 3 shows the descriptive statistics of the variable Instructional Support with its dimensions Language Modeling, Concept Development, and Feedback Quality, which were measured with the Classroom Assessment Scoring System (CLASS) by Pianta, La Paro, and Hamre (2008). It is shown that participants in the comparison condition had baseline scores on the quality of instructional support that were approximately half a point lower than participants in the intervention condition. Follow-up scores in the intervention condition were more than one scale point (on a scale of 1 to 7) higher than in the comparison condition.

Table 3. Descriptive statistics: instructional support quality.

	Comparison condition^a			Intervention condition^b
	Pre	Post	Follow-up	Pre	Post	Follow-up
Language modeling	2.17 (1.19)	2.50 (1.31)	2.67 (1.07)	2.81 (1.22)	4.16 (1.39)	4.45 (1.18)
Concept development	1.58 (0.79)	1.83 (0.83)	2.17 (0.58)	1.90 (0.79)	3.32 (1.35)	3.26 (1.03)
Feedback quality	1.67 (0.98)	2.00 (0.85)	2.17 (1.11)	2.29 (1.10)	3.68 (1.51)	3.84 (1.21)
Instructional support	1.81 (0.92)	2.11 (0.90)	2.33 (0.86)	2.33 (0.93)	3.72 (1.32)	3.85 (1.01)

Note. The means are given in the table. Standard deviations are given in parentheses. ^an = 12; ^bn = 31.

A descriptive analysis of the transfer-relevant factors shows that colleague support is in the middle range in both conditions (on a five-point Likert scale). Supervisor support for the transfer of what was learned in the PD program is slightly lower. This low level is equally noticeable in both conditions. Transfer motivation and self-efficacy are high in both conditions. There is little variance (SD < 0.7) in participants’ responses to these two variables (see Table 4).

Table 4. Descriptive statistics: transfer-relevant factors (post-intervention).

	Comparison condition^a	Intervention condition^b
Colleague support	3.44 (0.80)	3.15 (0.80)
Supervisor support	2.36 (1.27)	2.28 (0.85)
Transfer motivation	3.89 (0.64)	3.91 (0.57)
Self-efficacy	4.19 (0.45)	3.94 (0.46)

Note. The means are given in the table. Standard deviations are given in parentheses.^a n = 12; ^b n = 31.

Longitudinal effects on instructional support quality

The intraclass correlation (ICC) was calculated based on the null model. Results indicate that 18.5% of the total variance was due to between-person differences in the quality of instructional support. Therefore, a multilevel analysis was conducted to examine whether there were interindividual differences in changes in instructional support over time, with time (D1 fixed at post data point, D2 fixed at follow-up data point) as the first level predictor (Model 1). The predictors time D1 (b = 1.09, p < .001) and time D2 (b = 1.24, p < .001) were statistically significant, indicating an increase from pretest to posttest as well as an increase from pretest to follow-up test. Next, we tested for treatment effectiveness by including the predictor treatment in Model 2 and additionally including the interaction term treatment x time in Model 3. Results revealed that the main effect of treatment in Model 2 was statistically significant (b = 1.14, p < .001). The interaction effects of treatment x time D1 (b = 1.08, p < .05) and treatment x time D2 (b = 0.99, p < .05) in Model 3 were also statistically significant. These results indicate the effectiveness of the TwC approach, as the improvement from pretest to posttest and from pretest to follow-up test was greater than in the comparison condition (see Table 5).

Table 5. Multilevel modeling results: instructional support quality.

Fixed effects	Model 1		Model 2		Model 3
	Est.	SE	Est.	SE	Est.	SE
Level 1 – intrapers.
Time.D1^a	1.09***	0.23	1.09***	0.23	0.31	0.41
Time.D2^a	1.24***	0.19	1.24***	0.19	0.53	0.33
Level 2 – interpers.
Intercept	2.19***	0.14	1.36***	0.22	1.81***	0.27
Treat^b			1.14***	0.23	0.53	0.32
Cross-level-interaction
Time.D1xTreat^b					1.08*	0.48
Time.D2xTreat^b					0.99*	0.39
Random effects
Level 1 – intrapers.	0.47		0.00		0.00
Level 2 – interpers.	0.82		0.97		0.93
Slope for Time.D1^a	1.33		1.49		1.42
Slope for Time.D2^a	1.03		1.23		1.16
Model summary
AIC	414.33		646.70		762.08
BIC	442.69		674.98		795.82
LogLik	−197.16		−313.35		−369.04

Note. AIC, BIC, and LogLik are used to represent Akaike and Bayesian Information Criterion as well as log-likelihood value, respectively.

⁺indicates p < .1 * indicates p < .05. ** indicates p < .01. *** indicates p < .001.

^aTime was dummy-coded (D1: development from pretest to posttest; D2: development from pretest to follow-up test).

^bTreatment was dummy-coded.

Differential effects of the PD program by transfer conditions

In Model 4, the predictors supervisor support and colleague support were included to test whether changes from pretest to posttest and from pretest to follow-up test in the quality of instructional support from participation in PD programs differed by colleague and supervisor support. These predictors were included at Level 2 and centered around the grand mean. Results indicate a statistically significant negative interaction between time D1 and treatment and supervisor support (b = − 1.09, p < .05; see Table 6). This finding demonstrates that those who reported minimal supervisor support for transferring what they learned in PD made greater gains from pretest to posttest as a result of participating in the TwC approach, compared to those who reported higher supervisor support. The interaction terms of time D1/time D2 and treatment remained statistically significant (b = 1.25, p < .05; b = 1.15, p < .01).

Table 6. Multilevel modeling results: instructional support quality and transfer conditions.

Fixed effects	Model 4		Model 5		Model 6
	Est.	SE	Est.	SE	Est.	SE
Level 1 – intrapers.
Time.D1^a	0.16	0.43	0.32	0.47	0.15	0.48
Time.D2^a	0.38	0.33	0.55	0.37	0.38	0.35
Level 2 – interpers.
Intercept	1.83***	0.27	1.66***	0.29	1.70***	0.29
Treat^b	0.45	0.32	0.59^+	0.34	0.54	0.33
COLL^c	−0.02	0.36			−0.03	0.36
SUPERV^c	−0.37	0.23			−0.51*	0.25
TMOT^c			−0.40	0.47	−0.83^+	0.49
SELFEFF^c			0.77	0.66	0.72	0.63
Cross-level-interaction
Time.D1xTreat^b	1.25*	0.50	1.10*	0.55	1.27*	0.55
Time.D2xTreat^b	1.15**	0.38	1.06*	0.43	1.19**	0.40
Time.D1xTreat^bx COLL^c	−0.26^+	0.68			−0.33	0.71
Time.D2xTreat^bx COLL^c	−0.98	0.52			−1.10*	0.51
Time.D1xTreat^bx SUPERV^c	−1.09*	0.49			−1.32*	0.54
Time.D2xTreat^bx SUPERV^c	−0.70^+	0.38			−0.98*	0.39
Time.D1xTreat^bx TMOT^c			−0.53	0.92	−0.84	0.96
Time.D2xTreat^bx TMOT^c			−0.85	0.72	−0.84	0.70
Time.D1xTreat^bx SELFEFF^c			0.30	1.24	0.28	1.21
Time.D2xTreat^bx SELFEFF^c			0.52	0.97	0.66	0.87
Random effects
Level 1 – intrapers.	0.00		0.00		0.00
Level 2 – interpers.	0.93		0.92		0.88
Slope for Time.D1^a	1.44		1.50		1.46
Slope for Time.D2^a	1.15		1.17		1.05
Model summary
AIC	1485.93		1484.72		2215.24
BIC	1550.30		1549.09		2307.56
LogLik	−718.97		−718.36		−1071.62

Note. AIC, BIC, and LogLik are used to represent Akaike and Bayesian Information Criterion as well as log-likelihood value, respectively.

⁺indicates p < .1 * indicates p < .05. ** indicates p < .01. *** indicates p < .001.

^aTime was dummy-coded (D1: development from pretest to posttest; D2: development from pretest to follow-up test).

^bTreatment was dummy-coded.

^cThese factors were centered around the grand mean.

In Model 5, the predictors self-efficacy and transfer motivation were included to test the influence of the characteristics of ECEC professionals. These predictors were included at Level 2 and centered around the grand mean. Neither the main effect of the predictors self-efficacy and transfer motivation nor the interaction effects between these predictors and time and treatment were statistically significant. However, the interaction terms of time D1/time D2 and treatment remain statistically significant (b = 1.10, p < .05; b = 1.06, p < .05).

Model 6 included all predictors of the work environment and the characteristics of the ECEC professionals. Results revealed a statistically significant negative main effect of supervisor support (b = −0.51, p < .05). Furthermore, the interaction effects between time D1/time D2 and treatment and supervisor support were statistically significant. The negative interaction effect indicates that individuals who reported minimal supervisor support made greater gains from pretest to posttest and from pretest to follow-up test through their participation in the TwC approach than those who reported higher supervisor support (b = −1.32, p < .05; b = −0.98, p < .05; see Figure 1). The results also suggest that there was a negative interaction effect between time D2 and treatment and reported colleague support (b = −1.10, p < .05; see also Figure 1). This interaction effect was not statistically significant for time D1, i.e., the development from pretest to posttest. As in Models 4 and 5, the interactions between time D1/time D2 and treatment in Model 6 were positive and statistically significant (b = 1.27, p < .05; b = 1.19, p < .01).

Figure 1. Plots: Time, treatment, and support from colleagues/supervisor.

Determining effect sizes

The effect sizes were calculated according to Rights and Sterba (2019) based on linear models.2 Model 1 with the single predictor time explained a total of 40.5% of the variance, with 14.9% explained by the predictor time and 25.6% by random intercepts. Model 3 with the interaction effect time x treatment explained a total of 43.2% of the variance, with 33.9% explained by all predictors (time, treatment) and 9.2% by random intercepts. Model 4 with the additional predictors colleague and supervisor support explained a total of 46.6% of the variance (38.3% by predictors and 8.3% by random intercepts) and Model 5 with the additional predictors transfer motivation and self-efficacy explained a total of 44.2% of the variance (35.6% by predictors and 8.6% by random intercepts). The overall model with all predictors explained a total of 48.4% of the variance, with 40.4% explained by the predictors and 8.0% by random intercepts.

Discussion

The present study investigated the differential long-term effects of a language-based PD program for ECEC professionals on the quality of instructional support. It was examined whether the TwC approach is equally effective for all ECEC professionals and whether the conditions of the work environment and the characteristics of the ECEC professionals play a role in the transfer of the acquired competencies from the PD program to pedagogical practice. This was accomplished by adopting transfer approaches from organizational psychology and applying them to the ECEC field. The question of whether changes from pretest to posttest and from pretest to follow-up differ between treatment conditions can be answered based on the experimental design of the study. It has been shown that there are more favorable developments in the quality of instructional support in the treatment condition than in the comparison condition, both from pretest to posttest and from pretest to follow-up test. This result demonstrates the overall effectiveness of the TwC approach.

Interesting results emerged regarding the differential effects due to colleague and supervisor support. The study revealed that individuals who reported minimal supervisor support in transferring what they learned in PD made greater gains from pretest to posttest and follow-up test through participation in the TwC approach compared to those who reported higher supervisor support. A similar picture emerges for colleague support, but only for the development from pretest to posttest. This finding suggests that the acquired language promotion strategies can be implemented and increase instructional support quality even when they perceive minimal support from colleagues and supervisors. Given the assumption that interactions between ECEC professionals and children are positively influenced by team quality (Wertfein, Mueller, & Danay, 2013), the result seems rather counterintuitive. It is conceivable that professionals who perceive minimal support from colleagues or supervisors in implementing language promotion strategies may use their experiences with children as feedback that reinforces the use of these strategies. The importance of the perceived response from children to the use of strategies learned in PD has also been emphasized in qualitative studies (Kämpfe, Betz, & Kucharz, 2021). Furthermore, PD programs may provide an additional setting in which professional exchange can take place. This assumption is supported by research showing that the opportunity for social interaction is a motive for participation in PD programs (Rzejak et al., 2014). This argument is plausible, as many opportunities for exchange and collaboration were created within the TwC approach. However, exchange with colleagues and supervisors from one’s own institution was not an explicit focus of the PD approach. It should also be noted that only one ECEC professional per institution was trained, not other team members. Therefore, it is also conceivable that participants in the TwC program, especially those who perceived minimal support, did not experience dissonance from their supervisors or colleagues who used other forms of language education with children. The role of colleagues and supervisors in the implementation of acquired competencies needs to be further investigated, as support was rather low in both conditions. The identification of supportive structures is a major concern, especially considering the low quality of instructional support in the ECEC field (Eckhardt & Egert, 2017; Hu, Dieker, Yang, & Yang, 2016; Peisner-Feinberg, Schaaf, Hildebrandt, & Pan, 2015).

While interactions with treatment and time were found for colleague and supervisor support, transfer motivation and self-efficacy of ECEC professionals did not have a significant impact on PD effects. Given these findings, it can be assumed that all ECEC professionals benefit equally from the PD program, regardless of how motivated or self-efficacious they perceive themselves to be. The low variance in the self-efficacy variable was a striking finding (see Table 4). This could be because the PD program increased the self-efficacy of all professionals, and therefore there was little difference between the professionals after the PD program. The assumption of a reciprocal effect, i.e., self-efficacy as a cause and consequence of the PD program, is supported by research findings (Holzberger, Philipp, & Kunter, 2013). A reciprocal effect could not be investigated in the present study because self-efficacy and transfer motivation were measured only once (after the training). However, it can be argued that, in contrast to the context effects, the motivational characteristics of the professionals are less stable and more subject to change.

Overall, the results indicate that the TwC approach is effective and that language promotion strategies can be implemented even with minimal perceived support from colleagues and supervisors. This finding is remarkable because the implementation of language education integrated into daily routines seems to be more challenging than additive language education programs, which are generally based on pre-prepared materials and a manual. In contrast, the creation of spontaneous language stimuli is fundamental to integrated language interventions (Resa, Groeneveld, Turani, & Anders, 2017). This includes, for example, asking open-ended questions, self- and parallel-talk, and communication extensions (Kammermeyer et al., 2017). Given the high demands on ECEC professionals to implement child-centered approaches (Walsh, McGuinness, Sproule, & Trew, 2010), high-quality in-service PD programs have proven to be particularly effective interventions for implementing such approaches. The results of this study, which emphasize the importance of PD programs for improving the quality of instructional support, are consistent with recent findings from various meta-analyses of PD programs from different nations (Egert, Dederer, & Fukkink, 2020; Lee & Sung, 2023; Markussen-Brown et al., 2017; Werner, Linting, Vermeer, & Van IJzendoorn, 2016).

Study limitations

When interpreting the results, the limited sample size (N = 50) must be taken into account. It is therefore difficult to draw conclusions about the population. However, the present study is characterized by an experimental study design, which has a high ecological validity due to the field research approach. Furthermore, it remains unclear how the two hours of individual coaching affected the development of ECEC professionals’ instructional support. Since there was no experimental variation of PD formats (e.g., coursework, coaching), it is not possible to assess the contribution of individual coaching in terms of effects in the present study. These aspects will be addressed in a current research project on improving the quality of instructional support through different PD formats using the TwC approach by further investigating the promising results on a broader empirical basis. Another limitation is the one-time measurement of transfer factors, which does not adequately account for the potential interaction between the PD program and the transfer factors (especially motivation and self-efficacy). Contextual factors, i.e., support from supervisors and colleagues, are likely to be more stable. In this context, it remains unclear how initial conditions in terms of participants’ transfer factors affect the implementation of language promotion strategies in pedagogical practice.

Conclusions

The present study investigated the differential long-term effects of the TwC approach. Particular attention was paid to transfer-relevant factors, taking into account both the characteristics of ECEC professionals and the work environment. The results suggest that (1) the TwC approach is equally effective for all professionals, and (2) the strategies learned in the PD program can be implemented even with minimal perceived support from colleagues and supervisors. It is noteworthy that motivational factors (transfer motivation and self-efficacy) were not found to be significant. This finding should be further investigated in larger studies.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The work was supported by the Deutsche Forschungsgemeinschaft [252333091].

Notes

1. All study participants provided informed consent, and the study design was approved by the appropriate ethics review board.

2. Calculation of effect sizes based on the contrast models (where time was fixed at the post- or follow-up data point) was not possible because there was no level 1 residual. For linear models 1 to 6, the same predictors were used with the same centering methods as in the contrast models.