Evidence for measuring teachers’ core practices

Figures & data

Table 1. Coding of quality requirements.

Requirement	Aspect	Description	Score
Validity	Content	The constructs, i.e. teachers’ practices, are described in previous studies and pilots.	0 = No evidence 1 = Previous study or pilot 2 = Previous study and pilot
	Construct	The operationalisation of the constructs are described in terms of Factor-, Rasch- and/or Item Characteristic Curves (ICC)-analyses.	0 = No evidence 1 = One analysis 2 = Two or more analyses
	Criterion	The outcome of the measurement is related to the outcomes of other measurements that intend to measure the same construct.	0 = No evidence or low correlation 1 = Correlation between .70 – .80 with measurement of related construct or a theoretical argument(s) that the outcome differs from that of the measurement of an unrelated construct 2 = Correlation > .80 with measurements of related construct and multiple theoretical arguments for criterion validity
Reliability	Internal consistency	The items measure the construct consistently in terms of Cronbach’s Alpha or Kuder-Richardson analyses	0 = No evidence or low consistency score 1 = Consistency score between .70 – .80 2 = Consistency score> .80 and multiple analyses
	Stability	The instrument measures the construct consistently over time in terms of test-retest and split-half analyses.	0 = No evidence or low consistency score 1 = Consistency score between .70 – .80. 2 = Consistency score > .80 and multiple analyses
	Interrater agreement	They assessors reach sufficient agreement in terms of percentage agreement and Cohen’s Kappa.	0 = No evidence or low percentage of agreement 1 = Percentage agreement and Cohens’ Kappa .60 – .80. 2 = Percentage agreement and Cohens’ Kappa > .80 and multiple analyses.
Utility		The measurement procedure is transparent and efficient in terms of required time, experienced difficulties, and suitability for the target group.	0 = No evidence. 1 = One type or multiple types of evidence.
Impact		The consequences of the measurement are described in terms of feedback and the relation with the obtained score.	0 = No evidence. 1 = One type or multiple types of evidence

Table 2. Overview of studies and teacher practices.

Study	Teacher Practice
Ang (2005), Azigwe et al. (2016), Bear et al. (2011), Bernardo et al. (2008), Bonney et al. (2015), Casas et al. (2015), Castillo et al. (2013), den Brok et al. (2002), Fan (2012), Fenzel et al. (2009)^1, Hill (2004), Huang et al. (2006), Jacobs et al. (2013), Kiany et al. (2011), Kosir et al. (2014), Kunter et al. (2007), Kyriakides (2005), Lang et al. (2005), Martin et al. (2010)^1, Maulana et al. (2015), Maulana et al. (2012), Murray et al. (2011a), Passini et al. (2015), Ryan et al. (2011), Sakiz (2012), Scrimin et al. (2014), She et al. (2000), Siddall, et al. (2013), Skinner et al. (2008)., Thijs et al. (2012), Veldman et al. (2013), Wallace et al. (2012), Walsh et al. (2010), Zullig et al. (2014)	Classroom Climate Examples: interpersonal behaviour, school climate, communication, rule clarity, encouragement.
Al-Shabatat (2014), Azigwe et al. (2016)^1, Barton et al. (2006)^2*, Beran et al. (2009), Beswick (2005), Boardman et al. (2004)^123, Brown, G.T.L. et al. (2015)^4, Driscoll et al. (2010), Emesini (2015), Flowers et al. (2000), Goh et al. (2010), Good et al. (2015)^1, Hailaya et al. (2014)^3, Hargreaves (2014)^1, Haydn et al. (2007), Kay et al. (2008)^3*, Khourey-Bowers et al. (2005)^1, Klug et al. (2014)^2, Kyriakides et al. (2009), Kyriakides et al. (2014), Larose et al. (2009), Martinez et al. (2016), Mehta et al. (2013), Meintjes et al. (2010)^3, Murray et al. (2011), Nelson et al. (2014), Nunnery et al. (2008)^1, Opdenakker et al. (2011), Panayiotou et al. (2014), Park et al. (2014) ^1, Peters et al. (2010) ^1, Pontefract et al. (2005) ^1, Reddy et al. (2015)^1*, Reddy et al. (2013) ^1*, Robertson et al. (2013)^2, Sach (2012), Schroeder et al. (2011), Schumacher et al. (2011) ^2, Watzke (2007), Williams et al. (2007) ^2*	Didactics Examples: assessments, designing learning tasks, instruction, feedback, research-based practice.
Castillo et al.(2013), Cengiz et al. (2015)^3, Daley et al. (2015)^3, Delvaux et al. (2013), Elstad et al. (2012), Huang, et al. (2006), Huang et al.(2009), Kalk et al. (2014), Landmann (2013), Martinez et al. (2016), Neves et al. (2014), Ordu (2016), Schoeman et al. (2012), Sirin, S. R. et al. (2010)^3, Veldman et al. (2013)^2, Vanlommel et al. (2016), Yates (2007), Zurlo et al. (2013)	Professional Development Examples: professionalisation, role in the school, communication with colleagues and parents.
Allen et al. (2013)^1, Brown, E.L. et al. (2015)^3, Gitomer et al. (2014)^1*, Reese et al. (2014) ^1, Strong et al. (2011),	Combination of aforementioned practices

All studies refer to questionnaires, except when one of the following codes is given: 1 = observation; 2 = interview; 3 = performance measurements and logbooks; An asterisks indicates that two instruments are included, of which at least one is a questionnaire an another is indicated by the predisplayed code, that is: 1* = observation and questionnaire; 2* = interview and questionnaire; 3* = perf. ass. and logs, and questionnaire

Table 3. Overview of teacher practices measured in assessment instruments.

	Type of assessment instrument
Teacher practice	Questionnaire	Observation	Interview	Perf. ass. and logs	Total
Pre-lesson activities	0	0	0	1	1 (1%)
Lesson activities	69	14	4	5	92 (72%)
Post-lesson activities	18	0	3	3	24 (19%)
Combined activities	4	4	1	1	10 (8%)
Total	91 (72%)	18 (14%)	8 (6%)	10 (8%)	127

Table 4. Overview of quality of evidence per type of assessment instruments.

		Type of assessment instrument
Quality Degree of Evidence		Questionnaire	Observation	Interview	Perf. meas. logbooks	Total
Validity	No evidence	10	4	2	0	16 (12,5%)
	Medium	69	13	6	8	96 (75,5%)
	Strong	12	1	0	2	15 (12,0%)
Reliability	No evidence	23	7	6	1	37 (29%)
	Medium	65	9	2	8	84 (66%)
	Strong	3	2	0	1	6 (5%)
Utility	No	63	12	5	8	88 (69%)
	One or multiple pieces	28	6	3	2	39 (31%)
Impact	No	34	7	4	6	51 (40%)
	One or multiple pieces	57	11	4	4	76 (60%)

Table 5. Overview quality of evidence per type of assessor.

		Type of assessor
Quality Degree of evidence		Self-assessment	Pupils	Others	Multiple	Total
Validity	No evidence	7	4	5	0	16 (13%)
	Medium	45	30	20	1	96 (76%)
	Strong	4	9	2	0	15 (12%)
Reliability	No evidence	14	11	12	0	37 (29%)
	Medium	40	31	12	1	84 (66%)
	Strong	2	1	3	0	6 (5%)
Utility	No	41	27	19	1	88 (69%)
	One or multiple pieces	15	16	8	0	39 (31%)
Impact	No	26	14	11	0	51 (40%)
	One or multiple pieces	30	29	16	1	76 (60%)

Martinez, A., S. D. Mcmahon, C. Coker, C, and C. B. Keys. 2016. “Teacher Behavioral Practices: Relations to Student Risk Behaviors, Learning Barriers, and School Climate.” Psychology in the Schools 53 (8): 817–830. doi:10.1002/pits.21946.

 Web of Science ®Google Scholar

Allen, J., A. Gregory, A. Mikami, J. Lun, B. Hamre, and R. Pianta. 2013. “Observations of Effective Teacher-Student Interactions in Secondary School Classrooms: Predicting Student Achievement with the Classroom Assessment Scoring System – Secondary.” Grantee Submission 42 (1): 76.

 Google Scholar