Design of a simple rubric to peer-evaluate the teamwork skills of engineering students

Figures & data

Table 1. Summary of discipline-specific teamwork competencies.

Discipline	References	Team competencies
Business & Management	Katzenbach and Smith (2015), Varela and Mead (2018), Jackson, Sibson, and Riebe (2014), Kotlyar and Krasman (2022), Halfhill and Nielsen (2007).	Leadership or Management Skills, Accountability, Conflict resolution, Communication.
Healthcare	Olupeliyawa, Hughes, and Balasooriya (2022), Mickan and Rodger (2000), Salas et al. (2008), Raponi et al. (2023), Fernandez et al. (2008), Weaver et al. (2010), Warde, Giannitrapani, and Pearson (2020), Greilich et al. (2023), Leasure et al. (2013).	Communication, Mutual Trust, Leadership, Adaptability, Feedback, Managing Conflict, Accountability.
Psychology	Salas, Sims, and Burke (2005), Ellis et al. (2005), Salas, Burke, and Cannon-Bowers (2000), Romero-Díaz de la Guardia et al. (2022)	Conflict Resolution, Communication, Feedback, Leadership, Mutual Trust, Adaptability.
Science and engineering	Chowdhury and Murzi (2019), Tejedor Herrán et al. (2022), Nyarko and Petcovic (2023), Koh, Hong, and Seah (2014), Chaiwichit and Samart (2022), Hebles Ortiz, Alonso-Dos-Santos, and Yániz-Álvarez-de-Eulate (2022), Thite et al. (2021), Al-Hammoud et al. (2017), Ercan and Khan (2017), Hurst et al. (2016).	Conflict Management, Communication, Accountability, Feedback, Leadership, Trust, Ethics.

Table 2. DRIVE rubric.

Fundamental teamwork skills	Simplified behavioural attributes
Dispute Resolution (Conflict Management)	Possesses good negotiation skills and routinely achieves a win-win outcome
	Resolves destructive conflict constructively and maintains team cohesiveness
	Consistently adapts to the changing demands of the task
	Contributes appropriately to a healthy debate
Feedback	Provides constructive feedback without being offensive.
	Accepts the feedback in non-protective and positive way
Information Sharing (Communication)	Uses positive: (i) vocal/written tone (ii) Facial expression (iii) Body Language
	Maintains a respectful, polite, and constructive approach in communication
	Restates views to clarify if not clearly understood
	Listens intently without interrupting
Versatile Leadership	Motivates teammates by showing faith in teams’ ability to complete important tasks.
	Articulates merits of proposed ideas
	Constructively builds on others’ contributions.
	Brings inactive members into discussion.
	Helps and motivates team members complete tasks to high quality.
	Provides assistance and/or encouragement to team members.
Ethical behaviour (Accountability and Trust)	Completes all assigned tasks comprehensively by deadline.
	Earns trust of team members by timely completion and quality contributions.

Figure 1. Sample displaying the scoring criteria.

Table 3. Summary of Implementation.

Implementation	DRIVE rubric
Teamwork Activity	Case Study Analysis
Assessment Type	Mentor + Peer Marking
Data Acquired	Weekly Student Teamwork Marks
Student Cohort	202
Students per Team	5
Mode of Teaching	Online

Figure 2. Flow chart for data analysis.

Table 4. Sample covariance matrix of TA evaluation.

Sample covariance matrix	Dispute resolution	Feedback	Information sharing	Versatile leadership	Ethical behaviour
Dispute Resolution	0.155	0.151	0.136	0.174	0.151
Feedback	0.151	0.24	0.154	0.202	0.15
Information Sharing	0.136	0.154	0.267	0.208	0.144
Versatile Leadership	0.174	0.202	0.208	0.308	0.188
Ethical Behaviour	0.151	0.15	0.144	0.188	0.223

Table 5. Descriptive statistics of the DRIVE rubric.

Descriptive statistics
Fundamental teamwork skills	Information sharing		Versatile leadership		Dispute management		Ethical behaviour		Feedback
Raters	TA	Peer	TA	Peer	TA	Peer	TA	Peer	TA	Peer
N (Student Cohort)	202	202	202	202	202	202	202	202	202	202
Mean	3.331	3.798	3.14	3.71	3.299	3.665	3.568	3.825	3.282	3.786
Standard Deviation	0.517	0.312	0.555	0.375	0.393	0.403	0.472	0.286	0.489	0.286
Range	2.3	2	2.8	2.2	2.2	2.2	2.2	1.75	2.4	1.75
Minimum	1.7	1.8	1.2	1.8	1.8	1.8	1.8	2.25	1.6	2.25
Maximum	4	4	4	4	4	4	4	4	4	4

Table 6. Summary of EFA results – DRIVE rubric.

EFA statistics	DRIVE rubric
	TA Marking	Peer Marking
Cumulative variance Percentage (CV)	81.10%	77.36%
KMO Measure of Sampling Adequacy	0.878	0.892
DRIVE Factor loadings	TA Marking	Peer Marking
Dispute Management	0.946	0.827
Reflections and Feedback	0.891	0.881
Information Sharing	0.851	0.884
Versatile Leadership	0.920	0.909
Ethical Behaviour	0.892	0.895

Figure 3. DRIVE rubric CFA model – peer and TA assessment.

Table 7. CFA model fit indices.

Fit indices	Recommended value (good fit)	Obtained value (TA data)	Obtained value (peer data)
CMIN/df	≤3 (Kline 1998)	2.905	2.135
GFI	≥0.9 (Schumacker and Lomax 2016)	0.970	0.980
CFI	≥0.9 (Schumacker and Lomax 2016)	0.990	0.993
TLI	≥0.9 (Schumacker and Lomax 2016)	0.976	0.985
SRMR	≤0.05 (Schumacker and Lomax 2016)	.0182	0.0158

Table 8. CFA (TA and peer marking): standardised factor loadings, AVE & CR.

Factors	Standardised factor loadings – TA marking	Standardised factor loadings – peer marking
Dispute Resolution	0.950	0.770
Reflection Feedback	0.860	0.840
Information Sharing	0.790	0.850
Versatile Leadership	0.880	0.900
Ethical Behaviour	0.880	0.880
AVE	0.763	0.721
CR	0.941	0.920

Katzenbach, Jon R, and Douglas K. Smith. 2015. The Wisdom of Teams: Creating the High-Performance Organization. Brighton, MA: Harvard Business Review Press.

 Google Scholar

Varela, Otmar, and Esther Mead. 2018. “Teamwork Skill Assessment: Development of a Measure for Academia.” Journal of Education for Business 93 (4): 172–182. https://doi.org/10.1080/08832323.2018.1433124.

 Web of Science ®Google Scholar

Jackson, Denise, Ruth Sibson, and Linda Riebe. 2014. “Undergraduate Perceptions of the Development of Team-Working Skills.” Education+ Training 56 (1): 7–20.

 Google Scholar

Kotlyar, Igor, and Joe Krasman. 2022. “Virtual Simulation: New Method for Assessing Teamwork Skills.” International Journal of Selection and Assessment 30 (3): 344–360. https://doi.org/10.1111/ijsa.12368.

 Web of Science ®Google Scholar

Halfhill, Terry R., and Tjai M. Nielsen. 2007. “Quantifying the “Softer Side” of Management Education: An Example Using Teamwork Competencies.” Journal of Management Education 31 (1): 64–80.

 Google Scholar

Olupeliyawa, Asela M., Chris Hughes, and Chinthaka D. Balasooriya. 2022. “A Review of the Literature on Teamwork Competencies in Healthcare Practice and Training: Implications for Undergraduate Medical Education.” South-East Asian Journal of Medical Education 3 (2): 61–72. https://doi.org/10.4038/seajme.v3i2.454.

 Google Scholar

Mickan, Sharon, and Sylvia Rodger. 2000. “Characteristics of Effective Teams: A Literature Review.” Australian Health Review 23 (3): 201–208. https://doi.org/10.1071/AH000201.

 PubMedGoogle Scholar

Salas, Eduardo, Deborah DiazGranados, Sallie J. Weaver, and Heidi King. 2008. “Does Team Training Work? Principles for Health Care.” Academic Emergency Medicine 15 (11): 1002–1009. https://doi.org/10.1111/j.1553-2712.2008.00254.x.

 PubMed Web of Science ®Google Scholar

Raponi, Jayne-Marie, Erik W. Black, Caronne C. Rush, Gail Schneider Childs, and Amy V. Blue. 2023. “Dental Student Perceptions of Teamwork during a Community-Engaged Interprofessional Learning Experience.” European Journal of Dental Education 27 (1): 56–62. https://doi.org/10.1111/eje.12776.

 PubMed Web of Science ®Google Scholar

Fernandez, Rosemarie, John A. Vozenilek, Cullen B. Hegarty, Ivette Motola, Martin Reznek, Paul E. Phrampus, and Steve W.J. Kozlowski. 2008. “Developing Expert Medical Teams: Toward an Evidence-Based Approach.” Academic Emergency Medicine 15 (11): 1025–1036. https://doi.org/10.1111/j.1553-2712.2008.00232.x.

 PubMed Web of Science ®Google Scholar

Weaver, Sallie J., Michael A. Rosen, Eduardo Salas, Karyn D. Baum, and Heidi B. King. 2010. “Integrating the Science of Team Training: Guidelines for Continuing Education.” Journal of Continuing Education in the Health Professions 30 (4): 208–220. https://doi.org/10.1002/chp.20085.

 PubMed Web of Science ®Google Scholar

Warde, Carole M., Karleen F. Giannitrapani, and Marjorie L. Pearson. 2020. “Teaching Primary Care Teamwork: A Conceptual Model of Primary Care Team Performance.” The Clinical Teacher 17 (3): 249–254. https://doi.org/10.1111/tct.13037.

 PubMed Web of Science ®Google Scholar

Greilich, Philip E., Molly Kilcullen, Shannon Paquette, Elizabeth H. Lazzara, Shannon Scielzo, Jessica Hernandez, Richard Preble, Meghan Michael, Mozhdeh Sadighi, and Scott Tannenbaum. 2023. “Team FIRST Framework: Identifying Core Teamwork Competencies Critical to Interprofessional Healthcare Curricula.” Journal of Clinical and Translational Science 7 (1): e106. https://doi.org/10.1017/cts.2023.27.

 PubMedGoogle Scholar

Leasure, Emily L., Ronald R. Jones, Lauren B. Meade, Marla I. Sanger, Kris G. Thomas, Virginia P. Tilden, Judith L. Bowen, and Eric J. Warm. 2013. “There is no “I” in Teamwork in the Patient-Centered Medical Home.” Academic Medicine 88 (5): 585–592. https://doi.org/10.1097/ACM.0b013e31828b0289.

 PubMed Web of Science ®Google Scholar

Salas, Eduardo, Dana E. Sims, and C Shawn Burke. 2005. “Is there a “Big Five” in Teamwork?” Small Group Research 36 (5): 555–599. https://doi.org/10.1177/1046496405277134.

 Web of Science ®Google Scholar

Ellis, Aleksander P.J., Bradford S. Bell, Robert E. Ployhart, John R. Hollenbeck, and Daniel R. Ilgen. 2005. “An Evaluation of Generic Teamwork Skills Training with Action Teams: Effects on Cognitive and Skill-based Outcomes.” Personnel Psychology 58 (3): 641–672. https://doi.org/10.1111/j.1744-6570.2005.00617.x.

 Web of Science ®Google Scholar

Salas, Eduardo, C. Shawn Burke, and Janis A. Cannon-Bowers. 2000. “Teamwork: Emerging Principles.” International Journal of Management Reviews 2 (4): 339–356. https://doi.org/10.1111/1468-2370.00046.

 Google Scholar

Romero-Díaz de la Guardia, José Javier, Marina García-Garnica, Ramón Chacón-Cuberos, and Jorge Expósito-López. 2022. “Psychometric Validation of a Teamwork Skills Scale in a Vocational Training Context.” SAGE Open 12 (2): 21582440221103256.

 Web of Science ®Google Scholar

Chowdhury, Tahsin, and Homero Murzi. 2019. “Literature Review: Exploring Teamwork in Engineering Education.” Paper presented at the Proceedings of the 8th Research in Engineering Education Symposium, REES 2019-Making Connections, Cape Town, South Africa.

 Google Scholar

Tejedor Herrán, Blanca, Lluc Canals Casals, María Gonçalves Ageitos, Beatriz Amante García, and Marcel Macarulla Martí. 2022. “An Innovative Protocol for the Development and Evaluation of the Teamwork Competency in Virtual Environments.” Paper presented at the Proceedings from the 26th International Congress on Project Management and Engineering (Terrassa, July 2022)= Comunicaciones presentadas al XXVI Congreso Internacional de Dirección e Ingeniería de Proyectos, celebrado del 5 al 8 de julio de 2022, Terrassa, Barcelona.

 Google Scholar

Nyarko, Samuel Cornelius, and Heather L. Petcovic. 2023. “Essential Teamwork Skills: Perspectives of Environmental Geoscience Employers.” Journal of Geoscience Education 71 (1): 20–32. https://doi.org/10.1080/10899995.2022.2044665.

 Google Scholar

Koh, Elizabeth, Helen Hong, and Jimmy Seah. 2014. “An Analytic Frame and Multi-Method Approach to Measure Teamwork Competency.” Paper presented at the 2014 IEEE 14th International Conference on Advanced Learning Technologies, Athens, Greece.

 Google Scholar

Chaiwichit, Chianchana, and Swangjang Samart. 2022. “The Empirical Research of Teamwork Competency Factors and Prediction on Academic Achievement Using Machine Learning for Students in Thailand.” Перспективы науки и образования 4 (58): 540–556.

 Google Scholar

Hebles Ortiz, Melany, Manuel Alonso-Dos-Santos, and Concepción Yániz-Álvarez-de-Eulate. 2022. “Teamwork Competency Scale (TCS) from the Individual Perspective in University Students.”

 Google Scholar

Thite, Swapneel, Jayashri Ravishankar, Araceli Martinez Ortiz, and Eliathamby Ambikairajah. 2021. “Work in Progress: Review of Teaching Strategies Towards Development of a Framework for Online Teamwork.” Paper Presented at the 2021 ASEE Virtual Annual Conference Content Access, Long Beach, California.

 Google Scholar

Al-Hammoud, Rania, Ada Hurst, Andrea Prier, Mehrnaz Mostafapour, Chris Rennick, Carol Hulls, Erin Jobidon, Eugene Li, Jason Grove, and Sanjeev Bedi. 2017. “Teamwork for Engineering Students: Improving Skills through Experiential Teaching Modules.” Proceedings of the Canadian Engineering Education Association (CEEA).

 Google Scholar

Ercan, M. Fikret, and Rubaina Khan. 2017. “Teamwork as a Fundamental Skill for Engineering Graduates.” Paper presented at the 2017 IEEE 6th International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Tai Po, Hong Kong.

 Google Scholar

Hurst, Ada, Erin Jobidon, Andrea Prier, Taghi Khaniyev, Chris Rennick, Rania Al-Hammoud, Carol Hulls, Jason Grove, Samar Mohamed, and S. J. Johnson. 2016. “Towards a Multidisciplinary Teamwork Training Series for Undergraduate Engineering Students: Development and Assessment of two First-Year Workshops.” Paper Presented at the Proceedings of the American Association of Engineering Education (ASEE), New Orleans, Louisiana.

 Google Scholar

Kline, Rex B. 1998. Structural Equation Modeling. New York: Guilford.

 Google Scholar

Schumacker, E., and G. Lomax. 2016. A Beginner’s Guide to Structural Equation Modelling. 4th ed. London: Routledge.

 Google Scholar

Data availability Statement

The unidentified data can be obtained by sending a request e-mail to the corresponding author.