Integrating coding across the curriculum: a scoping review

ABSTRACT

Background and context

Coding and computational thinking are often taught through integrated curricula, despite a paucity of classroom-based research on their effectiveness.

Objective

To investigate evidence of learning resulting from cross-curricular coding tasks in middle-school classrooms, and the school environment factors that impact upon this.

Method

This scoping review synthesises recent empirical research on classroom-based integrated coding curricula in middle schools, and analyses the nature of student learning reported in the studies.

Findings

By analysing the way computational thinking has been operationalised, we contend that it conceptually conflates a range of learning outcomes. The analysis also reveals that the quality of student learning is subject to teacher knowledge and pedagogy, which are in turn heavily influenced by factors from multiple levels of school ecosystems.

Implications

Future research into integrated coding curricula should address specific outcomes in computer science, in the integrated subject, and general competencies, and consider school ecosystem factors.

KEYWORDS:

• Interdisciplinary projects
• pedagogical issues
• coding
• computational thinking
• school environment

1. Introduction

Following the popularisation of computational thinking (CT) as a 21^st century literacy for all (Wing, 2006), CT and coding are becoming increasingly common in school curricula around the world. Furthermore, it is estimated that one third of teachers are teaching coding by integrating it into other subjects (Rich et al., 2019). However, there is a paucity of empirical research on school-based cross-curricular learning of coding. Moreover, there has been no known work to date to synthesise the research relating to integrated coding curricula in schools. This is surprising given its global prevalence (Rich et al., 2019), especially considering countries such as Finland and some states in Australia indicate coding and CT should be integrated into every subject area (Mannila, 2023; NESA, 2017). This review provides an overview of classroom-based research related to integrating coding across the curriculum, synthesises the evidence of learning that resulted from these empirical studies, and documents school factors that influenced students’ learning in these environments.

2. Research questions

Literature selection and analysis were guided by the following questions:

1. What research has been conducted on integrating coding across the curriculum in middle schools?

2. What learning outcomes were evident from integrating coding across the curriculum in classroom-based studies?

3. What school environment factors are influential for students learning to code in non-computing/technology subjects?

3. Related work

Coding has been traditionally taught in single-subject contexts – usually in computer science (CS), but there has been suggestions since the 1970s that learning programming (e.g. using LOGO) may help students develop general problem-solving and thinking skills (Maddux & Johnson, 1997). Early work was pioneered by Seymour Papert at the MIT Media Lab who first coined the term computational thinking, which he used to describe the kind of thinking that is shaped and enhanced by the use of computers (Papert, 1980). Interest in programming waned in the following decade as schools became more interested in drill and practice programs and other educational software (Maddux & Johnson, 1997), but was revived through Wing’s (2006) influential work on computational thinking as an essential 21^st century literacy for all. Since then, much work has been undertaken internationally to develop CS standards and curriculum in K-12 education (Franklin et al., 2017; Grover et al., 2015). This has also been supported by the development of block-based programming languages such as Scratch and Blockly (Resnick et al., 2009; Trower & Gray, 2015), and much research has been conducted to ascertain how students may learn computational thinking through coding (Zhang & Nouri, 2019).

However, the single subject approach and technical emphasis reflected in CS education is not appealing to all educators (Barr & Stephenson, 2011). With Wing’s (2006) claim of universality of CT, many believe that CT should not be restricted to teaching within CS for vocational goals, and advocate for it to be taught as a transdisciplinary, 21^st century skill in the context of other subject areas (e.g. Li et al., 2020; Yadav et al., 2016). Whilst early research suggested that coding “expose[s] students to computational thinking” (Lye & Koh, 2014, p. 51) through students’ engagement in computational concepts, practices and perspectives (Brennan & Resnick, 2012), more recently, Resnick and Rusk (2020) have departed from their earlier stance on developing CT through coding. Instead, they advocate that coding should be used to teach computational fluency so students can learn to “think creatively, work collaboratively, and reason systematically” (Resnick & Rusk, 2020, p. 120). They argue that these benefits are more important for 21^st century learning than mastering technical skills (which CT implies). From their early success in engaging urban youth and online communities with coding and creative media and maker projects at computer clubhouses (see Kafai & Burke, 2014; Maloney et al., 2008; Stager & Martinez, 2018), Resnick and Rusk (2020) claim that coding creative projects can lead to interdisciplinary learning and general competencies, and should be incorporated into formal schooling:

Students [are] not just learning to code, they are coding to learn. They are not only learning important mathematical and computational concepts, they are deepening their understanding of ideas in other disciplines and developing a broad range of problem-solving, design, collaboration, and communication skills. (p. 121)

This suggests that coding does not only benefit learning in CS but also in other integrated subjects, as well as the development of general competencies.

Correspondingly, there is increasing international interest in integrating coding across the curriculum. Some countries like Finland even opt for an integrated approach over a single-subject approach to teaching CT and coding. Kim et al. (2023) suggested that integrated coding projects may serve as ways for students to present their ideas and thoughts. Mannila (2023) noted that integrating coding in other subjects at primary level may utilise coding as tools (e.g. for creating animations and geometric art), as glue (to create interactive artefacts such as those that make use of Makey Makey), and for ideation (i.e. understanding impact of programming in students lives and future potentials).

According to Rich et al. (2019) global survey, 30% of teachers are teaching coding in an integrated subject, with the most common subjects for integration being mathematics and science, followed by language arts. There is also interest in the academic community to connect CT with different subject areas. For example, Weintrop et al. (2016) defined CT for mathematics and science classrooms in alignment with the Next Generation Science Standards in the US, and contend that there is a reciprocal relationship between CT and learning in science and mathematics. Others have explored integrating coding into non-STEM areas, including English (Burke, 2017), social studies (von Wangenheim et al., 2017), and music (Burnard et al., 2017).

These early integration studies have also raised implementation challenges relating to school contextual issues such as classroom, curriculum and school organisation arrangements (Burke, 2017; Burnard et al., 2017; von Wangenheim et al., 2017). These issues reflect systemic influences that have also been reported relating to integrated STEM education in schools. Falloon et al. (2022) noted that these influences may occur at the micro (student, teacher, and classroom), meso (school organisation, staffing, and resources), exo (curriculum, assessment, and reporting), and macro (national politics and ideological influences) levels. It is conceivable that similar influences may impact upon students learning coding in cross-curricular contexts in schools.

With the increasing popularity of an integrated approach, there is a growing need to clarify its purpose, the relationship between coding and computational thinking, and their role in facilitating learning in the integrated subject and general competencies in formal education contexts. The goal of this scoping review is to investigate empirical evidence of learning that results from cross-curricular coding tasks in middle-school classrooms, and the school environment factors that impact upon learning outcomes. The middle school was chosen as the focus for this review, because, as noted by Beane (1992), integrated curricula are particularly relevant to middle schools, where developmentally, adolescents should experience a general education that equips them to participate in the wider world before they begin specialised studies in separate subjects in secondary school. According to Peters et al. (2020), scoping reviews are conducted to “provide an overview of the evidence or to answer questions regarding the nature and diversity of the evidence/knowledge available” (p. 408). In doing so, we aim to provide a broad overview of the learning that is evident from integrated coding curricula.

4. Methods

Scoping reviews are useful for examining how research is conducted on a topic and for identifying the types of evidence available in a field (Peters et al., 2020). Unlike systematic reviews which seek to inform or provide guidelines and recommendations, scoping reviews are particularly suitable for examining emerging evidence when the body of literature is heterogeneous (Peters et al., 2020; Tricco et al., 2016).

4.1. Inclusion criteria

In this review we follow the protocol outlined by Peters et al. (2020), starting with defining questions and search strategies based on their PCC elements (population, concept, and context). The population of interest was middle-school students, defined as Grades 5–8 (Beane, 1993). The concept of interest was coding in cross-curricular learning (Barnes, 2015) where students are coding or programming in a subject area other than computer science or technology, and the contexts were classrooms in regular schools. We were interested only in studies that involved an intervention conducted within classroom contexts because the school environment poses limitations on teachers, which ultimately influences students’ learning outcomes (Falloon et al., 2022). Because we were interested in evidence of learning, we included only empirical studies that collected and analysed student data.

4.2. Search strategy

Following Peters et al. (2020) recommendation, we intended the search strategy to be as comprehensive as possible within the constraints of time and resources. An initial search was conducted on ERIC and ACM Digital Library using the inclusion criteria as a basis for keywords. This initial search was followed by an analysis of the subject keywords and text contained in the title and abstract of the retrieved papers. In consultation with a research librarian, a revised search strategy was developed to use all identified keywords and index terms across five databases, namely, ACM Digital Library, ERIC, Scopus, PsycInfo, and EBSCOHost/Academic Search Premier. The final search strategy comprised three components, incorporating the concepts and population of interest:

(programming OR coding) AND

(“integrated curriculum” or “curriculum integration” or “cross-curricul*” or “interdisciplinary”) AND

(“middle school” OR “grade 5” OR “grade 6” or “grade 7” or “grade 8”)

The classroom context was not specified in the search because of difficulty defining precise keywords without eliminating potential studies of interest. Therefore, we decided to exclude non-classroom research at the manual screening stage. The search was conducted on 30 April 2023. Considering the limitation of time and resources, we determined that only full-text, English articles published in the last 5 years would be included.

4.3. Study selection process

Search results were imported into EndNote and duplicate entries were removed. For the remaining entries, the first author screened the titles and abstracts for relevance. After the first round of screening, full texts of the remaining articles were manually reviewed, and articles that did not address the PCC elements, i.e. middle-school students engaged in integrated coding curriculum in regular classrooms, were excluded.

4.4. Data extraction

For the articles selected for inclusion in the review, we abstracted data on study characteristics (e.g. subject of integration, methodology, number of students, duration of classroom intervention, coding language used, country), types of data collected, outcomes related to computer science and the other subject area, general competencies, and any teacher practice or school environmental factors that influenced students’ learning outcomes.

4.5. Synthesis

The synthesis included quantitative analysis (e.g. frequency analysis) of the study characteristics and qualitative analysis of learning outcomes and ecological system factors detailed in the studies.

5. Results

5.1. RQ1. What research has been conducted on cross-curricular approaches to coding in middle schools?

5.1.1. Literature search

The literature search resulted in 569 articles (see Figure 1). The citations were downloaded into EndNote, which identified and removed 8 duplicates. The first author screened all titles and abstracts and excluded 434 articles that were not empirical studies, not related to student learning, or not related to middle schools. For the remaining 130 articles, the full texts were downloaded and further manually screened for eligibility. Three additional references were identified by scanning reference lists and consulting other sources. Studies in which student data were not collected, that did not identify a second subject area, that did not include students in grades 5 to 8 as participants, or were not conducted in a school classroom, were excluded. This resulted in 21 studies being included for full analysis. A summary of the articles is provided in Table 1.

Figure 1. Flow diagram for the scoping review process adapted from the PRISMA statement by Moher et al. (2015).

Table 1. Studies included in the scoping review.

First Author	Year	Subject of integration	Method(s)	Number of students (classes/sites)	Length of intervention/class time	Coding language	Country
Ahmed	2019	Mathematics	Interviews	44	Unspecified	Scratch	Sweden
Atlin	2021	Mathematics	Quasi-experiment	188	10 weeks	BASIC	Turkey
Arawjo	2021	Intercultural education	Ethnography	2 classes (US) + 24 youths (Kenya)	5 weeks	Scratch	US, Kenya
Biddy	2021	Science	Design-based research	293	3 weeks	MakeCode	US
Boylan	2018	Mathematics	Randomised control trial	6232 (110 schools)	12+ hours/2 years	Scratch	UK
Celepkolu	2020	Science	Case study/mixed method	75	14 days	Snap	US
Gannon	2022	Mathematics	Design-based research	4 classes	12 modules	Scratch	US
Gautam	2020	Science	Case study/mixed method	21	5 x 50 min	NetLogo	US
Hsu, Chang, Wong, Aw	2022	Mandarin as a second language	Quasi-experiment	46	2 lessons (1 for Mandarin and 1 for CT integration)	(Unspecified)	Singapore
Hsu, Chang, Wu, Looi	2022	Chinese as a second language; English as a foreign language	Quasi-experiment	32	4 hours x 3 weeks	(Unspecified)	Singapore, Taiwan
Krakowski	2022	Science	Design-based research	478 (13 classes)	10 lessons/8 hours unit	Data Studio VPE	US
First author	Year	Subject of integration	Methodology	Number of students	Length of intervention/class time	Coding language	Country
Ma	2021	Chinese, mathematics, music, and art	Quasi-experiment	63	3 months	Scratch	China
Mannila	2023	STEAM	Case study/Mixed method	71	20 hours	Scratch	Finland
Petrie	2022a	Music	Case study/Mixed method	22	6 part lesson	Sonic Pi	New Zealand
Petrie	2022b	Music	Case study/Mixed method	22	6 x 90 min	Sonic Pi	New Zealand
Rodriguez-Martinez	2020	Mathematics	Quasi-experiment	47 (2 classes)	5 x 1 hour (CT) + 3 x 1 hr (Mathematics)	Scratch	Spain
Shaw	2021	Biology, engineering	Design-based research	196 (4 sites, 12 classrooms)	20 lessons	MakeCode	US
Spieler	2018	Physics, CS, arts, music, English	Quasi-experiment	478 (281 girls, 197 boys)	Varied	PocketCode	Austria
Tenhovirta	2022	STEAM	Educational design experiment	15	8–9 x 90-135min	GoGo Board	Finland
Woo	2022	English	Case study/mixed method	16 (2 classrooms)	7 x 80 min	Scratch	Australia
Xu	2019	Science	Case study/mixed method	16 (2 classrooms)	10 x 1-hour lessons	Python	US

5.1.2. Study characteristics

Tables 2 and 3 summarise the key study characteristics.

Table 2. Study characteristics.

Study characteristics (N = 21)		Count
Subject of integration
	Science/Engineering/STEAM	8 (38%)
	Mathematics	5 (24%)
	Language and Creative Arts	5 (24%)
	Multiple	(10%)
	Other (intercultural education)	1 (5%)
Continents
	Europe	7 (33%)
	North America	7 (33%)
	Asia	3 (14%)
	Australia/New Zealand	3 (14%)
	Multiple continents	1 (5%)
Coding language
	Block-based programming language	16 (76%)
	Scratch	8 (38%)
	MakeCode	2 (10%)
	Data Studio	1 (5%)
	GoGo Board	1 (5%)
	PocketCode	1 (5%)
	Snap	1 (5%)
	(Unspecified)	2 (10%)
	Text-based programming language	5 (24%)
	Sonic Pi	2 (10%)^a
	BASIC	1 (5%)
	NetLogo	1 (5%)
	Python	1 (5%)

^aThe two uses of Sonic Pi were two articles from the same intervention (Petrie, 2022a, 2022b).

Table 3. Research methods and number of student participants.

	Number of student participants
Method	1–30	31–99	100–199	200–999	1,000+	Total
Case study/mixed method	5	2				7
Ethnography		1				1
Interviews		1				1
Design-based research/ educational design experiments	1	1	1	2		5
Quasi-experiment		4	1	1		6
Randomised control trial					1	1
Total	6	9	2	3	1	21

5.1.2.1. Subject of integration

Of the 21 included studies, eight (38%) were integrated with science, engineering, or STEAM (integrated science, technology, engineering, arts, and mathematics). This was followed by mathematics (n = 5, 24%) and language and creative arts (n = 5, 24%). Two studies mentioned multiple subject areas (Ma et al., 2021; Spieler, 2018), and one was integrated with intercultural education (Arawjo & Mogos, 2021).

5.1.2.2. Location

One-third of the studies were conducted in Europe (n = 7, 33%) and another third in North America (n = 7, 33%). Several studies were conducted in Asia (n = 3, 14%) and Australia/New Zealand (n = 3, 14%). One study was conducted across North America and Africa.

5.1.2.3. Programming language

Approximately three quarters of the studies used block-based programming language (n = 16; 76%), with Scratch emerging as the most popular language (n = 8, 38%). One quarter of the studies were conducted with text-based programming languages (n = 5, 24%).

5.1.2.4. Length of intervention/class time

It is difficult to provide descriptive statistics for length of intervention because of the inconsistent way this is reported in the articles. However, the shortest intervention took place across two lessons (Hsu, Chang, Wong, et al., 2022) and the longest intervention spanned 2 years (Boylan et al., 2018). For the studies that reported class time, the minimum was 4 hours in total (Gautam et al., 2020) and the maximum in multiple studies was approximately 20 hours (Boylan et al., 2018; Mannila et al., 2023; Tenhovirta et al., 2022).

5.1.2.5. Methods and number of participants

Table 3 summarises the research methods and number of participants in the selected studies. Mixed methods/case studies (n = 7; 33%) and quasi-experiments (n = 6; 29%) were the most common methods used, and most studies included fewer than 100 students (n = 15; 71%). The largest study was conducted by Boylan et al. (2018), who evaluated the ScratchMaths project via a randomised control trial involving 6,232 students in 110 schools across seven districts in the UK.

5.2. RQ2. What kinds of learning outcomes were evident from integrating coding across the curriculum in classroom-based studies?

We analysed the types of data collected (Table 4) and the research outcomes, as stated in the published articles. Teacher and classroom data provided contextual information about the interventions, and will be addressed in RQ3. Student data included coding artefacts (multimedia projects, maker projects and robotics models), process data (screen-recordings), student self-reports (interviews and reflection/dairies), and structured questionnaires. Nearly all studies made use of questionnaires or interviews as key evidence of learning, with the one notable exception being Woo and Falloon (2022), who collected screen recordings to analyse students’ problem solving approaches in relation to CT.

Table 4. Data collection for integrated coding curriculum.

Types of data collected (N = 21)		Count
Teacher
	Interview	5 (24%)
	Reflection/diary entries	3 (14%)
	Observation of professional development	1 (5%)
Classroom
	Video recordings and photos	4 (19%)
	Observation	3 (14%)
	Field notes	3 (14%)
Student coding artefacts
	Multimedia project (animation, music, games)	6 (29%)
	Maker project	2 (10%)
	Robotics model	1 (5%)
Student process data
	Screen-recordings	1 (5%)
Student unstructured/semi-structured
	Interviews	8 (38%)
	Reflection/Diary	2 (10%)
Student structured questionnaires
	Computational thinking (Type 1)	7 (33%)
	Attitude and self-efficacy to CS	6 (29%)
	Knowledge of integrated subject	6 (29%)
	Attitude to the integrated subject	2 (10%)
	Computational thinking (Type 2)	2 (10%)
	Computational thinking (Type 3)	2 (10%)
	Evaluation of intervention	3 (14%)

The most frequently cited purpose for using questionnaires was to ascertain students’ attainment of CT (n = 11, 52%), but how computational thinking was operationalised in the questionnaires varied considerably. We identified three types of computational thinking questionnaires: Type 1 includes questions requiring students to read code and predict the program outcomes (e.g. the Computational Thinking test, Román-González et al., 2017) (n = 7, 33%); Type 2 includes questions based on computer science concepts, but do not require students to be able to read code (e.g. Snow et al., 2017) (n = 2, 10%); Type 3 includes questions relating to 21^st century skills and competencies, without reference to computer science concepts or needing students to read code (e.g. Korkmaz et al., 2017) (n = 2, 10%). Other questionnaire types evaluated students’ attitude and self-efficacy towards CS; knowledge about the integrated subject; attitude to the other subject; and students’ evaluation of the intervention.

Table 5 summarises the articles grouped by the subject of integration, and categorises the research focus of each article based on computer science knowledge and skills and attitudes, subject of integration knowledge and skills and attitudes, and competencies and other outcomes. Studies which report computational thinking as an outcome are categorised based on the instruments they used for collecting evidence for CT – that is, studies that made use of Types 1 and 2 questionnaires for CT are classified under “computer science knowledge and skill”, and those that used Type 3 questionnaires examining CT are classified under “competencies and other outcomes”.

Table 5. Studies by subject of integration and the outcomes of interest.

			Computer science		Competencies	Integrated subject
Subject of integration	First author	Year	Knowledge & skill	Attitude	& other outcomes	Knowledge & skill	Attitude
Mathematics	Ahmed	2019		x
	Altin	2021	x	x
	Boylan	2018	x			x
	Gannon	2022	x	x		x	x
	Rodriguez-Martinez	2020	x			x
Science/ STEAM	Biddy	2021					x
	Celepkolu	2020	x	x		x	x
	Gautam	2020	x			x
	Krakowski	2022	x			x	x
	Mannila	2023	x		x
	Shaw	2021				x
	Tenhovirta	2022			x
	Xu	2019	x			x
Language & creative arts	Woo	2022	x		x	x
	Hsu, Chang, Wong, Aw	2022		x		x
	Hsu, Chang, Wu, Looi	2022			x	x
	Petrie	2022a		x
	Petrie	2022b	x
Multiple/ other	Arawjo	2021			x
	Ma	2021	x	x	x
	Spieler	2018	x			x

5.2.1. Computer science outcomes

Of the 21 studies selected, 12 investigated and reported outcomes related to computer science knowledge and skills. Most studies simply reported an overall positive change in CT or CS scores (Altin et al., 2021; Boylan et al., 2018; Celepkolu et al., 2020; Rodríguez-Martínez et al., 2020; Spieler, 2018), but several qualitative studies were concerned with understanding of specific CS concepts. For example, Woo and Falloon (2022) studied students’ creation of coded animated narratives in Scratch. They found that multi-scene animations rely heavily on the use of the concept of concurrency (also known as parallelism and synchronisation), and their study revealed how students’ varied understanding of the concept influenced their problem-solving practices. Similarly, Petrie (2022a) studied students’ use of Sonic Pi for music composition and found the relevant concepts were sequences, loops, parallelism, and data, but not conditionals and operators. In both studies, the researchers noted a tendency for students to use trial and error as a strategy for coding, pointing to a limited understanding of relevant CS concepts as the principal issue. The same tendency for trial and error was noted by Hsu, Chang, Wu, et al. (2022) who studied students’ coding of social robots for language learning. Additionally, in Xu’s (2019) study where students were coding a smart tabletop greenhouse, the students were able to write code with loops and conditionals but had trouble explaining what the code meant. These studies question the depth of understanding of CS concepts even when students produced a working coding project. However, several studies demonstrated that students could improve their conceptual understanding when teachers adopted problem-based pedagogy (Ma et al., 2021), unplugged activities (Krakowski et al., 2022), and gave explicit explanations based on both the computational and scientific models of interest (Gautam et al., 2020).

The most frequently discussed CS skill was debugging, and students’ outcomes varied between studies. In studies involving maker projects and physical computing, students typically would only use a small amount of code to add music and sound (Mannila et al., 2023) or to program sensors and rotate motors (Xu, 2019). In these studies, students were able to correct their own codes. However, the results were less positive in studies where students were creating multimedia projects that had more complex coding. In Petrie’s (2022a) study, where students used the text-based programming language Sonic Pi to compose music – despite Petrie explicitly teaching students the debugging process, they were unable to articulate understanding of that process after the intervention. In Woo and Falloon’s (2022) study where students were coding animated narratives, the students often bypassed challenging problems instead of engaging in debugging, due to the open-ended nature of the task which allowed them to pursue alternative creative solutions or modify their intended outcome away from what was originally planned. Results from these studies suggest that students’ adequacy of debugging skills may vary depending on the complexity of the coding project.

Of the seven studies in which students’ sense of self-efficacy and attitude toward CS were investigated, five indicated an overall positive change after the integrated unit of study (Altin et al., 2021; Celepkolu et al., 2020; Hsu, Chang, Wu, et al., 2022; Ma et al., 2021; Petrie, 2022b). However, students’ perception of programming was heavily influenced by surface features of the language they were exposed to (Ahmed et al., 2019), and their experiences varied significantly between classes because of teacher delivery (Gannon et al., 2022).

5.2.2. Subject-specific outcomes

5.2.2.1. Mathematics

Of the five studies that integrated coding with mathematics, only two studies investigated students’ mathematical knowledge, and a third investigated students’ attitudes toward mathematics. All three studies utilised Scratch. Although some improvement in mathematical knowledge was found in a small-scale quasi-experiment between two classrooms (Rodríguez-Martínez et al., 2020), a large-scale randomised control trial by Boylan et al. (2018) found no evidence that integrating coding enhanced learning outcomes in mathematics compared with traditional methods. Moreover, contrary to claims that integrating coding would make learning mathematics more meaningful, Gannon et al. (2022) noted that many students did not understand why they were using coding to solve mathematics problems, and achievement-oriented students found it difficult to start the task when they were asked to simply “explore”.

5.2.2.2. Science/STEAM

Eight studies were conducted where coding was integrated into science/engineering or STEAM. These studies can be further categorised into two types: those that used coding for data visualisation, and those that used it in maker and robotics projects.

All studies exploring coding for data visualisation were conducted in the US, and a range of languages were used for this purpose: Snap, NetLogo, Data Studio, and Python (Biddy et al., 2021; Celepkolu et al., 2020; Gautam et al., 2020; Krakowski et al., 2022). Krakowski et al. (2022) engaged students with a big-data analysis method using locally relevant data. By telling the story of a single datum, they ensured that students could reason about how data manipulation relates to visualisation. Their strong conceptual scaffolding resulted in high student engagement, with students reporting “productive struggle” (p. 40) in their endeavours. Deeper scientific understanding was also reported by Gautam et al. (2020), who analysed students’ discourse and attributed the positive outcomes to the multiple representations enabled by coding, and teachers’ explicit explanations. However, in other studies, students have reported that the relationship between coding and scientific concepts is not obvious (Celepkolu et al., 2020).

Four studies utilised coding as a component of maker or robotics projects (Mannila et al., 2023; Shaw et al., 2021; Tenhovirta et al., 2022; Xu, 2019). For three of these, the main role of coding was to simply enable music, sound, and basic movement in the students’ models, which would have only required writing a few lines of commands, and as such, coding would have played a minor role in student learning. In Xu’s (2019) study, students built a smart greenhouse using MakeCode and utilised coding to gain precise control over experimental variables such as airflow, humidity, and temperature. Xu (2019) found that when the teacher framed the task as a scientific inquiry, students tended to explain their coding functionalities in terms of how they contributed to the inquiry. However, when another teacher taught coding as a set of instructions to be copied, then students did not connect the purpose of coding to scientific inquiry.

5.2.2.3. Language and creative arts

Two studies focused on language learning and the use of robotics (Hsu, Chang, Wong, et al., 2022; Hsu, Chang, Wu, et al., 2022). In both cases, language learning (Mandarin and/or English) comprised students’ giving verbal code commands to the robots and negotiating with their teammates about the task. Coding in these studies was used as a stimulus for conversation, and students in both studies showed significant improvement in their language learning. Woo and Falloon (2022) reported on students successfully coding animated narratives in Scratch to fulfil the state English syllabus requirements relating to learning about coherent narratives, although screen recording revealed that most students merely engaged in trial and error when solving coding problems.

5.2.2.4. Other studies

Spieler (2018) studied the integration of Pocket Code with a range of subjects (physics, computer science, arts, music, English) and defined the learning goal as transference of game mechanics (CS) and reaching the other subject knowledge goal, but she did not specify what these were. Her study was primarily concerned with inclusion of female students in CS. Importantly for the present discussion, she did not find any evidence that interdisciplinary courses presented an advantage for female students over single-subject CS courses.

5.2.3. General competencies and other outcomes

Problem solving, creativity, and collaboration are key themes addressed in studies that relate to competencies.

Through their problem-based approach to teaching coding, Ma et al. (2021) found that students reported significant improvements in both critical thinking and problem solving on the Computational Thinking Scale (CTS) (Korkmaz et al., 2017), as well as the coding-based questionnaire Computational Thinking test (CTt) (Román-González et al., 2017). This corresponds with Woo and Falloon’s (2022) study, which found that problem decomposition was only operationalised by students with deep knowledge of coding, thus challenging the notion that problem decomposition in CT is a generalised problem solving skill (Woo & Falloon, 2022). Together, these studies indicate a possible interdependence between CS conceptual knowledge and systematic problem solving capability in coding tasks. Mannila et al. (2023) presented the only study that explicitly addressed creativity. However, while the students in their study demonstrated creativity, tinkering, and debugging in their maker project, the researchers observed that these attributes were general characteristics of the whole project, rather than specifically related to coding or programming.

Collaboration was discussed in six studies. When reflecting on their maker projects, students in Mannila et al. (2023) study often mentioned collaboration in their diary entries and an appreciation for working with others, with some groups mentioning division of labour or role specialisation as specific organisational strategies they used. However, improved collaboration in other studies was less evident. In the two studies that applied CTS (Korkmaz et al., 2017), no significant differences were found in students’ reported levels of cooperation in the pre- and post-tests (Hsu, Chang, Wu, et al., 2022; Ma et al., 2021). Moreover, Spieler (2018) reported that female students were more likely to reach learning goals individually than in groups, and appeared less comfortable and less productive when working in groups. These studies problematised the often-argued relationship between coding and collaboration, suggesting that other factors influencing levels of collaboration may be at play. To enable peer teaching, Tenhovirta et al. (2022) noted that students’ existing social skills and social networks need to be considered, and if intercultural understanding is required between students, Arawjo and Mogos (2021) determined that it is important for teachers to explicitly teach social and conflict-resolution skills.

5.3. RQ3. What school environment factors are influential for students learning to code in non-computing/technology subjects?

5.3.1. Microsystem – Teacher knowledge and efficacy

Teachers’ knowledge and confidence were common concerns across studies (Gannon et al., 2022; Mannila et al., 2023; Xu, 2019). In some studies, researchers delivered classroom instruction which may have contributed to more positive results, especially for CS outcomes (Celepkolu et al., 2020; Ma et al., 2021; Petrie, 2022a, 2022b; Rodríguez-Martínez et al., 2020). On the other hand, outcomes were more varied with non-specialist teachers, because reviewed research indicated they can have different emphases in their classes (Biddy et al., 2021; Xu, 2019). On this point, Mannila et al. (2023) warned that insufficient focus on teaching the basics of programming could mean that students do not develop a broader understanding of computer science that these programs often intend to foster.

At the same time, Mannila et al. (2023) noted that giving teachers flexibility to adapt the curriculum to the local context was crucial for them being able to optimise different resources and institutional support, and ultimately for them being able to carry out the project. Thus, research suggests a fine balance exists between clarifying and ensuring the delivery of computer science outcomes, and giving teachers sufficient control and flexibility to design and teach a curriculum responsive to local needs and environments.

5.3.2. Mesosystem – School organisation, staffing, resources

Mannila et al. (2023) argued that “school leaders and other actors that can provide resources and organizational support should be included in the co-agency to further facilitate this type of cross-curricular project” (p. 14). Boylan et al. (2018) also documented the importance of school leadership to supporting curriculum innovations of this nature. They determined that implementation was enhanced when schools provided time for teachers to work through and learn from specialised training materials, and if they were supported in teaching CS concepts as part of integrated curricula, despite the pressure of high-stake national assessments. These studies suggest the important role that school leadership plays in moderating the time and professional knowledge constraints faced by teachers.

5.3.3. Exosystem – Curriculum and syllabus, assessment, reporting

As integrated coding curriculum moves from the research domain to the regular curriculum, researchers are increasingly concerned about compliance with state and national syllabus and curriculum standards (Boylan et al., 2018; Gautam et al., 2020; Mannila et al., 2023; Woo & Falloon, 2022). Countries vary in how coding and computational thinking are defined, and therefore there are differences in emphasis in studies from different countries. For example, Finland’s national curriculum requirement for programming and creative production may explain why the research on maker programs was conducted there (Mannila et al., 2023; Tenhovirta et al., 2022). In the US, the proliferation of data visualisation studies may have benefited from the work of Weintrop et al. (2016), who aligned CT with the national science standards. In contrast, Woo and Falloon’s (2022) Australian study found that students tended to use general problem-solving strategies to approach coding problems, often in the form of trial and error or opting instead for a “creative” solution, possibly because the state syllabus does not specify CS knowledge development.

5.3.4. Macrosystem – Drivers and ideological influences

The macrosystem contains drivers and ideological influences on activities in the exo and meso systems. However, none of the studies we analysed explicitly considered macrosystem influences.

6. Discussion

In reviewing the research on integrated coding curriculum, the first point of note is the paucity of empirical studies that were situated in naturalistic classroom contexts. Considering the global prevalence of curriculum integration, research providing empirical evidence of learning through coding across the curriculum is comparatively scarce. Within the limited studies, there is very little consensus on evaluation methods or the kind of learning that is expected. A reason for the confusion appears to be the lack of a common understanding of computational thinking. Whilst nearly all the studies we reviewed reported overall positive results related to “computational thinking”, computational thinking was often only vaguely defined and the goals and what constituted evidence for it, varied greatly.

We contend that instead of pursuing a definition of CT, CT should be viewed as a “movement” that unites computer scientists, creative coding advocates and school educators to bring CS into schools. This view of CT is consistent with what sociologist Susan Leigh Star (2010) called boundary objects, which she defined as “a sort of arrangement that allow different groups to work together without consensus” (p. 602). She developed the concept from her fieldwork among scientists and others cooperating across disciplinary boundaries. A core feature of a boundary object is that it has a vague, common definition, which would be tailored and made more specific to local use within a community of practice. This is evident in this analysis where researchers appear to work under an overarching but vague concept of CT to pursue quite distinct learning goals, which are operationalised and measured using different instruments. If viewed through the lens of boundary objects, we suggest that the lack of a common definition of CT is not problematic, but rather may support mobilising multiple communities of researchers and educators to cooperate in advancing coding and computer science in schools.

However, according to Star (2010), the varied local definitions of boundary objects makes it only “useful for work that is NOT interdisciplinary”. (p. 605, her emphasis). The integration of coding into school-based curricula requires interdisciplinary work, and that demands a clearer conceptual basis than computational thinking can currently offer. By studying how computational thinking is operationalised in empirical studies, it may now be possible to clarify the learning outcomes from integrating coding into other subjects. In those studies where CT was defined in terms of disciplinary knowledge and skills of CS, students demonstrated the ability to only apply code procedures, and there was almost no evidence of CS conceptual understanding when they did this. Moreover, in the reviewed studies where CT was defined in terms of acquiring other subject knowledge, the only evidence of learning was in data visualisation tasks in science; evidence of learning in studies concentrating on mathematics was largely absent. Precisely if and how cross-curricular coding tasks can advance students’ learning in other subject areas, especially for creative and language arts, is yet to be determined. Moreover, when CT was defined in terms of general competencies, its relationship with problem solving, collaboration, and creativity seems contingent upon the task design and classroom environment.

It is worth noting that some of the articles suggested that students used abstraction as a thinking skill. However, definitions of abstraction varied greatly, even in the few articles that were reviewed: Abstraction has been viewed as “using scientific terminology, with some abstraction and conceptualization” (Mannila et al., 2023, p. 11, their italics), “production of new concepts” (Gautam et al., 2020, p. 393), or as a programming practice to “make a block” in Scratch (Woo & Falloon, 2022). However, it is beyond the scope of this review to disentangle its various meanings (for a more detailed discussion of this, see Ezeamuzie et al., 2022). Without a common definition it has not been possible to synthesise the findings related to abstraction in a meaningful way.

Overall, the studies presented consistent evidence of the importance of the teacher’s role in improving student learning. This should be understood against the backdrop of a global shortage of CS teachers, and the reliance on non-specialist teachers to fill this void poses undoubted limitations on student learning. Moreover, the few reviewed studies which considered the influence of school leadership consistently signalled their crucial role in supporting teachers to overcome time, knowledge and resource constraints, as well as exosystem pressures such as national assessments. Future studies should consider influence from multiple levels of the school ecosystem on learning coding across the curriculum. To this end, we recommend design-based research as a methodology (Anderson & Shattuck, 2012) to involve teachers as co-designers in an iterative process to address complex implementation issues (see, e.g. Biddy et al., 2021; Krakowski et al., 2022; Mannila et al., 2023). Lastly, to advance our understanding of how different national contexts may impact on research and students’ outcomes in integrated coding curricula in schools, more research should be encouraged outside of North America and Europe.

7. Conclusion

There is global recognition that students in the 21^st century need to have well-developed digital literacy and deeper knowledge of computing which is now ubiquitous in different environments, and a “movement” comprising multiple stakeholders aligned with the concept of “computational thinking” has been instrumental in promoting CS and coding in schools. However, by analysing the way computational thinking has been operationalised, we contend that it conceptually conflates a range of learning outcomes. By teasing apart various possible outcomes, this analysis suggests that research evidence for learning resulting from integrated coding curriculum is modest, and generally falls short of the high expectations that are often conveyed in literature and national curriculum and education policy statements. To avoid being caught up in the “hype”, we suggest that future research into integrated coding curriculum should address specific outcomes in computer science, in the integrated subject, and general competencies, and resist direct association with developing “computational thinking”. Furthermore, findings indicate that the quality of student learning is also highly dependent on teacher practice and school environmental factors, and we strongly advocate for future studies to adopt methodologies such as design-based research to ensure school ecosystem factors are thoroughly considered.

Credit author statement

Karen Woo: Conceptualization, Methodology, Data curation, Formal Analysis, Writing – Original draft preparation. Garry Falloon: Supervision, Writing – Reviewing and Editing

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by an Australian Research Council Discovery Projects Grant [DP190100228] and an Australian Government Research Training Program (RTP) Scholarship. The views expressed herein are those of the authors and are not necessarily those of the Australian Research Council.

Notes on contributors

Karen Woo

Karen Woo is a Ph.D. candidate at Macquarie School of Education at Macquarie University. Karen is researching the development of computational thinking and 21^st century competencies through coding. She is an experienced coding and robotics educator working with pre-service teachers and primary-age students. She also consults with schools on integration of technology in the classrooms.

Garry Falloon

Garry Falloon is Professor of STEM Education in the Macquarie School of Education at Macquarie University. His background includes 22 years teaching and leadership of primary and secondary schools in New Zealand, Education Foundation Manager at Telecom New Zealand, working with Microsoft in the Partners in Learning and Digital Learning Object projects, and as project lead for the New Zealand Government’s $10 m Digital Opportunities Project. His research interests include mobile learning, digital learning in primary and middle schools, online and blended learning, curriculum design, pedagogy and assessment in digitally-supported innovative learning environments, learning in primary science and technology, and educational research methods.