Capturing the spark: PISA, twenty-first century skills and the reconstruction of creativity

ABSTRACT

Creativity has fascinated scholars for generations, and its identification as one of the key ‘twenty-first century skills' necessary for economic growth has led to renewed interest. This creates two challenges for the OECD: its flagship Programme of International Student Assessment (PISA) does not directly measure creativity. Secondly, the increased importance attached to creativity has highlighted claims that high performers on PISA are largely nations stereotyped as lacking creativity. This challenges PISA's self-proclaimed status as the premier global benchmark for evaluating and comparing the quality of school systems and weakens its capacity to deliver its core mission; to identify ‘best practices' which ensure economic prosperity. We explore these challenges and examine both how the OECD has responded to them and is moving to include creativity in PISA 2022. We argue that, while a precise definition of creativity has defied scholars for centuries, the indications are that the OECD's metric will focus on a narrow, convergent and easily-measured conception associated with cognitive competencies and linked to enhancing human capital. In this way, the ‘messiness’ around the polysemic concept will be simultaneously both exploited and threatened, as new, measurable versions displace alternatives.

KEYWORDS:

• Creativity
• PISA
• twenty-first century skills
• knowledge economy
• OECD

Introduction

Since its inception in 2000, the OECD’s PISA programme has become the most influential comparative educational assessment and central to the emergence of global educational governance (Sellar and Lingard 2014; Högberg and Lindgren 2021; Lewis 2020). In parallel, as Tröhler (2022) argues, the OECD’s metrics have also been used by states to promote forms of banal nationalism and pursue imperial aspirations. PISA’s reach has grown, both geographically, and in terms of the scope of its metrics: ‘innovative domains’, which extend the assessments into areas designed to provide ‘information on how well-prepared students are for full participation in society’ (OECD 2015a) are now included in each round of tests. As Nordin and Sundberg (2014, 13) argue, ‘today, making major reforms in the education sector without reference to global or transnational indicators seems politically stillborn’. This paper focuses on the ‘innovative domain’ for the 2021 PISA test (postponed to 2022) – creativity. We provide a critical analysis of this initiative, firstly, by identifying the challenges which led to the OECD’s decision to develop a standardised measure of creativity; secondly, by exploring the trajectory by which the OECD strategy evolved – including its early responses to these challenges; and finally, by analysing the instrument in the context of the future success of PISA. We argue that the OECD’s measure of creativity relies on a narrow definition of creativity based on easily-measured cognitive skills associated with enhancing innovation and economic productivity, while ignoring more subtle and nuanced conceptualisations which defy standardised measurement.

The study is based on the thematic analysis of two sources of data. Firstly, the OECD materials which provide a historical record of the development of the creativity measure. This included the range of OECD reports, beginning from 2015 when the idea of a creativity assessment was first formalised by CERI, the OECD’s educational research arm, through the pilot materials (2015) to the draft framework for measuring creativity (2019). Secondly, we analysed media materials which related to creativity and its relationship with PISA. This included two strands; YouTube videos and webcasts made by the OECD, primarily delivered by its Director of Education Andreas Schleicher, which promoted the importance of creativity; and media coverage of PISA results.

Our analytic approach was thematic, using Multimodal Discourse Analysis (MMDA), to elicit the key linguistic and discursive devices used in the marketing of materials and the promulgation of key messages. This approach sees all communicative acts as ‘text’, acknowledging that meaning can be conveyed through signs, colours, themes and image, as well as by words, both spoken and written (Bezemer and Kress 2017). A text is ‘remade’ by its interaction between producer and audience, and thus, meaning is dynamic and altered by the relationship between the entity producing the text, and the entity receiving it (O’ O’Halloran and Smith 2012; Kress 2003, 2013). Every linguistic or semiotic choice has the potential to express power relationships; thus, using MMDA to explore themes in OECD PISA materials offers the tools to analyse deeply the messages being transmitted by OECD actors, and to offer interpretations of how these messages might be received and ‘altered’ by their intended audiences in the media and policymaking spheres.

The inclusion of ‘innovative domains’ in PISA is characteristic of the OECD’s ‘humanitarian turn’ – a response to the 2030 Sustainable Development Goals (SDGs), with which the OECD’s learning framework 2030 is closely aligned (Xiaomin and Auld 2020; Robertson 2021). Coming after creative problem solving (2012); collaborative problem solving (2015b); and global competence (OECD 2018), the inclusion of creativity in 2022 also demonstrates an ongoing linkage to the ‘twenty-first century skills’ of Critical thinking, Collaboration, Creativity and Communication (P21 2006), which have become prominent in global educational policy discourse since the late twentieth century. This trend has also been portrayed as an attempt to maintain PISA’s relevance against rising criticism, and as one in a series of PISA by-products which may help to address ‘PISA fatigue’ (Sorensen, Ydesen, and Robertson 2021).

Our analysis is informed by the critical Political Economy perspective which interrogates the influence of capitalism in shaping cultural and political spheres, and thus stresses the importance of power relations (Dale 2000; Klees and Edwards 2014; Mundy and Verger 2015; Novelli 2013; Samoff 2013; Vavrus 2005). Analysis thus focusses on the historical-structural conditions, those involved in policy formation processes, and the structurally-oriented strategic choices made in the processes of policy formation and transfer (Edwards 2018; Jessop 2010). The metrics developed by international agencies serve to define what is valued and their adoption as a measure of performance can have a powerful influence on policy and practice. Robertson and Dale (2015) emphasise how these choices are influenced by discourses that limit what is envisioned, possible, and feasible. They call for a critical approach which recognises the interplay of factors in the cultural, political and economic spheres in education, to interrogate discourses around the globalisation forces working on education governance and enactment. From this perspective, globalisation is seen as a ‘project’ rather than something which ‘just happened’, leads to an exploration of actors whose agency is often obscured, but who are driving the project and, ultimately, benefiting from its ongoing dynamic nature. The resulting reshaping of education escapes the usual accountability systems associated with ‘traditional’ democracy, and global actors can define the meaning of core concepts such as creativity and quality (Sousa and Moss 2022) without being exposed to scrutiny from those whose role it is to hold power to account – especially the media.

We also draw on two associated bodies of literature. The first is that which has critically focussed on the ways in which the OECD has defined other polysemous concepts in their quest to create standardised measures, such as Global Competence and the wellbeing of five year olds (Sousa, Grey, and Oxley 2019; Auld and Morris 2019; Goren, Maxwell, and Yemini 2019). This literature has demonstrated how they have valorised Anglo-American definitions and focussed on what is easily measurable. Secondly, we draw on literature which positions international agencies such as the OECD and UNESCO as organisations which are increasingly competing to exert greater influence and enhance their power in the field of Global education Governance (Elfert and Ydesen 2020). Whilst UNESCO’s position is rooted in its promotion of a humanitarian agenda, that of the OECD derives from its expertise in measurement and the claim that its metrics are a proxy for educational quality and economic growth. Our analysis demonstrates the powerful influence of capitalism on the processes by which new discourses around creativity have emerged, and through which its essence is being oriented away from the cultural domain towards the core tenet underpinning PISA: namely that increased scores on its measures of pupil performance would translate into faster economic growth (Hanushek and Woessemann 2008, 2010; Schleicher 2018).

Creativity: the challenges

Education policy discourse has since the turn of the century paid increasing attention to creativity as a key ‘twenty-first century skill’, often explicitly linked with innovation and the advancement of the ‘Knowledge Economy’ (e.g.,Yusuf 2007; Newton and Newton 2014). Paradoxically, the OECD has been the main driver of that discourse promoting the teaching of creativity in schools, while at the same time, not explicitly measuring it on its triennial PISA tests. This is the first of two challenges we identify for the OECD, relating to the core purpose of PISA, namely, to identify and advocate the causes of high educational-economic performance. Schleicher has used many of his recent media appearances to stress the importance of ‘twenty-first century skills’ as the changing demands of an increasingly interconnected world threaten to make redundant the curriculum subjects which are currently the focus of PISA, and to highlight the need to develop metrics for the ‘new’ skills which the future now demands (e.g., YouTube 2014, 2019, 2022). This ‘anticipatory global governance’ (Berten and Kranke 2022), which positions the OECD as a ‘guardian of the future’ (Robertson 2022), is deemed necessary to ‘future proof’ education systems against uncertain futures. As the OECD explains; ‘the future is here, and education systems need to learn from it’ (OECD 2020). The OECD has thus created its own imperative for a PISA metric to measure creativity: it is central to the future-oriented policymaking demanded by the uncertain world we live in, and therefore to maintaining the ongoing relevance of the PISA instrument.

The second challenge we identify centres on perceptions of ‘others’, specifically, the use of stereotypes in media and other discourses. The most successful performers on PISA are stereotyped as less creative, more ‘traditional’, and focused on rote learning and long stressful hours in school. Shanghai, Singapore and Korea, while at the top of the PISA league tables, are also linked in media discourse with high rates of mental ill-health, related to exam stress and excessive pressure to succeed academically (Waldow 2019). The media is front and centre of PISA; without the media, PISA would not enjoy the prominence it does, and it is a deliberate strategy of the OECD to use the media to promote the policy messages of PISA (Stack 2007; Grey and Morris 2018; Crome 2022). Schleicher explains how it evolved, after the first round of PISA demonstrated poor results in Germany:

I realised this is really the wrong strategy … going top down, going to the people in charge isn’t going to change the system. And I actually changed strategy and thought, I’m going to go to work with the media, go to work with other people, and that has created a public demand for better education … . (Schleicher 2015, online)

However, the norms and practices by which the media operate (‘media logic’ – Altheide and Snow 1979) rely heavily on a set of ‘news values’ which determine what is reported and how it is presented.

Such stereotypes are, however, not limited to the media, as Waldow, Takayama, and Sung (2014) note, they have strongly influenced the choice of Nordic rather than Asian nations as reference nations in Germany, while some studies have suggested (eg Pu and Evans 2019; Tian and Low 2011) that western iterations of creativity and criticality are incompatible with Chinese political and cultural environments. Lockette asserts that China has been ‘furiously promoting creativity as a vital part of its education system since 2000’ (Lockette 2012, 34) and several reforms of the Gaokao, the university entrance exam, have been motivated by the perceived need to promote innovation and creativity in the Chinese workforce (Zhang, Zhao, and Lei 2012), weaknesses which were highlighted by students themselves in a study carried out by researchers at Beijing Normal University (Muthanna and Sang 2016). An (2021) describes a Chinese government research group established to combine Marxist values of education with the ‘Core Competencies and Values’ which students will need in the twenty-first century, acknowledging the importance of creativity and other ‘soft skills’. Below we explore how the OECD has dealt with those challenges.

Whilst these challenges encouraged the decision to finally assess creativity, that arguably presents the OECD with its most significant recent challenge around PISA, as it requires it to successfully resolve the issues which have exercised generations of scholars; how to define and measure a concept that is seen as fuzzy, polysemic, malleable, multifaceted, often distinguished from analytical and logical reasoning (Sternberg 1988; Gardiner 1993) and viewed by some (Unterhalter 2017; Meyer 2016) as inherently unmeasurable given its traditional associations with unpredictability, imagination, novelty and originality. Munday (2014) similarly argues the concept has no universal essence and should be viewed as a series of competing discourses. The challenge faced by the OECD is intensified as PISA is designed to operate across contexts and cultures. Grigorenko (2018, 5) argues that conceptualisations of creativity are dominated by western thought, ‘championed by the North American mentality’. Gormley extends that logic and argues that ‘from the plethora of creativity discourses, which could manifest in education policy, neoliberal themes throughout policy inform which of those versions are supported or ignored’ (2019, 2). Runco and Jaeger’s (2012) review of research argues that scholarship has failed to advance the definition much beyond what had been proposed in the nineteenth century; and Long’s (2014) review similarly concluded that the field was dominated by quantitative studies, which had not made significant inroads into reaching a common definition or measure. In summary, despite sustained and disparate attempts to define and measure creativity, there is an absence of a consensus.

Twenty-first century skills: the centrality of creativity

The first challenge we identify above for the OECD, is to maintain PISA’s relevance as the key measure of those new skills which it now argues will determine the future economic success of nations as they compete in the global ‘knowledge economy’. Recent years have seen a renewed interest in creativity as a teachable skill and it now is provided as degree programmes in a number of Universities (Aktas 2022). This focus on creativity, as with the global testing culture (Ydesen and Andreasen 2020; Elfert and Ydesen 2020) within which it is being incorporated, also had its primary origins in the USA. Concerns about the American economy in the early 1980s led to criticisms that schools were failing to provide a competitive workforce and a National Commission on Excellence in Education published ‘ A Nation at Risk: The Imperative for Educational Reform’ in 1983. This influential report used alarmist language imbued with nationalist overtones to warn Americans of the impending threats to their economic future unless a ‘learning society’, founded on the ‘ persistent and authentic American dream that superior performance can raise one's state in life and shape one's own future’ (ibid., online) be created. This learning society needed to nurture the appropriate skills necessary to increase economic output and ensure that America's ‘once unchallenged pre-eminence’ across many spheres of commerce and business did not continue to be ‘overtaken by competitors throughout the world’ (National Commission on Excellence in Education 1983 online).

Subsequently, a collaboration of American educational and business organisations compiled another report, ‘What Work Requires of Schools’ (US Dept of Labor 1991). This contains the seeds of the discourse subsequently promulgated by many global organisations. One section, entitled ‘Yesterday's Student/Today's Worker’ anticipates Andreas Schleicher's oft cited catchphrase, ‘Your School System today is your economy tomorrow’; – ‘every adult American will be literate and will possess the knowledge and skills necessary to compete in a global economy’, foreshadowing the economistic discourse which is now familiar (ibid., 23). The report envisions employment in the twenty-first century and details the ‘competencies’ the workforce will need. In 2002, their explicit linkage to the ‘global knowledge economy’, was identified by the Partnership for twenty-first Century Learning (P21) which was established ‘to position twenty-first century readiness at the centre of US K-12 education and to kick-start a national conversation on the importance of twenty-first century skills for all students’ (P21 Website). In 2006, they published ‘A State Leader's Action Guide to twenty-first century skills’. Addressing the question, ‘why, then, is America losing ground on measures of educational excellence, workforce competitiveness and economic innovation?’, they answered:

… education is still, for the most part, stuck in the twentieth century. By traditional metrics, American students are doing better in school … But these metrics aren’t sufficiently challenging students to meet the demands of the twenty-first century—especially in a world where international competitors are threatening US pre-eminence on multiple fronts. (P21 2006, 2)

Thus, Americas continued pre-eminence required new metrics and by 2010, corporations including Apple, Walt Disney, Crayola and Lego, had signed up to the P21 and funded a ‘common core toolkit’ to ‘embed’ the newly identified 4 Cs – Communication, Creativity, Critical Thinking and Collaboration, described as the essential ‘twenty-first century skills’ (also known as ‘transversal skills’) – alongside the 3 Rs in all American educational institutions. The P21’s ‘vision and mission’ statement focusses on equipping the young with the essential skills needed to thrive in a ‘world that is constantly changing’; ‘to prepare all students for the challenges of work, life, and citizenship in the twenty-first century and beyond, as well as ensure ongoing innovation in our economy and the health of our democracy’ (P21 online). Anderson-Levitt and Gardinier (2021) note that 21C skills were subsequently described as ‘competencies’ in the USA and were quietly incorporated within the Common Core State Standards of 2010.

These tenets quickly moved beyond their American roots: they now provide the vision for the policies of the major global organisations as well as many national policies. The mechanisms by which this discourse has diffused globally- from domestic US policy via international agencies – are beyond the scope of this paper, but over the last seven years, The World Economic Forum (WEF), UNICEF, UNESCO, the World Bank and the OECD have used the identical language of twenty-first century skills and called for ‘salient metrics’ (World Bank 2018) to facilitate evidence-based reform and introduce twenty-first century skills into schools.

This discourse has also entered policy space across continents; for example the same imperative for students to learn 21C skills is reported in South Korea (Kim et al. 2012); Ghana (Essel, Tachie-Menson, and Ahiaklo-Kuz 2017), Bangladesh, (Rahman 2019) and for out of school youth in the Philippines (Tindowen, Bassig, and Cagurangan 2017).

According to Villalba (2012, 1) ‘everyone agrees’ that creativity needs to be fostered and

should be used to judge the success of an educational system’ The OECD considers itself best placed to undertake this judgement, but that requires a reframing of earlier claims about quality, based on ‘traditional’ measures of Maths and Literacy.

National stereotypes

The second challenge we identified above, which centres on the role of the media in disseminating and altering messages around PISA, has been the object of limited scholarly attention (Lingard and Rawolle 2004; Baroutsis and Lingard 2017; Waldow, Takayama, and Sung 2014; Grey and Morris 2018). The influences of ‘media logic’ (Altheide and Snow 1979) on events before they reach public spaces as ‘news’ are often invisible, but nonetheless the ways in which messages are shaped and framed for public consumption act directly on perceptions in both policymaking and public spheres . As Luhmann famously stated, ‘whatever we know about our society, or indeed about the world in which we live, we know through the mass media’ (Luhmann 1996, 1), and the messages of PISA, transmitted primarily through the media, are also subject to its logic, which determines which ‘stories’ make headlines and which do not, and to the forces of mediatisation (Esser and Strömbäck 2014) which shapes how messages will be shaped for a media audience. A negativity bias is a recognised feature of media news reporting (Soroka and McAdams 2015), and the use of stereotypes in media communication is widely acknowledged (Dixon 2019).

The most successful performers on PISA have primarily been in East Asia; and the media in both western and Asian societies have relied heavily on national stereotypes in reporting the results. The PISA rankings play easily into notions of ‘us’ (‘how well did we do?) and ‘them’ (‘how did they beat us?’) and constitute a powerful form of what Billig (2005) terms ‘banal nationalism’. Media reports on PISA outcomes tend to reflect the stereotypical ‘otherness’ of those societies which ‘we’ are encouraged to emulate. To cite examples referencing China: a Norwegian headline asserts that ‘Chinese schoolchildren pay heavily for their success’, stating that the price of doing well in examinations is ‘tapt kreativitet’ – lost creativity (Dagbladet 2010). A French report states that ‘Chinese students are not encouraged to use their critical skills, so it is not surprising that they rarely show creativity’ (Duperron 2015); while a Swedish news outlet reports that Chinese children are subjected to ‘sausage learning’ and ‘long and demanding days’ (Liebermann 2013). A Spanish report states ‘compared with other education systems, China … prioritises arithmetic and rote learning over creativity … ’ (Alvarez-Diaz 2013); and in Russia, in a largely positive article about Chinese education, there is an acknowledgment that the education system is aimed at ‘passing exams through cramming’ (Doronin 2016). Academic research, too, acknowledges the existence of such stereotypes: Waldow, Takayama, and Sung (2014) note the centrality of negative reporting on Asia; Takayama describes an ‘exclusive focus on stereotyping in the negative [media] framing’ (2018, 609) of East Asian societies; and Komatsu and Rappleye (2018) challenge the accuracy of such stereotyped portrayals of examination stress in East Asian countries.

The discourse around the ‘global education race’, espoused by many western politicians heightens the tension around issues of national identity raised by educational comparison. That the successful Asian nations are also those with flourishing economies is central to their attraction to western policymakers, and negative portrayals of Asia in the west are now mixed with more positive aspirational attention. Asia is at once both attractive and threatening in terms of its ‘otherness’, and the binary portrayals of ‘us’ and ‘them’, with their orientalist/neocolonial overtones (You 2018), are deeply embedded. As Takayama (2011) notes, epistemic ‘knowledge’ in comparative policymaking is dominated by western thought and scholarship, and it is primarily North American economists who have driven the current ‘global’ education agenda. Other scholars focus on cultural explanations, for example, Kim suggests that because Asian education is motivated by extrinsic factors (examination results, social norms, family and the Confucian tradition), rather than the intrinsic ones conducive to creativity, fostering it in Asia is difficult. Some elements of Confucianism may, she suggests, serve as ‘cultural blocks to creativity’ (Kim 2005, 341), before asserting that ‘East Asian countries should appropriate some of America’s values into the educational process’ (345).

As these examples suggest, western nations are generally portrayed – often by implied contrast – as having less formal, more creative education systems and progressive, child-centred pedagogies which allow young people to develop more holistically. Other examples are more overt: Sahlberg, whose portrayals of Finland are often cited in opposition to what he calls the Global Education Reform Movement (GERM) facilitated by PISA data, argues that the aims of Finnish education include trying to put the happiness of the student, imagination, curiosity and creativity at the heart of learning (Sahlberg 2012, online). Several US media commentaries focus on the openness to questioning, critical thinking and creativity offered in American schools, suggesting that Asian societies would do well to adopt some American practices (Ringmar 2013; Rubin 2014); one report on a US school states:

This kind of self-directed, hands-on, creative, collaborative curriculum is rare in China and South Korea, where the main focus of high school is to prepare for college entrance exams, often through intense rote memorization and a standardized curriculum. (Voice of America 2011)

And following the broadcast of a BBC television series in which Chinese teachers attempted to introduce their methods into an English school, the Headteacher stated that ‘British pupils expect to have variety in their learning. They are not used to being incarcerated in a large group and in the same classroom studying a very narrow curriculum.’ (BBC 2015).

The origins of this stereotyping derive from the dichotomised models of ‘eastern’ and ‘western’ education, and particularly in the stylised ideals often portrayed as ‘Confucian education’, which is deemed to prevail in East Asia, and those of education based around modern, progressive pedagogy and critical and creative thinking, seen as ‘western’ models (You 2018; Elliott 2018). As Ryan and Louie (2007) note, both models are superficial:

Critical thinking, deep learning, lifelong and lifewide learning are heralded as the outcomes of Western education but these concepts are often under-theorised or lack agreed meanings. Equally fuzzy concepts such as ‘Asian values’ or ‘Confucian education’ are eulogised as keys to successful teaching and learning when Asia prospers economically. (Ryan and Louie 2007, 404)

The western media have not only perpetuated such stereotypes: several East Asian countries have introduced reforms to address the perceived lack of creativity in their education systems. Zhao (2015) states:

… these educational systems [in East Asia] have been struggling to produce creative and entrepreneurial citizens for decades. They have ironically looked up to America and other Western nations for strategies to move away from their traditional practices, which produce great test-takers at the cost of creativity. (130)

In Singapore, while there is pride in PISA rankings, recent media reports question the ‘obsession with grades’, which causes the country to ‘lag behind in efficacy of learning for the new age’ (Sinnakaruppan 2017) as well as causing stress and anxiety in young people (Zaccheus 2017; Davie 2017). The answer, for one newspaper commentator, is that ‘we need to leap forward onto a parallel curve, centred on nurturing creativity, innovation and entrepreneurship’ (Sinnakaruppan 2017, online). In Korea, Fendos (2017) reports schools and universities, have begun initiatives to boost creativity, with ‘institutes for creative education’ and ‘creativity centres’ springing up. In China, the link has been made between innovation and creativity, and Hruby (2016, online) reports that ‘huge sums are being spent on so-called creator spaces’ and that parents, frustrated by the ‘strict education system which … largely resists alternative ideas’ are increasingly opting to teach children at home.

Such stereotypes have proved persistent and still provide the default position for many media reports about Asian education systems.

As noted earlier, the OECD’s messages are tailored to appeal to media audiences which reproduce them, ‘in such a manner that reinforces the need for solutions and urgent decisions’ (Yariv-Mashal and Novoa 2003, 425). The OECD is aware that media actors will operate via their own logic, turning PISA into a news story, governed by values of newsworthiness (Grey 2020). Thus, we argue, the promotion of stereotypes was predictable, and initially it would have served to foreground the messages around equity and teacher quality promoted by PISA reports. More recently, however, Schleicher has moved to counter some of the negative reporting based on stereotypes which the media involvement in PISA has fostered. For example, he explained in 2019:

When it comes, for example, to cramming, I would think it's more prevalent in the UK rather than in Singapore or the four provinces of China. I don't think we should stereotype the systems … . That might have been true 20 or 30 years ago. Those countries are now developing very sophisticated skillsets among their young people that involve complex problem-solving skills, creative skills, critical-thinking skills. (TES 2019, online)

The OECD’s challenge lies in persuading its clients, and, importantly, the media, that PISA remains relevant in a future that depends increasingly on the ‘new’ skills or competencies, which it has helped to promote but does not measure.

Current responses to the challenge of creativity

The OECD has, to date, employed two strategies to deal with the challenges we identify. Firstly, it has asserted that existing PISA tests do indeed measure aspects of creativity and that, contrary to the stereotypes, the results of these measurements are consistent with the strong performance of East Asian nations. When challenged about creativity by a Swedish journalist in 2013, Schleicher replied:

… creativity has many dimensions. What I can say though is that when it comes to, for example, areas like Mathematics and Science, Swedish students actually do not show a high level of creativity - Swedish students are not good … if you could move to level 5 and 6 on the PISA tests … you can't just repeat what you learn in the classroom, you have to be able to integrate different fields of knowledge, you have to be able to extrapolate from what you know, you have to be able to apply mathematical knowledge and skills in unfamiliar and novel situations … . (Skolverket 2014)

Schleicher has also used this strategy, linking creativity with high attainment on PISA tests, to address media claims that poorly performing countries (like Sweden, above, and Australia, below) are ‘creative’, while high performers, like China, are not:

Chinese students are very good on ‘elaboration’ skills – such as reasoning, deep learning, intrinsic motivation, critical thinking, creativity and non-routine problem solving – where Australia is weak. (Schleicher, as quoted in Dodd 2017, 3)

A second strategy centres on directly associating creativity with the OECD’s measurement of ‘creative problem solving’, which was included as the ‘innovative domain’ in PISA 2012. When the results were reported, the emphasis was on ‘problem solving’ rather than ‘creative’ – with the former term appearing 51 times versus the latter 4 times in the report. Nonetheless, the results demonstrated a close correlation between East Asian high performers on the usual three domains of PISA and those who did well on the creative problem-solving tests. After Singapore came top in this test, Schleicher explained that ‘Singapore’s performance debunked criticism that its education system encouraged rote learning at the expense of developing creative skills’ (Siau Ming En 2014 online)

However, the increasing scrutiny of the western media (e.g Andrews et al. 2014), and the wider centrality of discourses around creativity in education, albeit that the OECD was instrumental in furthering, posed an ongoing challenge to the perception of PISA as the premier assessment of world education and in 2013, the OECD began exploring how to measure creativity directly. Developing a metric that demonstrates that high PISA performers are also measurably creative would serve the dual purpose of helping to address the challenges and ensuring the continued relevance of PISA.

The OECD and measuring creativity: A brief history

In 2013, the OECD published the results of an earlier study by Lucas et al to propose a model and conduct field trials of an instrument for ‘defining and assessing creativity along with practical suggestions on how it can be developed and tracked in schools’ (Lucas, Claxton, and Spencer 2013, 1). The study indicates the challenges faced – there are references to ‘difficult conversations with teachers’ over the idea of measuring creativity, including expressions of ‘anger, hostility and bewilderment’ (6) which the authors suggest were attributable to the way in which the data was presented; ‘ … the table looked all too much like the kinds of levels associated by teachers with attainment levels achieved in core subjects such as literacy or numeracy’ (6). Perceptively, they note that ‘the problem is that there is no consensus on what creativity is’ (7); nonetheless, they recommended that their tool for assessing creativity should be trialled with ‘the unconverted’ – the teachers taking part in the trial were self-selecting and already keen on the core premise – and used to assist teachers in collating and assessing data about their pupils’ creativity (27-28). It is important to stress that this working paper focused on the formative use of creativity testing, ‘while remaining agnostic about potential summative uses’ (18). In this initial exploration, there was no linkage to PISA, nor with its link to human capital.

In parallel, in a series of YouTube broadcasts, Schleicher, echoing the language of P21, described challenges to ‘the education industry’ around the ‘mismatch’ between what is being taught in schools and what employers need (e.g., YouTube 2013; YouTube 2014) because of an ‘evolution’ in workplace demands (YouTube 2018). These broadcasts emphasise the decreasing importance of ‘traditional’ and ‘routine cognitive’ skills, (including many forms of problem solving), which can largely be carried out by computers; and the need to develop character skills and personality traits that will equip students for the twenty-first century. When the results of PISA 2015 were released, not only was Schleicher continuing to assert that the ‘traditional’ PISA test could offer useful metrics on creativity; but also, that PISA was now reflecting ‘how the world and the skills that people need are changing’ (Gomes 2016). By 2017, Schleicher was asserting that ‘the industrial approach to schooling, where children were educated in batches of age and all go through the same standardised and prefabricated curriculum and instruction’, ‘hasn't really been conducive to nurturing the kind of creativity that we need in this age of accelerations’. … . ‘ Whichever way you construct your list of twenty-first century skills’, he states, ‘creativity will be at the heart of it’ (YouTube 2017).

Thus, he effectively consigns the very skills that PISA currently focusses on measuring, and links to economic success, to the past, while also asserting that his organisation is ‘reflecting’ a world that ‘has changed’. Auld and Morris (2019) demonstrate that this shifting of agency; recognising that ‘the world has changed’, by a process of ‘evolution’, and demonstrating that the OECD is responding to the need to measure it, is an established strategy used to promote other initiatives such as PISA-D (OECD 2013b).

In 2015, CERI, the OECD’s research arm, launched a project with 14 countries to explore ways of teaching and measuring creative and critical thinking, using the work of Lucas, Claxton, and Spencer (2013) as an initial conceptual framework. Pilot pedagogical materials were developed and several tests were produced by CERI and the participating countries, using ‘pedagogical activities that cultivate creativity and critical thinking while teaching the official curriculum’ (OECD 2015a, 9). These formed the basis of a series of interventions with a large multi-national groups of primary and secondary pupils in a controlled trial. These activities needed to be amenable to assessment four or five times a year, largely for qualitative purposes, (10). The overall aim was to benchmark ‘pre- and post-standardised tests of academic achievement and creativity … as measured by creativity tests’ (11). It was stated that ‘the conceptual framework developed in the project may contribute to the development of a possible module on creativity for PISA 2021’ (OECD 2015b online). Thus, the leap from the formative field trials of teaching materials carried out by Lucas et al, to the development of a standardised assessment for summative use (on PISA) was made.

In 2018, the contract for designing a creativity test for PISA 2021 was awarded to the US organisation ACT (American College Testing). The third draft framework for the 2021 PISA test, postponed to 2022 on account of the global pandemic, was published in 2019 (OECD 2019), and states that ‘Developing an international assessment of creative thinking can encourage positive changes in education policies and pedagogies’ (5). Creativity is defined thus:

the competence to engage productively in the generation, evaluation and improvement of ideas, that can result in original and effective solutions, advances in knowledge and impactful expressions of imagination. (8)

The document distinguishes between ‘big C’ and ‘little c’ forms of creativity, concluding that the latter can be ‘developed through practice and honed through education’ (ibid., 9). The latter is the focus of the assessment, which will be administered through a one-hour test carried out on a standard desktop computer. The four ‘domains’ tested, shown in Figure 1, arise from two categories of creative thought: ‘creative expression’ and ‘creative problem solving’ which are each subdivided. The stated hope is that differences between conceptualisations of creativity across cultures and countries will be reflected, and that this will uncover strategies for how creative thinking should be taught in schools, though how these differences will emerge is not made explicit in the framework.

Figure 1. Domains of assessment, PISA 2022 Creative thinking task (OECD 2019, 20).

The tests will also require a written task, which will demonstrate students ability to:

express their imagination in a written format, respecting the rules and conventions that make written communication understandable and appreciated for its originality by different audiences. (20)

Sample tasks include: captioning cartoons or fantasy illustrations; making ‘an imaginative correction’ to the work of others; using a digital drawing tool to create a design to answer a particular task; working with a virtual group to collaborate on a creative project; and formulating research questions or hypotheses to explain a scientific or mathematical phenomenon. Templates will be provided and credit given for all responses judged to be ‘valid’. Table 1 shows potential measurement criteria for responses across the four domains.

Table 1. Possible ways to measure creative thinking facets across domains (Source: OECD 2019, 26).

Responses to the tasks will be coded by trained coders, who will to decide whether such responses are ‘appropriate’ and ‘valid’. The OECD acknowledges, in several pages devoted to scoring, that

scoring challenges are greater for this assessment than for any other PISA domains, and are intrinsically related to the nature of this domain. (28)

A selection of training and examples are provided to assist in this complex task, as well as advice to ensure that ‘cross cultural validity’ is established. The document also acknowledges the potential limitations, noting the ‘trade off ‘ (44) between including a greater number of indicators ‘to better inform policymakers’ (ibid.) and reliability of reporting. A field trial to test the instrument before it is given to students as part of PISA 2022 is underway at the time of writing this article.

Discussion

We have argued that the OECD’s development of a test to measure creativity is required to help to resolve the dual challenges arising from the prophecy that in the ‘anticipated future’ of education ‘twenty-first century skills’ are necessary and need to be measured, and from to the difficulties arising from the stereotyping of high performers on PISA as lacking in creativity.

Criticism of pressurised school environments and lack of creativity in the high scoring countries on PISA has traditionally been portrayed by the media as a counterweight, or consolation, to depressing stories of relative western failure and decline. The ‘well, at least we are creative’ criticisms were countered by the OECD initially by the claim that creativity was partly measured by levels 5 and 6 on PISA tests, which focus on the extrapolation of knowledge and skills to unfamiliar settings. Those claims were difficult to sustain, as the discourse surrounding twenty-first century skills intensified and stressed the key role of creativity. The OECD, rather than acknowledging its own agency in defining those skills, positioned itself as expert responder to this development by providing the necessary metrics to evaluate this essential skill, and to offer advice on how best to teach it.

Changes to the ways in which education is seen, and the skills that are valued, are made in a global ‘policyspace’ and rely on a selection of what Rizvi terms ‘Global Imaginaries’ (Rizvi 2006); their origins are obscured by language that seems eminently objective, sensible and right-minded. So it is with creativity – as with other ‘fuzzy’ constructs, the ‘idea’ of teaching creativity is ‘clear and easy to express and understand’ (Gorur 2014, 62), but the underlying reduction, reframing and instrumentalisation which are necessary to make it measurable, are subject to the ‘street light fallacy’ (Auld and Morris 2019) – complex constructs become too complex for economic indicators to capture so the focus remains on what can easily be measured.

The concept of creativity has until now been characterised by ‘messiness’, complexity and the lack of an agreed definition: has been tolerated and understood by the various stakeholders in education. That tolerance of ambiguity is appropriate given the essentially unpredictable and divergent nature of the concept but is clearly problematic for the OECD. We have demonstrated that the OECD has exploited the space allowed by this malleability to reconstruct it for its own purposes.

As Lucas et al (see above) and Gormley (2019) note, many teachers do not share the OECD’s conception of creativity. Consequently, delivering a model of creativity based on assessment and marking will face resistance. However, by reframing creativity as a form of human capital, amenable to measurement and ranking in league tables, the OECD may resolve its own challenges. Simultaneously, the framing of the test in cognitive terms will demonstrate ‘creativity’ among high performing East Asian nations; and in countries that have done poorly on PISA, for example the US, the addition of a creativity dimension, which can be linked with the all-important economic constructs of innovation and entrepreneurship, will ensure their continued support for an educational test that had appeared to some domestic US commentators as irrelevant (Carnoy and Rothstein 2015).

The ‘rebranding’ of creativity in this way will ensure that whatever ‘globalisation-driven changes’ happen in the world of education, PISA will still be there to measure them. In the Asian nations typically stereotyped as lacking creativity, the new measure may advance the spread of western pedagogies, already dominant in the global knowledge economy and the development of human capital (Wu and Tarc 2016); while in the ‘more creative’ west curricular time and pupil access will be directed at the ‘new’ measurable constructions of creativity, as has happened in England (Johnes 2017), accelerating the marginalisation of those school subjects traditionally associated with both creativity and the cultural domain – art, music, design and drama.

Disclosure statement

No potential conflict of interest was reported by the author(s).