‘DiscoverQuants’: Integrating Quantitative Methods (QM) and Substantive Teaching for First Year Undergraduate Sociology Students

Abstract

This paper considers the rationale for, design and outputs of a project, based at the University of Surrey UK and funded by the Economic and Social Research Council (ESRC), which sought to integrate aspects of teaching substantive and Quantitative Methods (QM) teaching across first year sociology undergraduate programmes using a blended approach. The paper considers the nature of concerns regarding teaching QM within social science undergraduate programmes. It goes on to describe the rationale for this project, its design and its primary outputs. We consider a range of data related to student attitudes towards studying QM at university as well as their perspectives on the project and the implications for practice.

Keywords:

• Quantitative methods
• integration
• blended learning

Introduction

This paper considers the theoretical basis, design and implementation of a series of curriculum innovations aimed at integrating substantive teaching and quantitative methods (QM) for first year undergraduate students. It was developed and implemented by the Sociology Department, University of Surrey, UK and formed part of a wider Economic and Social Research Council (ESRC) ‘curriculum development initiative’ which sought to build capacity in QM across the social sciences at the undergraduate level (ESRC 2012). Our experiences relate primarily to QM integration within sociology. However, our findings have implications for QM integration across other social science disciplines. This initiative responded to concerns that social science programmes have failed to prepare students in QM. Such concerns are neither new nor confined to the social sciences (Williams et al. 2008). Indeed, the teaching of research methodology in British sociology has been under scrutiny since the expansion of the discipline during the1960s (Burgess & Bulmer 1981). The ESRC, founded in 1965, expressed the need for a higher level of numeracy within the social sciences as early as 1969 (Platt 2012). Government anxiety about the (apparently) weak numerical skills of social scientists should be understood as part of a wider concern about the number of scientifically qualified personnel and a shortage of students interested in studying mathematics and the physical sciences (Williams et al. 2008). However, concerns have perhaps been expressed more commonly – and more forcefully – since the turn of the century (MacInnes 2009).

This paper begins by setting out the nature of concerns pertaining to the teaching of QM within social science undergraduate programmes and some of the consequences for students. We then set out the rationale for our project in light of these debates along with the project aims and objectives. We then turn to project outputs and outcomes. Our aim is for others to learn from our experiences. Through examining a range of qualitative and quantitative data (focus groups, pre- and post-questionnaires, and web usage figures) and linking our findings to those of others we also hope to contribute to the knowledge base of students’ attitudes to and experiences of studying QM. In considering the outputs and findings from our project, we hope to modestly advance pedagogical understanding of innovation in teaching QM.

Social science, QM and the undergraduate curriculum

A number of problems have routinely been identified in undergraduate QM provision for social scientists. In a ‘baseline study’ funded by the Higher Education Academy (HEA) Williams et al. (2004a) found that whilst QM was widely taught, standards were mixed and barriers to effective teaching – including level of language, the nature of data used, expectations of staff, quality of teaching and a shortage of qualified, motivated teachers – were identified. In his review of British QM teaching MacInnes (2009) drew attention to the low priority and low status which has been accorded to teaching QM; the routinised and basic nature of provision; and, the lack of integration into the wider curriculum. Ultimately, he surmises, social science students in the UK graduate with only a narrow base of QM skills, little confidence in applying them and few go on to make use of their skills. This situation, as Markham (1991, p464) succinctly noted some time ago, “is good for neither students nor society”. It is not good for students Markham argued because it limits their intellectual, social and career development, themes which resonate through recent reports by the UK government, funding bodies and learned societies (MacInnes 2009, British Academy 2012, Nuffield Foundation 2012). Many social science students, suggests the Nuffield Foundation (2012, p1), “leave university with inadequate quantitative skills and expertise in their applications to their chosen areas of study”.

There are two types of explanation regarding why social science has ‘failed’ to prepare students in QM. The first set emanates from factors linked to student predisposition and preparation and the second from factors linked to university resources, culture and traditions. Student experience of QM before they commence their degrees is somewhat mixed, but generally limited; many have not confronted maths for some time; and, they may have had poor experiences of learning it at school (Murtonen & Lehtinen 2003, Murtonen 2005, Williams et al. 2008, Falkingham et al. 2009, MacInnes 2009, Falkingham & McGowan, 2011). This situation will inevitably affect students’ confidence to engage effectively with QM at university. Indeed, students may well select social science subjects precisely to avoid dealing with numbers altogether (Markham 1991, Williams et al. 2004a, Williams et al. 2008, Falkingham & McGowan 2011, Nuffield 2012). This situation may be compounded by discipline culture, traditions and resources. The marginalisation of QM teaching is in part explained by demographic and intellectual trends associated with the development of the discipline in the second part of the twentieth century (see Burgess & Bulmer 1981, Bulmer 1989, Williams 2000, Williams et al. 2004b, Platt 2012). Early cohorts of sociology lecturers had themselves little training in QM, a preference for theoretical (or qualitative) work and (perhaps) animosity towards QM. This situation inevitably interacts with the resources made available for QM. In addition, requiring higher staff–student ratios and bespoke teaching material, good QM teaching is expensive and vulnerable to cost-cutting exercises (MacInnes 2009). Taken together the result is, as MacInnes (2009, p44) put it, that “the teaching base is fragile”.

The response then has been to seek to underpin the foundations of QM teaching. This is where our project fits into the landscape of QM teaching. In partnership with the Higher Education Funding Council for England (HEFCE) and the British Academy, the ESRC funded twenty ‘curriculum innovation’ projects in early 2012. Our project sought to integrate aspects of QM and substantive teaching for year one sociology undergraduates using a ‘blended learning’ approach. The following sections explain the theoretical basis of and the rationale for our project in more detail.

Project rationale, aims and objectives

The Department of Sociology at Surrey has a long history of teaching QM which is firmly embedded in its undergraduate programmes. However, our project was informed by the observation that increasing the amount of QM modules and skills taught to social science students does not guarantee that they will develop and apply them in practice (Markham 1991, Atkinson et al. 2006). Instead, research has revealed that teaching QM in substantive courses helps students to learn QM skills (Bridges et al. 1998, Atkinson et al. 2006, Howery & Rodriguez 2006, Wilder 2009). Atkinson et al. (2006) for example, reported on the effectiveness of a research module taught in a large introductory sociology class. Integration helped students learn to interpret sociological data and increased their awareness of the substantive issues which in their exemplar was race and gender inequality (Atkinson et al. 2006). Integration of this sort is rare in practice (Sweet & Strand 2006, MacInnes 2009, Wilder 2010) and concerns have been raised that QM skills are not reinforced across the curriculum, are nearly always confined to specialist courses or components of wider methods courses, and that the amount of time spent on honing QM skills compared to other areas of the curricula is limited (MacInnes 2009). This gives students the impression that social science is about essay writing and evaluating arguments and that QM is neither important nor relevant to the discipline (Bulmer 1989, Falkingham et al. 2009, MacInnes 2009). In designing the project we further drew on the observation that sociology – which asks questions about the social world and engages with issues that are relevant, memorable and meaningful for students (Atkinson et al. 2006, Sweet & Strand 2006, Wilder 2010) – is a ‘natural’ subject for blending QM and substantive teaching and that ideally QM and substantive teaching would be integrated early on (Atkinson et al. 2006, Wilder 2010). Our project also built upon previous studies which had prioritised and attempted ‘integration’. Previous initiatives – which in some instances have involved a separate department providing statistical training to sociology students and staff (e.g. Falkingham et al. 2009) – have interpreted ‘integration’ loosely. Integration, we suggest, must be conceived as more than implementing a series of strategies linked by a common goal. In its truest sense, integration involves weaving and combining QM and substantive teaching. It might involve redesigning modules to, for example, incorporate quantitative exemplars into substantive teaching and vice versa, using data to structure sessions or enabling students to conduct quantitative analysis as part of their assessment or the development of new modules and courses such as those that embed enquiry-based learning into the curriculum. Our project also acknowledged the need to cope with diversity within the student body. Some students might be confident in substantive issues but anxious about QM, or vice versa. Indeed, many integration efforts to date have assumed students are a homogenous group, yet students come from different backgrounds, have different levels of experiences of QM and employ different learning styles.

Developing the integrated curricula

The curriculum innovation focused (primarily) on the first semester of the first year and was built around grounding ‘exemplars’ – an existing point of contact between a substantive module and QM – from substantive modules into QM teaching. These exemplars were determined on the basis of discussion between QM lecturers and those delivering substantive content. We identified six exemplars topics (Durkheim and suicide; Weber and Protestant ethic; stop and search; inequalities in health; poverty; age and crime). This enabled the 12-week QM module to spend up to two weeks on each exemplar, as appropriate. Once these exemplars were identified, two further changes were made: first, the QM module (which is compulsory for all students on all our degrees) was adapted so that it was delivered using those exemplars; second, the substantive courses incorporated material taken directly from the QM module.

To illustrate, principally discussing his notion of ‘social facts’ and using his (quantitative) study of suicide as a way of exploring further his approach to sociology, our sociological theory module has two hours dedicated to Durkheim’s rules of sociological method. The sociological theory lecturer made two changes to the session. First, the existing slides were supplemented with materials from the online resource (see below). Second, the lecturer made reference to how Durkheim’s analysis will be explored further in the QM module. The relevant QM session then used data on suicide, and Durkheim’s analysis, to meet one or more of its learning objectives.

Implementing a ‘blended learning’ environment – ‘DiscoverQuants’

Reflecting the observed need to account for the differing backgrounds, experiences and learning styles of students we also developed a ‘blended learning’ environment. Working with a professional web designer we created an interactive online tool. Designed to provide students with an intuitive framework to structure their learning, the online tool was organised around the revised QM sessions and included material which covered both the QM and substantive area. For each QM topic we incorporated a set of revision notes, worked examples (using the relevant exemplars from the substantive courses), definitions of key terms, ‘quizzes’ (at different levels of difficulty), data sets and a range of other resources including relevant readings and videos. To engender greater engagement with the online tool and add a degree of feedback to students on their progress, a number of related strategies were used. Aspects of ‘gamification’ – to visually display to users the extent to which they had used the tool and whether revision questions had been answered correctly – were incorporated throughout the resource. Students were encouraged to ‘discuss’ any issues they had related to the quantitative methods on an online forum integrated within the resource which was monitored by a nominated member of staff who could ‘intervene’ to answer queries posed by students as necessary. Students could also personalise the resource with avatars. The tool can be seen at www.discoverquants4all.surrey.ac.uk/.

To further emphasise to students how QM is integrated throughout their degree, materials from the online tool were incorporated in the lecture slides in both substantive and QM lecture/tutorial sessions. Students were encouraged to use the online tool but our approach did not rely on the students accessing the blended learning environment as the same material was included in the lectures and students were able to meet the learning objectives of both substantive and QM modules without ever accessing the online materials.

Generating commitment from staff

Since previous research has demonstrated that resistance of teaching staff – who have different perspectives, preferences and priorities – can be a major barrier to attempts to integrate QM and substantive teaching (Howery & Rodriguez 2006, Wilder 2010) it was recognised that commitment from all relevant lecturers was essential. Our approach was to stress that the curriculum innovation required minimal changes to the teaching of substantive modules: we were principally looking to adapt existing lecture slides so that they reflected screenshots/materials from the online learning resource; module learning outcomes, content, assessment and feedback would all be unchanged; and lectures would not have to be significantly re-written. The project team were also available to assist colleagues in making the necessary adjustments to further reduce the burden on their time, if necessary. All year one teaching staff supported the aims of the project and agreed to coordinate the first semester teaching in accordance with the plan. As noted, this sympathy should not be assumed, a point we return to in the conclusion. Even so, like any significant curriculum restructuring, the project was not without challenges. During the project the QM lecturer who was involved in the preparation of the bid left their post as did their replacement, with obvious implications for ensuring continuity of provision and motivation to incorporate the curriculum changes. There continue to be challenges engendered by the ‘fragility’ of the QM teaching base and a buoyant labour market and opportunities for QM skilled staff. Nevertheless, the project was enacted in full for the first semester of the 2012/13 academic year, with students on our undergraduate Sociology, Criminology and Sociology, and Sociology Culture and Media degrees all exposed to the new integrated curriculum.

Student perceptions of QM, teaching and the project

In order to inform its development and monitor implementation, understand outputs and outcomes and elicit lessons learnt, a range of data were collected in the course of the delivery of the project. First, a series of focus groups were conducted with different cohorts of students and at different stages of the implementation of the curriculum innovation. One focus group was conducted with third year students as part of the development of the proposal to the ESRC. Two focus groups were carried out with the first year cohort prior to teaching and two focus groups were carried out with the same cohort post teaching. These facilitated in-depth discussions of student attitudes to QM, their experiences of learning it, views on the online (and other) resources (as relevant) and views about the relationship between substantive and QM sociology. Focus groups were transcribed and analysed thematically by the project team. Second, students filled in attitude questionnaires before and after completing the course. These questionnaires were distributed to students during the QM lecture and covered issues including attitudes to studying QM, self-assessed ability and relevance of QM to their studies and career plans. Fifty students completed the pre-questionnaire and 32 the post-questionnaire. The post-questionnaire also asked for free text feedback on the DiscoverQuants website and how it was used by students. Third, we examined website usage data to inform discussion on how the students were using it within their studies.

The observational nature of the data sources means we cannot make direct claims about whether the project led to changes in students’ perceptions of QM, their confidence and competence or their assessments of the linkages between QM and substantive teaching. Instead, taken together we use these data to descriptively consider how our students perceive QM, how they understood the relationship between QM and substantive modules and how the online tools were utilised. Since we are not using these data to consider change over time the focus group responses have been merged together and we do not distinguish between pre and post in the discussion that follows.

Student attitudes towards QM

The pre-course questionnaires asked students a number of questions to reveal their attitudes towards QM. To start we summarise student responses to questions related to their overall interest in studying QM at university and their confidence with statistics. A mixed picture was evident when considering whether students had a favourable view of statistics, with similar proportions stating that they agree, disagree and neither agree nor disagree with the statement ‘I will like statistics’. However, only 3 in 10 students disagreed with the statement ‘I am interested in learning statistics’. We have seen that interest and confidence in studying QM at university is shaped by limited exposure to mathematics before students commence university (Murtonen & Lehtinen 2003, Murtonen 2005, Williams et al. 2008, Falkingham et al. 2009, Falkingham & McGowan, 2011) and our students were asked to self-assess their competence in mathematics in the pre-course questionnaire. Only a minority of students were either very confident or very unconfident in their ability, with the majority reporting what might be seen as average levels of confidence.

Our students were also asked questions related to fear of QM, following extant research on the subject which has found that many students are fearful (Williams et al. 2008, Falkingham et al. 2009, Falkingham & McGowan 2011). Near equal numbers of students agreed and disagreed with the statement ‘I feel insecure when I have to do statistics problems’ and just under half agreed with the statement that ‘I am scared by statistics’ indicating a certain degree of apprehension amongst our sample. However, we also find that there are clear minorities of students who are confident and many who are somewhat ambivalent. With only half of students agreeing with the statements ‘statistics is a complicated subject’ and ‘I find it difficult to understand statistical concepts’ and just over half agreeing that ‘I will make a lot of errors in statistics’ many students (though again by no means all) do not find QM all that challenging. The challenge of QM was also a theme that arose in the focus group data. Indeed, noting that many had not studied mathematics recently respondents certainly expressed concerns about studying QM at university. That said some were not put off by the challenge, one focus group respondent stated:

It’s a little bit intimidating but I think it will be like a good kind of challenge, like for me it’s one of my most challenging areas … I’m not so strong with numbers but I think it will be a good kind of challenge rather than scary, a little bit intimidating.

Some of this might link to the nature of QM teaching. Reflecting Falkingham & McGowan (2011), who found students saw lectures as the least favourite method of delivery, our focus groups revealed something of a preference for small group teaching, discussion and practice of QM. One student noted “I prefer working in a smaller group ‘cos I think if you’re into, sorry, interacting with people then, then there’s like a conversation that sticks in my head a bit more than someone just telling me”. Our experience demonstrates that discussion and practice can be facilitated via online tools. Workshops, online workbooks and lecture material could all be easily embedded into such resources giving students greater opportunity to practice.

A further set of issues that might shape student attitudes towards studying QM at university is student perceptions of the nature of the discipline that they are studying and its content. As noted, students may well select social science programmes because of their dislike of numbers and preference for essays based content which they believe will characterise them (Markham 1991, Williams et al. 2004a, Williams et al. 2008, Falkingham & McGowan 2011, Nuffield 2012). For example, Williams et al. (2004a) found that three quarters of sociology staff who completed their survey thought students selected sociology degrees precisely to avoid numbers. In the pre-course questionnaire our students were asked ‘If you had a choice, how likely is it that you would have chosen to take any course in statistics’. Whilst some students clearly were keen the overall message from our study is that most were unlikely to have proactively selected it.

Some have argued that students may not see the value of competence in QM – especially in terms of the career opportunities (MacInnes 2009). Our pre-questionnaire data (set out in Table 1) suggest that students generally did see the value of QM though in some areas more than others.

Table 1 Value of statistics.

Statistics is worthless	%	N
Agree	8	4
Disagree	68	34
Neither agree nor disagree	24	12
Statistics should be required as part of my professional training
Agree	36	18
Disagree	12	6
Neither agree nor disagree	52	26
Statistics will make me more employable
Agree	76	38
Disagree	4	2
Neither agree nor disagree	20	10
Statistics is not useful to the typical profession
Agree	8	4
Disagree	66	33
Neither agree nor disagree	26	13
I use statistics in my everyday life
Agree	20	10
Disagree	34	17
Neither agree nor disagree	46	23
Statistics is irrelevant to my life
Agree	16	8
Disagree	54	27
Neither agree nor disagree	30	15

Few students agree that ‘statistics is worthless’, only 1 in 10 students disagree with the statement ‘statistics should be required as part of my professional training’, fewer than 1 in 10 agree that ‘statistics is not useful to the typical profession’, whilst more than three quarters agree that ‘statistics will make me more employable’. There was therefore a particularly strong sense that QM was important for employability, something that also came out in the focus groups (c.f. Williams et al. 2008). This casts some doubt on MacInnes’ (2009) claim that students are unaware of the career advantages that flow from QM skills. Instead, it might be that students do not intend to pursue careers that involve QM. This is supported by Table 2, which shows the results to the question ‘In the field in which you hope to be employed when you finish university, how much will you use statistics?’

Table 2 Future directions.

In the field in which you hope to be employed when you finish University, how much will you use statistics	%	N
1 not at all
2	6	3
3	25	12
4	43	21
5	16	8
6	6	3
7 a great deal	4	2

There is a clear minority of students who believed it likely that they would use QM in their career but the majority seem to think it unlikely or at least are unsure.

To summarise the above, it is generally accepted within sociology that students have limited skills in mathematics, little interest in QM and are fearful of learning it. We found quite mixed results, with clear ‘ambivalence’ towards QM. Students do see the potential value of QM. However, whilst there was certainly evidence of competence and enthusiasm for QM amongst some, many were not looking forward to studying QM, probably wouldn’t have selected it and do not seem to see their future careers in jobs that call for QM. Perhaps it follows that students do not necessarily see the links between substantive courses and QM, which we now turn to.

Student understanding of the links between QM and substantive social science

Primarily considered through the focus group data, the overriding message is that students saw the substantive and QM modules as quite separate and distinct. There were no differences before or after the course. That said it isn’t the case that all students fail to see the connections between QM and substantive teaching. There were isolated examples of students seeing the links. One focus group attendee stated “I do think quantitative methods are relevant to the theory courses and modules I attended, both sociological and media, because quantitative methods provide you with an insight of how you could conduct research that would produce data to agree or disagree with these theories”. The same student went on to state that “I do remember the lecturer giving some real life examples of how this could be used, which was useful to help see the connection”. Another student, part of the third year focus group, also revealed that she saw connections between the substantive and research content developing as she came to the end of the programme.

In first and second year, I think, it was very divided, because we were mainly looking at grand theories […] in the final year when we were looking at up-to-date data and we read articles and we look at their methods, it definitely becomes, ‘ah, research methods, I did that’. And during revising, ‘ah I remember that’. You do feel it’s finally coming together.

Of course, this did not reflect on our curriculum innovation. If on the whole students did not see clear links between the substantive and QM teaching some certainly did see links between sociologists – and sociology as a discipline – integrating substantive content and QM. One student stating “I suppose the point of research methods is to test the theories”. However, even here we see a clear disconnect between perceptions of substantive theories and QM, with an implicit suggestion from the student that research methods and empirical data are distinct from theory generation. This artificial separation of QM (and research methods more generally) from broader theoretical development in the eyes of students is a worrying reflection of the broader schisms that remain in the discipline of sociology, and demonstrate that more substantial integration is still needed, a point we return to in the conclusion.

Usage of and understanding of the online tool

We asked students, in the survey and focus groups conducted after the project, their views on the online tools. Student feedback and the usage data indicated that the resources were well used and that the students found the website to be useful.

Table 3 Visits to the DisoverQuants website.

Month	Number of individual visits	Average visit duration (minutes)	Number of unique visitors
Oct-12	312	14.67	170
Nov-12	266	13.52	103
Dec-12	100	14.19	65
Jan-13	308	30.54	92
Feb-13	4	4.22	3
Mar-13	1	0.00	1
Apr-13	8	27.71	2
May-13	5	4.69	5
Jun-13	3	3.61	3
Jul-13	3	0.07	3
Aug-13	26	20.29	15
Sep-13	79	6.85	47
Oct-13	156	12.25	120
Nov-13	360	16.16	219
Dec-13	31	3.87	28
Total/Mean	1662	11.51

Table 3 demonstrates that there have been 1662 visits to the website, with a maximum of 219 unique visitors per month and each visit lasting on average just over ten minutes. This suggests a significant degree of engagement with the web resource, and that the majority of students visited the site at least once (students who had access to the resource in October 2012 retained access to the resource in their second year as a revision tool which explains the high number of unique visitors recorded in November 2013). We should stress that the number of visits and length of visits were shaped by the point in the academic year, something we will return to. As well as the extent of student usage of the website we were also interested in the ‘depth’ of their usage, revealed in Table 4.

Table 4 ‘Depth’ of visit.

Page Depth	Visits (N)	Visits (%)
<1	46	2.8
1	293	17.6
2	530	31.9
3	107	6.4
4	181	10.9
5	58	3.5
6	93	5.6
7	55	3.3
8	63	3.8
9	25	1.5
10	40	2.4
11	25	1.5
12	34	2.0
13	19	1.1
14	25	1.5
15	7	0.4
16	11	0.7
17	4	0.2
18	3	0.2
19	3	0.2
20+	40	2.4
Total	1662	100

We can see that although there was a great deal of variance, some students were certainly making extensive use of the website visiting multiple pages.

Students made positive comments on its functionality, looks and the nature and depth of the content assembled. A number of student comments linked to the ‘quizzes’ embedded in the tools, generally seen as useful for practice, especially at exam time. Some students, albeit a minority, reported that the questions were too basic and would have welcomed more advanced questions. Others stated that they would have liked greater feedback on their answers on the quizzes on the online tool, rather than simply informing them whether they were ‘right or wrong’. We suggest that solutions to this need to be considered early on. This might be achieved through technology or through engagement from staff – via online discussion forums, or face-to-face discussions in seminars, for example.

Whilst the online tool was viewed to be useful, the student responses in focus groups and the free text parts of the post-questionnaire indicated they were using the website in ‘instrumental’ ways. First, they were using it when they were directed to in seminars under supervision of tutors. Second, they were using it specifically to aid revision and coursework. Analysis of the website usage data showed very clear peaks in usage in the period immediately before deadlines. Indeed, Table 3 demonstrates peaks at the beginning and end of the semester which indicates usage as the students are being introduced to the courses and then at the end when they are being assessed. Whilst it was not intended that the online tool should be primarily used in this way, some students certainly found the website useful for this purpose. Third, some students saw it as a repository for lecture notes. In a free text response to the post-course survey one student explained that it was “useful to help understand some of the material covered in the lectures as it explained it in a different way”. Others felt the opposite. One stated it “was sometimes a bit confusing as the descriptions and explanations were sometimes worded slightly differently to the lectures and the lecture notes”. It may be very difficult to shift this instrumental mindset and orient the students towards a more rounded usage of online tools. However, raising awareness and exposure to the online tools and what they offer in terms of breadth and depth of context and the ways that students could use them might help in this regard.

Implications for understanding students’ attitudes to and experiences of studying QM

We hoped to contribute to the existing knowledge base of students’ attitudes to and experiences of studying QM. Whilst it is generally accepted that sociology students have limited skills in mathematics, little interest in QM and are fearful of learning it, we found mixed results. Whilst minorities of students were highly positive and confident or negative and lacking in confidence, most students exhibited what might be best termed as ‘ambivalence’ towards QM. Students do see the potential value of QM, most clearly for their careers. However, most were not looking forward to studying QM, probably would not have selected it and do not seem to see their future careers in jobs that call for QM skills. Whilst not denying that some students lack skills and confidence, we are drawn to concur with Williams et al. (2008) that the issue for sociology might well be less to do with an overall numeric deficit amongst cohorts of sociology students, but more to do with a lack of student interest in QM. Some of this ‘ambivalence’ might be to do with their experiences of studying mathematics prior to university study and expectations of their programmes. As noted, it is generally taken that student experience of QM before they commence their degrees is mixed but limited; students may not have confronted maths for some time or had poor experiences of learning it at school and students may well select a social science degree programme precisely because of their dislike of numbers and preference for essays based subjects which they believe will characterise undergraduate study in sociology (and social science more generally) (Markham 1991, Williams et al. 2004a, Williams et al. 2008, Falkingham & McGowan 2011, Nuffield 2012). It seems quite likely that current cohorts of students believe sociology is not a numeric discipline and nearer an ‘art’ than a ‘science’ (Murtonen & Lehtinen 2003, Murtonen 2005, Williams et al. 2008, Nuffield 2012). Indeed, there may be a perception amongst schools, guidance teachers and school students, note the Nuffield Foundation (2012, p10), that the social sciences are an unattractive destination for students with good QM skills. Whilst Williams et al. (2008, p1003) suggest that “the views held by present undergraduates do not augur well for a methodologically pluralist discipline in the future, or more generally for key numeric and analytic skills sociology graduates can bring to other professions and occupations”, it may be that student perceptions of the subject and attitudes can be changed in order to facilitate student engagement with QM content once they arrive at university or to attract students who are numerically oriented in the first place. A couple of themes are worth noting in the current context.

It is clear that some of the problem starts in schools. Indeed, there have been wider expressions of concern about poor skills in mathematics amongst British school leavers (Hodgen et al. 2010, ACME 2011, Vorderman et al. 2011) and (amongst other things) the implications for developing QM skills within the university sector (MacInnes 2009, Nuffield Foundation 2012, British Academy 2012). For the British Academy (2012, p3):

The problem starts in schools. Too many students enter higher education with poor numerical skills, little confidence in their mathematical abilities or an appreciation of their relevance. This is a severe handicap in developing quantitative skills more generally.

It may be that wider government initiatives to raise the numeracy of the whole student population will have positive implications for attempts to raise QM capacity in undergraduate programmes. Much more specifically, the prospect of developing links with schools to reinforce the status of sociology as a discipline with QM content and so suitable for students with numerical skills has been mooted (MacInnes 2009, Nuffield Foundation 2012). This is a theme evident in the Quantitative Methods Programme (hereafter ‘QM Programme’), an investment of some £15.5 million from the Nuffield Foundation, the Economic and Social Research Council (ESRC) and the Higher Education Funding Council for England (HEFCE), which aims to promote a ‘step-change’ in quantitative methods training for UK social science undergraduates. The QM Programme promotes institutional change, the production of quantitatively skilled undergraduates and the creation of links between undergraduate and postgraduate training to benefit academic research and meet the needs of the wider labour market (Nuffield Foundation 2012, p2). It aims to support QM capacity in a small number of universities with pre-existing strengths in order to signal the importance of quantitative skills to schools, school students, and undergraduates and to promote work with schools so students better appreciate the role of QM in social science degrees, thus creating pathways to system change (Nuffield Foundation 2012).

The current economic climate may start to shift student attitudes, expectations and choices. It has been suggested that students may start to become savvier about developing skills transferable into the job market whilst they are at university and in so doing they may select QM, especially if the benefits of doing so are made clear to them (MacInnes 2009). In fact, comparatively little research has unpicked students’ perceptions of the skills that they generate from their university studies, the factors that influence their perceptions of employability or the efficacy of methodologies in promoting optimal skills development (Tibby 2012). However, we do know that students lack understanding of the skills employers want (Tibby 2012) and find it difficult to translate their skills and experiences into the language that employers value (Knight & Yorke 2003). This may represent a particular challenge for social science students. Williams et al. (2008) found that 50% of sociology students were unsure about whether employers thought sociology was a ‘good degree’ and a further 22 per cent believed they did not. However, as Platt (2012) notes, any student move towards making decisions on the basis of perceived employability outcomes may well alter student choices away from sociology as well as module choices within it. At the time of writing we do not know much about prospective student understanding of the discipline of sociology, their future career aspirations and how these will shape university choices. In considering this matter and its impact on the development of QM capacity within sociology teaching, we concur with Williams et al. (2008) that we need to know more about who chooses to read sociology at university and “to what extent attitudes toward the status of sociology as a ‘science’ or ‘arts’ subject change through the undergraduate career as a result of exposure to methods or styles of sociology” (Williams et al. 2008, p1016). This is something that the QM Programme may help to unpick.

Implications for advancing pedagogical understanding of innovation in teaching QM

In considering the outputs and findings from our project, we hoped to modestly advance pedagogical understanding of innovation in teaching QM. Our students differed in the extent to which they made links between substantive sociology and QM, but we do know that the perception persisted that QM and substantive sociology were separate. This raises the question of why? One issue was no doubt ‘dosage’. Our project consisted of drawing together six exemplars where QM and substantive courses overlap. Although we aimed to embed greater integration between QM and substantive teaching, the structure of our programmes still treat QM and substantive teaching separately. Crucially, assessment remained distinct. Reflecting the observations of MacInnes (2009) (and others) in one (post-project) focus group students discussed at some length how QM was used neither in substantive teaching nor in assessment. This, they stated, fuelled their perception that they had little in common. Taken together the threads of our innovation may not have been enough for the students to see the links between the QM and substantive teaching, and we believe that more needs to be done to reinforce the linkages over time. It is worth bearing in mind that these students had only completed one semester at university, with data from our final year focus groups suggesting that the linkages between substantive subjects and QM become more visible over the course of the degree.

Certainly, we do not advocate abandoning the approach given the weight of evidence on the matter, quite the opposite. The problem of ‘compartmentalising’ QM teaching is being considered by the QM Programme. It acknowledges that “addressing this shortage of social science researchers will not be met by developing additional ‘stand-alone’ modules of statistical training” and stresses the integration of QM at all stages of degree programmes, deeper and more frequent exposure to QM and greater embedding of QM techniques within substantive fields (Nuffield Foundation 2012, p14). A welcome perspective, it is nevertheless clear that any attempt to diffuse QM teaching through the undergraduate curriculum represents a challenge. As well as a practical challenge (as demonstrated by the outcomes of our project) the views of staff teaching substantive modules (who have different perspectives, preferences and priorities) can be a major barrier (Howery & Rodriguez 2006, Wilder 2010). Indeed, there has been enduring tension between theory and methods, ambivalence to the need to empirically test theoretical ideas and resistance to the acquisition of skills with which to do so within the culture of sociology (Bulmer 1989). The lack of quantitative research, poor skills and even a positive hostility towards QM within British sociology has often been remarked upon (Platt 2012). Readers will not be surprised to hear us suggest that the integration of QM and substantive teaching will be difficult to achieve in isolation of wider cultural changes within sociology. As Bulmer & Burgess (1981, p589) put it “methodology teaching will continue to languish as long as empirical inquiry is not adequately integrated into the main substance of core sociology”.

Concluding comment

Mooted as a promising approach to generating positive outcomes in QM teaching, there remain questions about the theory and practice of linking substantive and QM teaching. We have described how our approach to integration stressed points of commonality between QM and substantive sociology and the use of exemplars in a blended learning environment. The project gave us an excellent opportunity to (re)consider the focus of our QM and substantive teaching, develop new tools and to engage in research on students’ attitudes towards and expectations of QM. Whilst we have seen that the project did not have all of the outcomes that we expected, we have to be realistic. The nature of contemporary QM teaching is embedded in the interaction of historical and cultural traditions within the discipline, the resources that have been made available and student perceptions, attitudes and experiences. Since QM teaching cannot be understood as phenomenon that operates independently of the discipline as a whole, any response to the perceived problems within the capacity of QM teaching needs to be understood within the wider disciplinary context which produces and reproduces it. As Bulmer & Burgess (1981, p589) put it many years ago “the future of methodology teaching will be determined within the discipline as a whole”. Nevertheless, this innovation and others like it provide useful insights and lessons on which we can draw in taking forward QM teaching into the twenty-first century.