Teaching Quantitative Methods

As Payne points out, the failure of most undergraduate social science degree programmes in the UK to give students a good grounding in social statistics has been a chronic problem for some forty years. For such a situation to be so enduring there must be substantial reasons for it: those involved would only tolerate it if they did not share the perspective that it constituted ‘failure’. These reasons must go beyond the pedagogical, technical or intellectual challenges of teaching statistics. They must also account for why, for most university social science faculty, teaching statistics has not been any kind of priority. They must account for why, for most students, this has not only been no disappointment, but rather a source of relief. And they must account for why those who employ social science graduates have been content to recruit those with few quantitative skills.

It is tempting to turn to the epistemological battles of the late 1960s and 1970s for an explanation. At least in Sociology there were many voices who argued that quantitative evidence as such was of little use, and Wright Mills’ influential Sociological Imagination (1959), whatever its other merits, and whatever the actual view of its author, encouraged such an attitude. ‘Quants’ was not only seen as ‘positivist’ or ‘empiricist’ (terms used with little consistency or clarity of meaning) in its application, but as implying an ‘uncritical’ attitude, indifference or even hostility to social change and a focus on trivial or piecemeal substantive issues. Some took this further and rejected the idea that the social ‘sciences’ could be such, but were closer to the humanities than the ‘natural’ sciences, so that the whole issue of ‘method’ was of questionable relevance. As Williams et al. (2008) have shown, this certainly seems to be the view of sociology that students arrive at university with. Debates in Politics were rather different, partly because of the heterogeneity of subject matter (it is easier to study party behaviour than voter behaviour using qualitative evidence) and partly because of its more modest growth in this period. Social anthropology, with some notable exceptions, had never been much interested in numbers.

However, there must have been more to the evaporation of statistics from the undergraduate curriculum than this. There are perhaps other reasons, although it would be difficult to quantify their effect. One was the poor standard of methodological training of most faculty recruited in the rapid expansion of the social sciences in UK universities at that time. Doctorates were mostly artisanal affairs with little attention to systematic training until the reforms of the 1990s. A second reason was the early specialisation of school students in subjects of study so that those intending to study the social sciences were increasingly unlikely to be strong at mathematics, unless they intended to read Economics. This combination was hardly likely to foster any interest in statistics.

A third factor was the way in which the social sciences came to understand theory and the importance allotted to it in undergraduate programmes. On the one hand theory tended to be taught in specialist theory modules that gave pride of place to what might be called ‘grand theorists’ who offered some kind of universal perspective on understanding society. Here I have in mind not just the ‘founding figures’ such as Weber or Marx, but also contemporary attempts to make general statements about the nature of the social, such as Rawls, Giddens, Bourdieu, Beck or Butler.

In contrast to the natural sciences, where ‘theory’ more usually takes the form of empirically testable propositions, ‘grand theory’ typically offered more comprehensive accounts that often included the implicit or explicit claim that such testing was largely irrelevant because the essence of the theory was to render an interpretation of the social world that only the theory itself made possible in the first place. This had its impact on the study of substantive empirical issues because the latter could now be treated as a matter not of using empirical evidence to accept or reject competing theories but of showing how rival theories might interpret much the same evidence differently. It would be exaggerating to say that a lot of social science paid relatively little attention to empirical evidence, but it is striking that much of the evidence that it dealt with came to focus on different kinds of micro-accounts with little concern about their generalisability.

A fourth factor was that employers were happy to recruit innumerate social science graduates. There is indirect evidence that numerate graduates fare better in the graduate labour market, but it is difficult to know how far this is because of their numeracy or other factors (the numerate may also have other skills that make them attractive). However, not least because of their endless practice in writing essays, social science students could usually be relied on to be good not only at ‘thinking outside the box’ but also at communication skills. This may now be changing as the digital economy produces increasing amounts of transactional and administrative data which organisations seek to understand and use. Increasingly employers will be looking for graduates who can communicate and count.

A final factor has been the remarkable inattention to methods teaching – quantitative or qualitative – in UK undergraduate social science. Typically natural science undergraduates have three hours a week of labs in their first year, in which they learn about how to ‘do’ science. Their social science counterparts often have none. Methods is usually one or at best two units in second year with typically low status given to its teaching, and all too often little coherent attempt to integrate it with other curriculum content. What arguably ought to be the foundation of a degree programme can become a kind of bolted on accessory. As Harris et al. show in their survey of geography, this situation has been compounded by the remarkably lax QAA benchmark statements that many social science disciplines have produced. Universities themselves may also share some of the blame. It is clear that the social sciences have usually been treated as being much closer to the humanities than natural sciences, not just in terms of their object of study, but in how that study proceeds. With the partial exception of psychology, ‘lab’ work and a data infrastructure have not been seen as fundamental to the social sciences. Like the humanities scholar, all the social scientist was imagined to need was a desk.

One damaging, consequence of all this is that what ought to be a purely heuristic classification-‘quant and qual’-has come to be seen as a methodological and epictomological one, certainly by students, and all too often by staff. No empirical study in the social sciences can be purely quantitative. Before they can be counted phenomena have to be defined and categorised. Conversely, it is difficult to imagine the wider significance of any ‘qualitative’ study no matter how brilliant the interpretation or ethnography offered, if a case cannot be made for its plausible generalisation to other contexts, situations or periods. Generalisation without some form of quantification is hard to imagine. In practice, when faced with the need to find empirical material, such as for a final year dissertation, students overwhelmingly choose what looks like a good compromise, the ‘semi-structured’ interview, which will hopefully elicit sufficient material to make some post hoc observation on the issue at stake. The emphasis is typically on the ‘semi’ rather than the ‘structure’, and all too often takes the form of inviting respondents to become the social scientist by revealing their ‘understanding’ or experience of the issue at stake.

Given these circumstances, dragging statistics back towards a more central position in social science degree programmes is a daunting task. However, it is one that is being made at least a little easier by the gradual accumulation of evidence about what seems to work best in different situations, by the exchange of knowledge about ‘best practice’, by the growth of online and other resources to help such teaching, and by the growing realisation amongst students that ‘QM’ skills are worth having. This last point however, needs to be qualified, as Bullock shows, by the recognition that students who see the value of QM skills quite clearly, may nevertheless not see themselves as someone who is capable of developing such skills.

How might we summarise what we (think) we knows. Reviewing the contributions to this special issue, and the literature these contributions draw upon, I think we can safely make the following arguments.

Students learn better if they are convinced of the relevance of QM to what they are interested in. This is easier to recognise in theory than it is to achieve in practice. The social world is typically a complex place, associations and patterns do not jump out of the data and announce themselves with the clarity that students are sometimes used to encounter in the world of social theory. There is hardly any shortage of engaging examples of the power of quantitative evidence to trump intuition, conviction or assumption. The challenge is to enable students to grasp that such a confrontation is not just another ‘compare and contrast’ exercise as applied to different schools of thought, but about the way empirical evidence stands behind everything we might claim to know about the social world.

This does mean that ploughing through levels of measurement and the fundamentals of univariate and bivariate descriptive statistics is probably not the best place to start, no matter how much sense it makes in terms of teaching the fundamentals. Technology can play an increasingly powerful role. Visualisation is a key resource, as Signoretti and Milligan et al. show. The work of Chris Wild and colleagues at Auckland has been pathbreaking in this regard. They have taken the visualisation of data beyond the presentation of results to the demonstration of the whole process of data analysis, including that most difficult concept to have students grasp: the sampling distribution.

Students learn better if they are confident of their ability to learn. Excluding Economics, only around 10% of social science undergraduates arrive with an A-Level in Maths. A much higher proportion, probably the majority, arrive with either a fear of maths or at least the conviction that they cannot ‘do’ numbers. This problem may not be of universities’ making, but it is universities that have to deal with the consequences. The STEM subjects, whose recruits typically have A-Level Maths, have invested heavily in what is in effect remedial maths education which assumes very little prior knowledge. What can social science teachers do? Perhaps the fist priority is to disabuse students of the idea that statistics is all about numbers. It certainly uses them, but QM is more about the logic of evidence handling than fancy maths. However, students still need to be comfortable enough with the numbers to keep their focus on what it is that they are doing with them and why, not the machinery of the calculations. There are some basics that students do need a firm grasp of, but they are all in GCSE Maths: fractions, decimals; rules of addition, subtraction, division and multiplication; powers and the concept of an equation. That’s it.

Maths anxiety shades into a rather different kind of anxiety when learning statistics: there can be answers to problems that are straightforwardly correct or incorrect. Social science students used to writing essays and other discursive work are aware that they may produce a poor piece of work, and may worry about clearly identifying exactly what makes a good essay (hence the salience of ‘feedback’ in NSS scores) but they don’t foresee being told that an essay is just ‘wrong’. They have accordingly little fear of ‘failure’. In my experience this makes many absolutely able students disproportionately anxious about failure when faced with questions that have clearly correct and incorrect answers. One solution to this is frequent short assessments, although it substantially increases the teaching workload. Another option is to think about novel ways to introduce elementary ideas, as Cohen illustrates, emphasising active learning and class activities rather than a traditional lecture format: to the extent that students can ‘discover’ concepts like levels of measurement or a frequency distribution for themselves, they are more likely to master the idea.

As Milligan et al. make clear, diversity of ability, especially related to the statistics content of other courses students may be doing, is a major challenge. Material that strong students find so straightforward as to become boring goes over the heads of weaker students, or comes at too fast a pace for them to assimilate and understand it. There is no easy solution to this, but both blended learning and peer learning can help. Where it is possible to get stronger and weaker students to work collaboratively the weaker students not only learn from their peers, they force the latter to clarify and deepen their own understanding. Blended learning, where course material is produced online, can also help weaker students to go at their own pace and allows resources to be focused on them, with extra tutorials or computer labs.

Embedding and integration. For several years now, there has been a desire to rescue QM from the ‘methods’ course silo and embed or integrate QM into substantive courses. This is easier said than successfully done. Teachers of non-methods courses have to be persuaded of the benefits of ceding some of their class time to methods issues. Assessment has to be changed to accommodate the new material. Bullock explores many of these issues. Perhaps the major challenge here is to persuade colleagues of the virtue of paying more attention to the way in which empirical evidence is treated: is it there to illustrate theory or to test it? Is the relevant discussion about the logical and conceptual coherence of alternative theories, or about their capacity to make sense of the empirical evidence, or about the real difficulties of securing the kind of evidence that is needed to settle debates in the literature?

Enthusiasm. There is one final resource that is perhaps the most important of all: the enthusiasm of teachers. There is nothing like excitement to stimulate students’ efforts to learn. When they grasp the formidable analytic power that even an elementary grasp of statistics can give them, and when they discover they can master skills they may have assumed were beyond them, students can come to share this excitement.

Over the last few years there have been several meetings and workshops that have brought together university QM teachers from across the UK. I have always been struck by the energy and commitment with which participants face the difficult, sometimes lonely and hardly prestigious task of improving QM teaching. I think that energy and commitment comes from the conviction that, ultimately QM is exciting and subversive. This is the bug that we hope our students can catch too. The articles in this special issue will hopefully help us to help them do just that.