Convincing Students? Quantitative Junkies, Avoiders and Converts on a Cross-Disciplinary Course Using Quantitative Narratives

Abstract

Amidst growing concern about the shortage of social science undergraduate students with even basic quantitative methods skills, student apprehension is recognised as a barrier to learning quantitative methods. A recent ESRC-funded project has sought to overcome such fear and anxiety through the design of a cross-disciplinary social sciences unit for first-year undergraduates. The unit aimed to capture students’ imaginations by the use of ‘quantitative narratives’ – descriptions of current social issues or controversies that allow quantitative concepts to be introduced in a contextualised way. This paper presents findings from the qualitative evaluation of the unit. It considers the attitudes and experiences of students who covered a spectrum of social science subjects, self-cited levels of confidence and prior experience of statistics. A typology of students taking the course is presented, revealing the challenge of meeting the needs of all students. Conclusions consider the implications of this evaluation both for the development of quantitative methods curricula and wider considerations for cross-disciplinary teaching in higher education.

Keywords:

• quantitative methods
• teaching
• higher education

Introduction

In the UK, various reports have highlighted the lack of suitably trained social science students with the quantitative skills required either for the workplace or for higher levels of study. The problem begins in schools and is compounded by recent trends away from quantitative social science in universities (with the exception of economics and psychology departments) (ACME 2011, ESRC et al. 2010, Hodgen et al. 2010, MacInnes 2010).

As part of a coordinated and sustained response to this skills shortage, the Economic and Social Research Council (ESRC) together with the Higher Education Funding Council for England and the British Academy, funded a curriculum initiative programme to improve numeracy, statistical literacy and students’ engagement with quantitative methods. The ‘Convincing Stories? Numbers as Evidence in the Social Sciences’ unit about which we report here was set up as part of the initiative. The unit was run over 12 weeks, for two hours per week. It was available to all first year undergraduates across the university but was specifically aimed at social science students. Seventy-five students signed up for the unit in its inaugural year (over 100 in the subsequent year), from a range of disciplines that included geography, childhood studies, social policy, sociology, and psychology. Each week the unit aimed to introduce a statistical concept to students, through the use of a contemporary story or problem, to some degree modelling itself on popular books like Best (2012) or Dilnot & Blastland (2008). Examples included an overview of the manipulation of data and charts in newspaper stories, consideration to whether the reform of university finance in England (the introduction of the £9000 per year fees) were as ‘progressive’ as the Government white paper claimed, the introduction of ideas around sampling and measurement through discussion of community cohesion, and regression analysis through different conceptualisations of good parenting. The unit description, accessible to students choosing the course, states:

Quantitative methods are central to social and scientific research, to business and to industry, and knowledge of them is a transferable skill that is attractive in the jobs market. This innovative unit, sponsored by the British Academy, Economic and Social Research Council and the Higher Education Funding Council for England, offers an introduction to quantitative social science, looking at how numbers are used (and abused) to create “stories” in the media, public policy, and in social and scientific debate. The aim of the unit is to prepare students for the sorts of methods and techniques they will encounter in their own discipline by discussing and debating the ideas and concepts that are used to create evidence in an uncertain world, and upon which decisions are made. The unit will encourage students to engage critically with research and debate in their own subject areas, placing them in a better position to learn quantitative skills, to conduct their own research and to enhance their studies. This is not a class on statistics but a class about how and why numbers are used in society. Students who have little or no interest in quantitative methods, who are anxious about mathematics or who simply want to get a head start in their studies are especially welcome on the unit. (University of Bristol 2012)

The important question that we address here is whether we lived-up to our own marketing, offering a course that would both help and engage students. As such, this paper presents findings from a student-focused evaluation of the Convincing Stories unit, with a particular interest in the implications of the cross-disciplinary nature of the unit. The evaluation took a mixed-methods approach, using pre- and post-course surveys, learning diaries from students, and a series of focus groups over the duration of the course. It is important to note that the students were active participants in qualitative research taking place on a quantitative course. This appeals to our dislike of creating unnecessary divisions between quantitative and qualitative approaches, instead fostering an appreciation of a range of social research methods. Furthermore, it enables us to understand in much more depth how useful our chosen approach is in teaching quantitative methods – much more so than a traditional end-of-course questionnaire.

The challenge of teaching statistics in the social sciences

Teaching quantitative methods to students in the social sciences brings a number of potential problems to the fore. Many students are anxious about their ability to work quantitatively, and do not enjoy it (e.g. Bridges et al. 1998, Jackson & Johnson 2013). Students may struggle to see the point of learning quantitative methods and statistics, and not understand how it may be relevant to their discipline or to their future career (Hannover & Kessels 2004). These factors can affect the students’ level of engagement, motivation and success in their learning (Murtonen et al. 2008, Ramos & Carvalho 2011). The research literature has suggested that teaching quantitative skills through substantive, subject-based content (rather than directly introducing abstract quantitative content) can help reduce anxiety and frustration (Bridges et al. 1998, Jackson & Johnson 2013). Giving students opportunity to construct their own meanings, discuss problems with their peers, and recognise and confront their own mistakes can help support successful learning (Garfield & Ben-Zvi 2007). Gelman & Cortina (2009) seek to demonstrate the relevance of quantitative approaches across the social sciences whilst textbooks such as Frankfort-Nachmias & Leon-Guerrero (2011) and Marsh & Elliott (2008), although more traditionally focused, nevertheless make considerable effort to teach statistics within a social and public policy context.

Teaching across disciplinary divides is widely acknowledged as potentially problematic in the research literature. Academic disciplines can be seen as different cultures, with their own epistemologies, languages, assumptions, and methods of teaching (Bradbeer 1999, Nikitina 2005, Spelt et al. 2009). Universities are usually structured according to subjects (Eisen et al. 2009), with little opportunities to work in a cross-disciplinary way. As students become inculcated in their ‘home’ discipline (and English education becomes specialised at a relatively early stage), transgressing disciplinary boundaries and working with students from other departments can become extremely difficult (Woods 2007). It is not just for students that disciplinary silos become problematic. Communication between teaching staff is crucial, so that they understand each others’ own disciplinary assumptions and processes, and can develop a common language and common understandings of processes (Jackson & Johnson 2013, Keebaugh et al. 2009, Rylands et al. 2013, Spelt et al. 2009, ). Integrating statistics teaching across subject or disciplinary boundaries is not necessarily straightforward though: Garfield & Ben-Zvi (2007) point out that statistics is a discipline in itself, but also cuts across other disciplines – and is seen as a part of those other disciplines.

The range of students taking the Convincing Stories unit creates an interesting dynamic therefore. They are from different disciplines, each with their own disciplinary interests, and potentially with very different ways of thinking about quantitative methods. One part of what we were trying to do was to create a greater understanding and appreciation of shared interests across quantitative social science. This, in itself, can be important to the students’ learning since much of social science is now interested in the co-production of knowledge. Teaching a range of students across disciplinary boundaries presents an exciting challenge.

In the context of increasing cross-disciplinarity, there is a need to consider this challenge in more detail. Few researchers have explored the complexity of teaching quantitative methods to such a range of students: from across several disciplines and with such different starting points. Ramos & Carvalho (2011), however, offer some insight into the variety of undergraduate students who study quantitative methods. Their research, conducted in a Portuguese university, profiled students in four different categories. Some enjoyed the challenge of the subject, and recognised the importance of quantitative methods for their future career. Others were bored by it, obliged to do it by the demands of their course but felt that it was not important for their future. A third category struggled with the subject matter, although believed that it was important for their degree and career to do well. A final group felt at ease with the subject, and attributed much importance to their success in their study of quantitative methods.

Ramos & Carvalho begin to highlight the range of abilities and attitudes towards quantitative methods that can exist among undergraduate students. There is still much to explore, however, about how this relates to the teaching of quantitative methods, particularly to a cohort from across a range of disciplines. The current study provides a deeper understanding of first year undergraduates’ attitudes towards and confidence in quantitative methods, and how these underpin their engagement with the field.

About the course

The content of the unit as it was taught to students in the year 2012–13 can be viewed at http://convincingstories.wordpress.com/. The aim, it will be recalled, was “to prepare students for the sorts of methods and techniques they will encounter in their own discipline by discussing and debating the ideas and concepts that are used to create evidence in an uncertain world, and upon which decisions are made” (University of Bristol 2012). Put another way, the aim was to teach students some methods and principles of statistics but doing so in a way that was to some extent ‘disguised’ (or, rather, made more approachable) by the quantitative narratives. In regard to the statistical content, we were not concerned to provide a curriculum progressing through description, inference and relational statistics and passing through probability in a classically textbook style. On the other hand, we did want to present some ‘big ideas’ about uncertainty, how to present data effectively, thinking through the processes of analysis to include conceptualisation, measurement and sampling, the basic ideas of inference, how to explore relationships but also the difficulties of establishing causation. The content of the classes are summarised in Table 1. Some of the topics are quite advanced. We were, perhaps, naïve in our belief that complex topics can be taught simply. As will become evident in our analysis of the student feedback (below), some students found the classes to be too challenging.

Table 1 A summary of the ideas taught in the Convincing Stories course.

Session	Story	The ‘take home’ ideas
1	Social Science in an Uncertain World. Or, fizzy drinks increase depression?	Measurement, data and analysis can be used to test ideas. A lot of statistics is about trying to handle uncertainty, to make comparisons as meaningful as possible, to control for other possible causes, and to only report differences between groups when the balance of evidence is that the difference is unlikely due to chance.
2	Progressive politics in bittersweet Britain?	Graphs and tables can be very persuasive and are an essential tool for drawing knowledge from data. Different presentations create different interpretations and conclusions. There is perhaps a tension between graphs (and infographics) that aim to grab the attention and look good and those that present information simply and honestly and allow the reader to draw their own conclusions.
3	Community Cohesion: Will it really mean an end to rioting?	Presented the stages of defining a concept (community cohesion), then measuring it, considering how valid the measurement is and issues around sampling. Also that for any of these stages, differences of approach could lead to differing results and opinions.
4	IQ research prompts warning over drinking alcohol during pregnancy.	Built upon the themes of the previous session, focusing especially on media reporting and the way that findings given in scientific journals that are always tentative and subject to modification by the evidence can easily become sensationalised.
5	Did a hospital really kill its patients? Making judgements under uncertainty.	Re-introduced ideas of natural variation and uncertainty. Used funnel plots to introduce confidence intervals, standard errors and statistical power. Discussion of good and bad estimates in relation to bias and precision.
6	Does Labour rely on the votes of benefit claimants?	Introduced the idea of relating two variables by cross-tabulation and also inferential statistics to question whether what is observed differs from what might be expected (i.e. the chi-squared test).
7	What makes a good parent?	Extended the previous session to consider the idea of ‘controlling’ for more than one variable – i.e. regression analysis.
8	Causality in the social sciences (or why we need longitudinal data).	The risk of jumping to false conclusions about one thing causing another and the difficulty of actually establishing causal linkages. The differences between cross-sectional and longitudinal analysis, and the importance of the latter in measuring neighbourhood effects.
9	Research, evidence-based policy and systematic reviews.	The distinction between primary and secondary research, and the use of systematic reviews for evidence-based policy as a means to evaluate and form decisions from across studies.

For assessment, the students were set two assignments. The first was an individual project that encouraged students to look at either Significance magazine (http://www.significancemagazine.org/) produced jointly by the American Statistical Association and Royal Statistical Society, or the ESRC’s Britain in magazine to look for an article of relevance to the student’s own discipline and to reflect on how statistical methods (in the broadest sense) were being used to tell a story. The second assignment was a group presentation, requiring students to look at data available from the Guardian’s data store (www.theguardian.com/data), to consider a research question that might be applied to it, to contextualise that question within a wider research literature and to give some indication of the sorts of statistical procedures that would be required to answer the question. Students were not required to complete the procedures, just to show knowledge of them. They were however required to create a suitable graphic from the data, using open source resources (specifically, Google Docs and Charts, to which the Guardian’s data are linked). The group presentations were peer assessed, with students providing feedback on each other.

All lectures were recorded using Panopto, which is a sound and screen capture system available to us as part of the university’s trialling of the system. The presentations available on the website are the raw files, as recorded at the time. There has been no editing or post-production of them. Despite the recording, students were required to be physically present in the classes; a registration was taken and penalties were threatened (although not actually invoked) for non-attendance. This was because we wanted all students to be taking part in the classes, contributing to discussion or other in-class exercises.

In planning the classes, we had intended to make use of the videos and interactive resources available at http://economicsnetwork.ac.uk/statistics. In practice, we found it hard to integrate them with our quantitative narratives. We would, however, still recommend them as a useful learning resource for students.

Methods

The mixed-method evaluation of the unit comprised several facets: pre- and post-unit questionnaires, learning diaries, and focus groups with students at the start, middle and end of the unit. All were designed to explore students’ expectations, levels of confidence, engagement, and motivation in exploring and using quantitative ideas.

Data collection

At the start of the first lecture of the unit, and at the end of the unit, students completed a questionnaire (see Tables 2 and 3). In addition to questions about the unit, both questionnaires asked whether the student had taken any A levels with a statistical element, what degree course they were taking, and whether they had taken any units with a statistical element for their degree so far.

Table 2 Student pre-unit questionnaire.

Question	Details
Q1.	Why have you chosen to take this unit?	Open-ended question
Q2.	What are your expectations of the unit?	Open-ended question
Q3.	Do you have any specific concerns about the unit?	Yes/No	If yes, please give details.
Q4.	Overall, how confident, or not, are you with using and manipulating numbers to tell a story?	Very confident/fairly confident/not very confident/not confident at all	Please explain why you have stated this confidence level.
Q5.	How likely, or not, do you think you are to use quantitative methods in your future studies (for example, taking up other optional quantitative units or a quantitative based dissertation)?	Very likely/fairly likely/not very likely/not likely at all/I have no idea	Please explain why you have stated this level of likelihood.
Q6.	Did you take maths or statistics A level?	Yes/no/another subject with quantitative elements (please specify)	If yes, please give details.
Q7.	Have you taken any units so far in your degree with a statistical element?	Yes/no	If yes, please give details.
Q8.	What course are you currently enrolled in?	Psychology/childhood studies/sociology/geography/social policy/economics/other (please specify)

Table 3 Student end-of-unit questionnaire.

Q1.	How far has the unit met your expectations?	Very much so/to some extent/not much/not at all	Please explain your answer.
Q2.	Looking back, do you think that you would choose this unit again?	Yes/no	Please explain your answer.
Q3.	If you had any specific concerns at the beginning of the course, how far have these been addressed during the unit?	Very much so/to some extent/not much/not at all/not applicable	Please explain your answer.
Q4.	Overall, how confident, or not, are you now with using and manipulating numbers to tell a story?	Very confident/fairly confident/not very confident/not confident at all	Please explain your answer.
Q5.	How helpful, or not, was the written assessment in developing your statistical awareness?	Very helpful/fairly helpful/not very helpful/not helpful at all	Please explain your answer.
Q6.	How helpful, or not, was the final assignment in developing your statistical awareness?	Very helpful/fairly helpful/not very helpful/not helpful at all	Please explain your answer.
Q7.	How likely, or not, do you think you are to use quantitative methods in your future studies (for example, taking up other optional quantitative units or a quantitative based dissertation)?	Very likely/fairly likely/not very likely/not likely at all/I have no idea
Q8.	Is there anything else you would like to share about your experiences of this unit?	Yes/no	If yes, please write here.
Q9.	Did you take maths or statistics A level?	Yes/no/another subject with quantitative elements (please specify)	If yes, please give details.
Q10.	Have you taken any units so far in your degree with a statistical element?	Yes/no	If yes, please give details.
Q11.	What course are you currently enrolled in?	Psychology/childhood studies/sociology/geography/social policy/economics/other (please specify)

Responses were anonymous, and 70 pre-unit questionnaires and 75 end-of-unit questionnaires were completed. Because both questionnaires were anonymous (to encourage open and honest responses) pre-unit and post-unit questionnaires could not be linked.

The students were also asked to complete learning diaries; time was allocated for this at the end of each lecture. Learning diaries were an integral part of the unit and students were asked to submit them at the end of the unit. However, they were given the option to withdraw them from the research study. In the first lecture, students were shown an example of a completed diary entry (for a lecture on parent–child relationships in social psychology) and given the opportunity to ask questions. Students took the learning diaries away after each session, and could add to them during the weeks between the lectures. Fifty-eight learning diaries were completed, and no students withdrew their diaries from the research project.

Focus groups were carried out with two groups of students at three different time periods: within two weeks of the start of the unit; about halfway through the unit; and at the end of the unit. Participating students had indicated in the pre-unit questionnaire that they were willing to participate in focus groups about how they were finding the unit. One group comprised students who had indicated in the pre-unit questionnaire that they were confident at manipulating numbers to tell a story, the other comprised students who had indicated that they were less confident. Ten students were invited to each focus group. Numbers attending each group varied from one (less confident, final time point) to nine (more confident, second time point). Reasons for this variation in numbers are not clear but it may be that the less confident students had disengaged from the course by this point, or perhaps did not want to discuss their lack of confidence.

Analysis strategy

Data used for analysis included responses to open questions on the student questionnaires, learning diaries and focus group transcripts. Analysis was carried out collaboratively by the first and second authors. A first round of reading enabled the first and second authors to familiarise themselves with the dataset. This was followed by a second reading by both authors during which initial codes were developed in response to the content of the data, through discussion. These codes were then refined and re-defined during further discussion and a second round of reading.

Ethics

This project was run in accordance with the authors’ institutional ethical procedure. The purpose of the project was explained to the students at the start of the unit. It was made clear that they had the right to withdraw their questionnaire data and learning diaries from the research project with no impact on their unit assessment. The first author, who collected the data, was not part of the teaching team for the unit and had no input into student assessment.

Findings

Students were asked in the pre-questionnaires and first focus groups to explain why they had chosen to take this open unit. This revealed a range of reasons for students signing up. These were categorised as generic interest, support, career, confidence and skills development with some overlap between these groups. Students who expressed higher levels of confidence at the start of the unit tended to explicitly state their interest in learning more about statistics and their application:

I'm interested in stats work and also more interested in human geography/social sciences so it sounded very good…I would like to be shown better ways to use figures to give facts and also become more aware of stats being used to bias opinions. (Geography, pre-questionnaire.)

Conversely, those with lower confidence and experience were seeking confidence building and support in basic statistics:

Why have you chosen to take this unit?

Very new to what I am used to – gain greater understanding of how numbers are used to present information.

What are your expectations of the unit?

Understand the importance of numbers. How understanding them can give a greater insight into what is being present. Become more confident in using them myself. (Geography, pre-questionnaire.)

The range in motivation for choosing the unit had implications for their expectations. These included being able to do the following:

• Interpret quantitative analysis to understand others’ work and be critical of it.

• Apply such analysis to help create their own arguments.

• Carry out the analysis itself through using the relevant computer packages.

There were also many who stated that they just wanted to learn more about statistics, without clarifying a particular aspect, indicating that they had broad and underdeveloped expectations of the unit. This already suggests the differences in students regarding their confidence levels and expectations of the unit with some wanting confidence building and support at basic levels, and others wanting to be pushed onto higher levels of statistical analysis.

As with the range in expectation, there was a variety in student attitudes towards the unit, as shown in the extracts from three student learning diaries given below.

These three learning diary examples suggest both very different experiences of the course and different trajectories regarding their confidence levels. The student from Table 4 demonstrates low levels of confidence in statistics and clearly struggles with some of the concepts introduced during the unit. Other students in their learning diaries and in the second focus group, particularly among those from sociology, social policy and childhood studies, shared similar assessment of the unit. Some stated that they were intimidated by the lectures with more statistical content (for example week five) and found it hard to keep up.

I am not enjoying it at all, like not at all… I feel like I’ve just found the last few really boring. I feel like it’s not true that we don’t have to know anything about statistics or be comfortable with them. I just think that was like one of the main reasons I took the unit… I just get lost… I kind of tune out … it’s quick and there’s a lot of new vocabulary that I, concepts that I’ve never heard of. (Social policy, focus group 2.)

Table 4 Childhood studies, learning diary (our italicised highlighting).

Week 1	How did you find the lecture today?	The lecture was surprisingly enjoyable. I managed to understand most of, if not all, of the numerical theory.
Do you have any worries or concerns?	I worry about my mathematical failings.
How did you feel about quantitative methods before the lecture and how do you feel now?	Very nervous and uninformed. Now more informed and slightly hopeful/less nervous.
Week 5	How did you find the lecture today?	I found this week harder than previous weeks as the formula was confusing. I enjoyed it less and difficult.
Do you have any worries or concerns?	I’m worried I don’t understand and find it boring.
How did you feel about quantitative methods before the lecture and how do you feel now?	More confused.
Week 8	How did you find the lecture today?	I found this lecture easier to understand.
Do you have any worries or concerns?	Feel tired.
How did you feel about quantitative methods before the lecture and how do you feel now?	Bit better.

Some of these students wrote in their learning diaries that they planned to catch up after the lecture because they did not feel they had understood the key concepts. Others simply shared that they found the lectures boring showing low levels of engagement in the lecture content.

Table 5 reflects the students who had positive experiences of the unit and demonstrate increased confidence and interest in statistics. This example suggests that as understanding increases so too does confidence – the opposite effect of the lack of understanding leading to disengagement seen in Table 4.

I think it has increased [my confidence] because before I just thought, “I can’t do anything with stats because I’m not mathsy enough, I don’t understand it,” whereas now I’m just like, “Well I do understand and I can build on it,” so I’m not as afraid of numbers as I was. So that’s kind of opened up more for me personally. (Geography, focus group 3.)

Table 5 Sociology, learning diary (our italicised highlighting).

Week 1	How did you find the lecture today?	The lecture as an introductory session was interesting…looking forward to approaching the topics raised in greater detail.
Do you have any worries or concerns?	None yet.
How did you feel about quantitative methods before the lecture and how do you feel now?	Intrigued, interested, engaging.
Week 5	How did you find the lecture today?	Really thought provoking – backing up ‘stories’ with numbers.
Do you have any worries or concerns?	Not especially but do not find numbers/figures especially easy!
How did you feel about quantitative methods before the lecture and how do you feel now?	Ok.
Week 9	How did you find the lecture today?	Interesting and insightful. Good explanations and insightful slides/presentation.
Do you have any worries or concerns?	No.
How did you feel about quantitative methods before the lecture and how do you feel now?	I feel quite confident and that my understanding has grown and been tested in a positive way.

This was predominantly seen among geography and psychology students who were undertaking core units in research methods alongside this open unit. They felt that this was a useful complementary unit for those who had felt intimidated by statistics but were beginning to find them more accessible.

Table 6 reflects the students who felt that the unit was not advanced enough and wanted more application and statistical analysis. These were often the same students who stated interest in statistics and high levels of confidence in mathematics, with many having done it for A level. The final comments from the student in Table 6 suggested that their expectations for the unit had not been met because they found the unit too basic and their recommendation for improving the unit was to ‘speed up the pace and learn more complex stats’. At the end of the unit, in both the focus groups and post-questionnaires, students were asked to reflect on the unit. Some students, predominantly from geography and psychology, wrote in their post-questionnaires that they “thought it would be more statistical and challenging” (geography) and felt it “just skimmed over topics that are samey, didn’t focus on statistical manipulation or specific methods” (psychology). It seems that these students were expecting a unit on statistical methods although this was not at all how the unit was promoted.

Table 6 Psychology, learning diary (our italicised highlighting).

Week 1	How did you find the lecture today?	Very engaging and interesting. Clearly set out learning topics for future lectures.
Do you have any worries or concerns?	No.
How did you feel about quantitative methods before the lecture and how do you feel now?	I felt confident in using them before, and now.
Week 5	How did you find the lecture today?	Informative, engaging and interactive but I learned nothing new.
Do you have any worries or concerns?	No.
How did you feel about quantitative methods before the lecture and how do you feel now?	I felt confident before and still do now.
Week 8	How did you find the lecture today?	Engaging, good slides and examples used but nothing new.
Do you have any worries or concerns?	No.
How did you feel about quantitative methods before the lecture and how do you feel now?	Confident before and now.

Among students who felt disengaged during the unit, there was also the sense that their expectations had not been fully met. They tended to feel that they had not been able to follow the level of statistics presented:

I don’t feel I have gained any additional insight into statistical analysis. It still goes well over my head…. It’s frustrating to genuinely not feel like you’re making any improvement… I have attended most of the lectures, but couldn't even really follow what was going on, and this hasn't changed. (Sociology, post-questionnaire.)

Some of these students also showed concern that this may impact on their future studies:

I think that anyone who hadn’t done any of those things before would find it quite difficult, just because of the level, the amount of new information…And I am a little concerned now, I don’t know about social research, is it a part of a compulsory unit for me? And now I am like oh God is it going to be about statistics? I hope not! (Social policy, focus group 3.)

There were also many students who were positive about the unit on reflection feeling that it had met their expectations. These were those who had worried that it would be heavy on statistical content but found that it was actually making statistics, both in this unit and elsewhere in their course, more accessible. This was especially, but not exclusively, seen among the geography and psychology students who had felt less confident in the statistics they were learning in other compulsory methods units.

Discussion

Through the analysis of learning diaries, questionnaires, and focus groups presented in the previous section, we suggest that there were different types of students on the course. These can be broadly categorised into the following typology:

Type 1 – Quantitative Junkies: these students were those who were overqualified for the unit and often shared feelings of being underwhelmed with the content of the unit. They tended to study geography and psychology. They are students who already understand the benefits of quantitative methods and are seeking more training, particularly in the process of doing statistical analysis and more advanced understanding of quantitative methods. They are also students who are receiving substantial levels of quantitative teaching as core teaching within their disciplines. These are the students for whom the unit was not advanced enough and who were often driven by high levels of confidence in maths and statistics, having studied these subjects at A level.

Type 2 – Quantitative Avoiders: these students tended to be in the Faculty of Social Sciences (e.g. sociology, social policy and childhood studies). They started out with low confidence and some interest in quantitative methods, often having not studied any subject with maths content since GCSE. During the course of the unit, they became turned off quantitative methods and, most disappointingly, often showed lower levels of confidence at the end of the unit than the start. They shared that they frequently found the lectures too hard, leaving them overwhelmed and increasingly disengaged. They were particularly intimidated by more complex lectures where statistical analysis was introduced and found it hard to keep up; some of these wrote in their learning diaries about catching up after class (follow-up exercises were provided).

Type 3 – Quantitative Converts: these were the students for whom the unit can be seen to be hitting the mark. At the end of the unit, they were positive citing either that they felt that it matched their expectations or that it made quantitative methods more accessible than they had been anticipating. These students tended to be either the less confident ones doing statistics elsewhere in their course, such as in geography methods, or some of the students doing courses in the Faculty of Social Sciences.

Since the quantitative converts tended to express expectation in engaging in more quantitative methods at the end of the unit, they could be seen as the main or, at least, the easiest target group for the university and the ESRC as they aim to make the field more attractive. There are clear lessons here for the merits of the unit for enabling such conversion in confidence and perception. This group of students has shown that some who begin their degree with low confidence and engagement in quantitative methods can be converted. Given time and suitable follow-on teaching, it may even be possible to shift converts to junkies. It is important, however, to consider how the avoiders can be changed to converts: at the start of the course these two groups were superficially similar but in some way the course resulted in different outcomes for these types of student.

Perhaps this is inevitable; perhaps it is impossible to design a course that engages all students equally. Still it remains our view that a truly successful attempt to integrate quantitative methods more centrally in, or throughout the whole social science curriculum, will be able to engage all students – even those who have been classified as avoiders in this paper. While the typology was not arranged solely according to discipline, some disciplines had more junkies than others (for example, geographical sciences and psychology, which have a strong quantitative leaning), and others had more avoiders (for example, childhood studies). Would it therefore be appropriate to locate quantitative methods securely within disciplines, using a very basic approach to quantitative methods in some cases, aiming different courses at junkies and avoiders? This would entail a trade-off between, on the one hand, reinforcing traditional disciplinary boundaries that would go against one of the primary aims of this unit, and on the other hand allowing avoiders to be more directly supported in developing their confidence. However, peer tutoring among university students has been found to be successful across a range of disciplines and topics (e.g. Arco-Tirado et al. 2011, Dancer et al. 2014, De Backer et al. 2012). Therefore integrating some kind of peer support, with junkies or converts supporting the avoiders, could be an alternative to improving avoider confidence. This could also serve to break down the traditional divisions between disciplinary boundaries.

Attempting to teach across disciplines, which brings with it differing levels of confidence, expectation and subsequent engagement, is not only challenging but also may not be conducive to the aim of making quantitative methods more appealing across undergraduate students in the social sciences. More consideration is needed for the types of teaching practice and content that will appeal to these different groups (Garfield & Ben-Zvi 2007, Jackson & Johnson 2013, Keebaugh et al. 2009). For example, to enable the engagement of quantitative avoiders, smaller groups allowing for support and participation may be beneficial. The development of the typology has also raised a number of key questions and areas of potential future research. How do individuals shift between groups, and what role do module content and teaching practices play in these shifts? What is the critical turning point for becoming converts or avoiders? What is the influence of the attitudes of peers and peer groups on a student’s own learning? What can we learn from our findings for the engagement of social science undergraduates in quantitative methods more widely? And, what may be further challenges in teaching quantitative methods in an embedded and cross-disciplinary way?

Conclusions

We would suggest that the unit has been successful in engaging a specific group of students who, although crossing disciplinary boundaries, shared typological characteristics. However, it has also seemingly failed at successfully meeting the needs of quantitative junkies and avoiders. This typology supports Ramos & Carvalho (2011) who identified a similar range of student engagement with quantitative methods. To extend this previous research, we suggest that the identification of these typologies highlights the pedagogical challenges of engaging students of such varied expectation, confidence and discipline within one student group.

It is important that we learn from our experiences, and consider how we can engage a wider range of students. In an entirely large-group format, the approaches and attitudes of different disciplines to using quantitative methods are not always easy to challenge or engage with. It is also the case that the course instructors, when given two hours to teach, tended to fall back towards filling it with lecturing. This is not a good way of getting the students’ interest. Gelman & Nolan (2002) offer various ideas for more active student participation. For the most recent academic year the course has been revised to incorporate regular small-group seminars and guided reading with the students split into disciplinary groupings. As there is quite a strong overlap between the disciplinary backgrounds of the students and which of the types they fall into, we hope this will help to encourage and engage students. Placing students within disciplinary groups for some parts of the course may enable us to begin to address and work with disciplinary cultures that have a historical lack of engagement in quantitative methods.

Going forward, the unit forms a core part of the University of Bristol’s new degree programmes, funded under the Nuffield Federation/ESRC/HEFCE Q-Step initiative (see www.bris.ac.uk/qstep) which offer enhanced quantitative training for students in the social sciences. Initial discussions suggest that staff across undergraduate programmes are supportive of the unit and its role within the wider initiative, although we are yet to explore this rigorously. Further exploration of staff attitudes may be needed as the programmes are implemented. These programmes also place a strong emphasis on cross-disciplinary teaching offering a rich resource for further evaluation of the pedagogic advantages or otherwise of bringing students from different disciplines together to learn quantitative methods within a broader framework of quantitative social science.

Acknowledgements

The unit and this research were both funded under ESRC grant ES/J011681/1. We are extremely grateful to all the students and staff who have and continue to contribute to the course, and to the referees of our initial paper for their useful observations and comments. Opinions expressed and any errors of interpretation are our own.