A scientometric analysis of multicriteria decision-making research

ABSTRACT

Multi-criteria decision-making (MCDM) is an important decision-making technique for incorporation into Decision Support Systems (DSS). This paper uses bibliographic techniques to characterise the MCDM research field. This research identifies the historical origin of MCDM in both Operations Research/Management Science (OR/MS) and Computer Science. This research identifies the increasing importance of environmental and medical applications in addition to traditional engineering and business applications. It also examines the core literature and the extent to which newer papers are cited.

KEYWORDS:

• Multicriteria decision making
• MCDM
• DSS
• scientiometrics

Introduction

Multi-criteria decision-making (MCDM) is a decision-making approach that involves considering multiple criteria when evaluating and choosing among alternative courses of action. Decision Support Systems (DSS) are concerned with supporting decision-makers in making decisions and are intrinsically concerned with choosing between alternatives, therefore MCDM methods have always been important in DSS. Researchers have noted the relevance of MCDM to DSS from the earliest days, in both individual DSS (Eom, 1998b; Eom & Min, 1999) and group DSS (Pervan, 1998). The DSS field has evolved over the years (Keenan, 2016), in particular, there has been a growth in new application fields such as environmental science and medicine that were less important in the early years of DSS, when business and engineering applications predominated. These new areas of application make substantial use of MCDM techniques and so MCDM continues to feature as an important reference discipline in recent DSS research (Keenan, 2021) as it did in earlier studies (Eom, 1998a).

DSS concerns the provision of support for specific decisions, using methods rooted in specific disciplines combined with decision-making methods and visualisation techniques that can span disciplines. However, there is a natural tendency for DSS builders to be most familiar with their own discipline, a consequence of this is that the DSS field is somewhat fragmented, and this can mean that useful innovation does not spread between different disciplinary areas. This research-in-progress paper uses scientometric techniques to look at the use of MCDM within different disciplines, to examine the extent of this fragmentation and to examine if new techniques become widely used.

MCDM research

Bibliographic databases such as Scopus or Web of Science (WOS) (Pranckutė, 2021) provide valuable information on academic disciplines as they record both the articles published and the articles cited (Keenan, 2021). In this study, we used the Scopus database. Scopus indexes scientific journals, allows the download of the abstracts and cited articles for aggregate sets of papers and there are bibliographic tools to analyse this data. Scopus indexes more publications than WOS (Thelwall & Sud, 2022), and allows the download of aggregate information. In 2023, Scopus had over 90 million records drawn from 27,950 active peer-reviewed journals.

Bibliographic approaches have been used to examine MCDM publications, although some of these are within specific sub-domains. Yu et al. (2018) examined MCDM using WOS data, Basílio et al. (2022) examined both Scopus and WOS data using a search that included a variety of keywords based on different methods. In this paper, we wanted to search for the general concept without specifying the detailed methods. We used a search on the titles, abstracts, and keywords in Scopus for the search terms ‘multi-criteria* decision’ OR ‘multi criter* decision’ OR ‘multicriter* decision’) We have only included journal articles, as the Scopus indexing of book chapters and conference proceedings is less consistent than that of journals. This search produced 23,135 citing records. We downloaded this data set and further processed it using bibliographic software, this processing identified that 5% of the records had some formatting issues and we excluded them. Figure 1 shows the growth in papers over the years.

Figure 1. number of MCDM papers in Scopus from 1974–2023.

The visualisations below use Leydesdorff’s approach to group journals in clusters. The number of citations and citing journals in each cluster in part reflects the different approaches to citation in different disciplines; these differing citation practices are an issue for research using these databases. This visualisation uses the Vosviewer software (van Eck & Waltman, 2010, 2017) using a disciplinary layout reflecting the analysis of all WOS publications by Leydesdorff et al. (2016). The overall layout of the visualisation reflects citation links across the entire Scopus database in 2012 (Leydesdorff et al., 2016). Note that some disciplines are represented in grey in both visualisations, this reflects disciplines in Scopus which do not have any publications with the search keywords, for example the social science disciplines on the bottom left of the visualisations do not use MCDM.

Figure 2 shows a visualisation of journal articles with the search keywords for the first 40 years of MCDM research. The cluster on the top left includes traditional modelling journals in mathematical and Operations Research/Management Science journals. The middle left grouping includes some engineering-related application areas.

Figure 2. Visualisation of journals with publications in MCDM from 1974–2013.

Figure 3 shows the publications in the last decade, this shows an increase in publications in environmental journals (appropriately coloured green) and a marked increase in the number of publications in the medical area, on the bottom right of the visualisation. This growth in environmental and medical applications is also found in DSS generally (Keenan, 2016).

Figure 3. Visualisation of journals with publications in MCDM from 2014–2023.

The above visualisations use a classification scheme by Leydesdorff et al. (2016). Scopus has its own groupings of disciplines, namely the All-Science Journal Classification Codes (ASJC) schema. Scopus identifies four major areas: Life Sciences, Social Sciences, Physical Sciences, and Health Sciences. There are some limitations to the use of these groupings as a DSS publication can be in two top-level groupings, typically combining a computer science classification with an application area, for instance, the journal Decision Support Systems is in both the Social Sciences and Physical Sciences. However, these groupings still provide useful insight into the disciplinary composition of DSS papers. The Social Sciences category contains journals such as the European Journal of Operational Research or the Journal of the Operational Research Society. These journals were important in the earlier years of MCDM. In the 20th century, there were 8 times as many MCDM publications in the Social Science grouping as the Life and Health Sciences together, while in the most recent 5 year period from 2019–2023, there are only twice as many.

The chart in Figure 4 shows MCDM papers in three time periods. In all periods just over half of papers are classified in the Physical Sciences grouping, in part this reflects the overlapping definition of publications. Papers in the Social Science category have declined from 39% in the earliest period to 25% in the most recent decade. The proportion of Life Sciences papers has almost doubled, and the proportion of Health papers has increased from 3.6% of MCDM papers in the earliest period to 5.7% in the most recent decade.

Figure 4. Disciplinary composition of MCDM publications.

Scientometric analysis

In this research, we used the CRExplorer software (Thor et al., 2016, 2018) to clean the data by removing references without dates and to identify similar references. This software also allows the merging of citations where there are small differences in the reference. For instance, where the number of initials of an author is different or there is a slightly different abbreviation for a journal. We excluded publications without a year and excluded references before 1950, which often are an error. This provided an initial number of 947,061 references from 23,490 citing publications in the period 1974 to 2023. Most of these citing publications 18,057, were in the period from 2014 to 2023.

The basic citation network is a mathematical graph where each node represents a citing document, and each directed link represents a citation from the current document to an earlier cited document. A simple cited reference search on WOS or Scopus will show all publications which cite an earlier publication. Bibliographic coupling groups newer publications which cite the same earlier publication so can be used to group publications with the same influences. Co-citation coupling (Small, 1973) looks at all of the references cited by a document and forms an undirected co-citation network linking documents which are cited together. Co-citation coupling looks backwards from a document to the citations it contains, which presumably represent the academic influences on the article. Co-citation coupled networks can be derived from individual publications, authors, or journals.

While the earlier figures represent the journals in which MCDM papers were published, the co-citation network for journals indicates the journals and books that those papers cited. Figure 5 represents citations from all MCDM journal papers from 1974 to 2023 and shows the 48 most cited sources. These are grouped into 3 major clusters: on the top left are computer science journals, the bottom left are mostly OR/MS journals and on the right-hand side are journals related to environmental and energy applications. Table 1 shows the most cited journals, and these play a central part in the network; the European Journal of Operational Research (EJOR), Expert Systems and Applications, and the Journal of Cleaner Production. Journals such as Information Sciences, Energy or International Journal of Production Economics have a substantial number of citations and play a central role within the clusters. The set of cited journals reflects the movement towards environmental and energy-related applications. Note that although MCDM techniques are often used in DSS, the journal Decision Support Systems does not have an important role.

Figure 5. Co-citation network for journals 1974–2023.

Table 1. Most commonly cited journals by MCDM journal papers 1974–2023.

Journal	Citations
European Journal of Operations Research	19572
Expert Systems and Applications	19313
Journal of Cleaner Production	17039
Sustainability 9	8433
Fuzzy Sets and Systems	8212
Information Sciences	7527
Energy	6458
Renewable and Sustainable Energy Reviews	6100
International Journal of Production Economics	5691
Computers and Industrial Engineering	5445

The co-citation network of individual papers shown in Figure 6 has three clusters. On the left side, the dominant approach is based on Saaty (1980). The seminal paper by Zadeh (1965) is in a relatively central position in the network. The cluster at the top has a widely cited paper on fuzzy logic by Atanassov and Stoeva (1986). The right-hand cluster is mostly computer science journals and typically contains newer references than the left-hand cluster.

Figure 6. Co-citation network of papers for MCDM journal articles 1974–2023.

In addition to greater diversity of application, there is an increasing number of papers published outside North America, Europe, and Australia/New Zealand, before 2014 60% of papers originated from these regions, but this fell to 42% in the most recent decade. Using the most recent version of the Vosviewer software it is possible to produce a bibliographic network of countries. This indicates that research in the US plays a less important role in MCDM research than in many academic disciplines, with Asian countries more prominent in this field.

MCDM within new domains

It is useful to examine the citation networks within the four Scopus groupings illustrated in Figure 4. As the vast majority of MCDM papers are in the Physical Sciences category, some insight can be gained by examining the other categories. In the Life Sciences category, the main citing journals are Water (Switzerland), Environmental Earth Sciences and Ecological Indicators. The Life Sciences co-citation network shown in Figure 7 has three clusters. The bottom left cluster is dominated by citations to papers by Saaty (1980), with work in the GIS field by Malczewski (1999, 2006) also important in the network.

Figure 7. Co-citation network of MCDM life sciences 2014–2023.

The Health Sciences co-citation network shown in Figure 8 also has a large cluster based around the work of Saaty. Another significant cluster cites the work of Belton and Stewart (2002) and the work of ISPOR – The Professional Society for Health Economics and Outcomes Research (Marsh et al., 2016; Thokala et al., 2016). A smaller cluster on the right is fluenced by the earlier work of Hwang et al. (1981) and the later work of Rezaei (2015). Further investigation is required to identify why these groups are not closely related.

Figure 8. Co-citation network of MCDM health sciences 2014–2023.

Conclusion

The DSS field now includes an increasing diversity of disciplines, who have very different decisions to make, but who share the use of general decision-making and technologies to help make those decisions. Given that the fields are quite separate in the content of their decisions and so draw from different disciplinary literature, there is a concern that their use of decision-making methods and systems may also be fragmented, and that best practice is not disseminated equally throughout different fields. MCDM is a widely used technique in DSS and is used in diverse disciplines. Consequently, MCDM papers represent a suitable subset of the DSS literature to investigate this. There is some commonality in the methods across different disciplines, but it is largely evidence of the continuing influence of AHP, an early 1980s technique. The AHP approach has been subsequently criticised, for instance in the work of Barzilai and Golany (1994) or Brugha (2000), but that criticism is not evident in the bibliographic networks produced here, which suggests that applications are rooted in older methods. This may reflect the availability of AHP software or its use in education or it may reflect the new applications are based on earlier work in that field. This research-in-progress paper identifies in outline the issue of uneven dissemination of best practice through different DSS sub-fields. DSS research has traditionally been largely focussed on business and engineering-related applications but should provide insight to all decision-makers, including newer areas of application. Further work in this area hopes to further interrogate the citation patterns to further understand the latter issue by using bibliographic network approaches to identify the connectivity or lack of it between different subfields and researchers in different geographic regions.

Disclosure statement

No potential conflict of interest was reported by the author(s).