830
Views
1
CrossRef citations to date
0
Altmetric
COGNITIVE & EXPERIMENTAL PSYCHOLOGY

The network structure of dysfunctional metacognitions, CAS strategies, and symptoms

ORCID Icon, &
Article: 2205258 | Received 10 Sep 2022, Accepted 17 Apr 2023, Published online: 25 Apr 2023

Abstract

In the metacognitive model of psychological disorders, metacognitive strategies and corresponding underlying metacognitive beliefs intensify and maintain emotional distress symptoms. In the current study, our three objectives were to evaluate and replicate the network structure of dysfunctional metacognitions as assessed with the MCQ-30, to examine its stability when adding relevant covariates in the form of metacognitive strategies (worry and rumination) and symptoms (anxiety and depression), and to evaluate how different sets of dysfunctional metacognitions are more or less strongly linked differently to metacognitive strategies and symptoms. A cross-sectional university sample with a mean age of 26 years (N = 440; Males = 156, Females = 283) completed the Metacognitions Questionnaire–30, Penn State Worry Questionnaire, Ruminative Response Scale, and Hopkins Symptom Checklist. Data were analysed using psychological network analysis in R-studio statistical software. The network structure of dysfunctional metacognitions replicated well with item clusters that correspond to clinically meaningful substructures in the metacognitive model. Negative metacognitive beliefs and beliefs about uncontrollability might have more functional significance in the mutual connections between dysfunctional meta-domains as well as the connections with metacognitive strategies and symptoms. For worry and anxiety, negative beliefs about uncontrollability and corresponding danger of worry were more prominently connected in the network structure. For rumination, cognitive self-consciousness was more prominent, whereas for depression, need for control was more prominently connected. Support was found for mutual interdependence between different sets of dysfunctional metacognitive beliefs, that metacognitive beliefs are linked to but separate from metacognitive strategies, and that these may function together in affecting emotional distress symptoms

1. Introduction

The Self-regulatory Executive Function (S-REF) model (Wells & Matthews, Citation1994, Citation1996) postulates that psychological disorders are maintained by a maladaptive thinking style called the Cognitive Attentional Syndrome (CAS). The CAS consists of worry/rumination, inflexible self-attention/threat monitoring, and maladaptive coping behaviours and is directed and influenced by dysfunctional metacognitions (i.e. beliefs about cognition) (Wells, Citation2009). Thus, dysfunctional metacognitive beliefs are regarded as an underlying cause of psychological disorders (Wells, Citation2009, Citation2019). Dysfunctional metacognitive beliefs in psychological disorders have been assessed by using the Metacognitions Questionnaire 30 (MCQ–30; Wells & Cartwright-Hatton, Citation2004) adapted from the original MCQ–65 (Cartwright-Hatton & Wells, Citation1997). The questionnaire consists of five distinct domains of metacognitive knowledge on separate subscales, namely, (i) Positive beliefs about worry (POS) – which assesses the extent to which a person believes that worrying is useful (e.g.’ Worrying helps me to avoid problems in the future’), (ii) Negative beliefs about the uncontrollability and corresponding danger of worry (NEG) – which assesses the extent to which a person thinks that worrying is uncontrollable and dangerous (e.g. ‘When I start worrying, I cannot stop’), (iii) Cognitive confidence (CC) – which assesses confidence in memory (e.g. ;’I have a poor memory’), (iv) Need for control (NC) – which assesses the extent to which a person believes that certain types of thoughts need to be controlled (e.g. ‘It is bad to think certain thoughts’), and (v) Cognitive self-consciousness (CSC) – which measures the tendency to monitor one’s own thoughts (e.g. ‘I constantly examine my thoughts’).

Several past studies have investigated the contribution of different sets of metacognitive beliefs in trait worry and obsessive-compulsive symptoms (Wells & Papageorgiou, Citation1998), hypochondriasis (Bouman & Meijer, Citation1999), predisposition to auditory hallucinations (Morrison et al., Citation2000), depression (Papageorgiou & Wells, Citation2003), post-traumatic stress disorder (Roussis & Wells, Citation2006), state anxiety (Spada et al., Citation2008), alcohol use and problem drinking (Spada et al., Citation2009), trait-anxiety (Nordahl, Hjemdal, et al., Citation2019), gambling disorder (Rogier et al., Citation2021), and common symptoms of psychopathology (Nordahl, Ødegaard et al., Citation2019). Overall, results have supported positive relations between metacognitive beliefs and psychopathology while articulating that metacognitive therapy (MCT; Wells, Citation2009) could show favourable treatment outcomes by effectively producing metacognitive change. A view supported in two meta-analytic reviews that examined the efficacy of MCT for anxiety and depression (Normann et al., Citation2014) and various disorders including depression, generalized anxiety, post-traumatic stress, comorbid, and mixed emotional problems (Normann & Morina, Citation2018). Additional evidence from a recent meta-analysis found differences in the prevalence and importance of different sets of dysfunctional metacognitions for various disorders (Sun et al., Citation2020). Results indicated that specific sets of dysfunctional metacognitions were more prominent in some disorders than others. Negative beliefs about uncontrollability and danger of thought were more prevalent in generalized anxiety disorder, whereas in obsessive-compulsive disorders, cognitive self-consciousness was more prevalent. The authors concluded that different sets of dysfunctional metacognitions are more or less prevalent and important across different disorders.

The metacognitive model highlights the mutual interdependence between metacognitive beliefs that function together in psychological disorders (Wells, Citation2009). However, our search found only one study with the application of a statistical model (e.g. psychological network analyses) to the metacognitive theory to provide information about how metacognitive beliefs mutually influence each other. This is a new perspective that emerged a decade ago as an alternative to conceptualizing psychological constructs – psychological network analysis – in the frame of Complexity Science (see Braithwaite, Citation2022). This perspective proposes that symptoms reciprocally and dynamically reinforce each other, forming a causally connected network system, such as psychological disorder (Borsboom & Cramer, Citation2013; Borsboom, Citation2008) unlike the common factor model, which estimates psychological disorders as the consequence of an underlying latent factor that influences the observed symptoms. Network analysis can be used to examine the importance or centrality of different sets of dysfunctional metacognitive beliefs, their mutual interdependence, or associations, thus opening a way to empirically determine and provide specific information about how different metacognitive beliefs mutually interact, often reciprocally, and reinforce each other. With the network approach, it is also possible to examine how multiple constructs interact in a complex system of networks. For instance, it is possible to examine how different metacognitive beliefs influence the activation of other maladaptive domains such as a community of anxiety symptoms through the so-called bridge connections.

Nordahl et al. (Citation2022) examined the relations between variables that assess metacognitive beliefs and found that items of the MCQ–30 appeared to cluster in meaningful substructures corresponding to the metacognitive theory. Two nodes with the highest node strength, and therefore important targets to affect the MCQ–30 network structure were MCQ26 (‘I do not trust my memory’) and MCQ9 (‘My worrying thoughts persists no matter how I try to stop them’). This observation implies that, targeting or facilitating greater changes in MCQ26 and MCQ9 in treatment is likely to have the largest impact on the whole network (e.g. more global change of dysfunctional metacognitions). Most of the items in the network structure were positively connected, supporting the notion that items may co-occur or mutually influence each other. This is contrary to several assumptions in the Common Factor Analysis approach to latent variable modelling. Among them, (i) the assumption that the latent cause (e.g. disorder or disease) is distinct from its symptoms, (ii) the disorder causes the symptoms, and (iii) the local independence assumption, which means that symptoms reflecting the disorder are functionally independent when conditionalizing on the presence of the latent common cause. The network approach can therefore provide interesting nuances in the nature and structure of dysfunctional metacognitions and their mutual interdependence. Further results in the study (Nordahl et al., Citation2022) revealed that, ‘need for control’ showed the highest score across three commonly used centrality measures (strength, closeness, and betweenness), indicating that need for control is important in the network structure, and its activation has a strong influence on other sets of metacognitive beliefs. Targeting this central belief domain means changes in the network structure can be affected quickly, making it a potential target for effective metacognitive change.

As reviewed in previous sections, the S-REF model (Wells & Matthews, Citation1994, Citation1996) highlights dysfunctional metacognitions (i.e. beliefs about cognition) as directing or influencing maladaptive thinking (e.g. worry and rumination—CAS strategies), which is also involved in symptoms (e.g. anxiety, and depression). Thus, dysfunctional metacognitions are regarded as an underlying cause of psychological disorders (Wells, Citation2009, Citation2019). Until recently, no study applied psychological network analysis to explore the structure of dysfunctional metacognitions. It remains to be investigated (i) whether the network structure of dysfunctional metacognitions can be replicated, adding to the generalizability of the metacognitive model as well as (ii) to investigate its stability (i.e. whether the network structure will remain unchanged when adding relevant covariates) and finally (iii) how different sets of dysfunctional metacognitive beliefs are linked differently to CAS strategies and symptoms. This study sought to fill these gaps.

Our first goal was to replicate the network structure of the MCQ–30 in a university sample. Replications across different samples can contribute to generalizability of findings related to the metacognitive model. For our second goal, we examined whether the MCQ–30 network structure will remain unchanged when adding relevant covariates in the domains of CAS (worry and rumination) and symptoms (anxiety and depression) as well as gender and age. A stable and unchanged MCQ–30 network structure after controlling for the effect of CAS strategies and symptoms would be important in explicating, for example, that metacognitions are different from CAS strategies (cognition) and symptoms. Because of the known associations between metacognitive beliefs, age, and gender, it was necessary to include them as routine covariates. Wells and Cartwright-Hatton (Citation2004) found no gender differences, while others (e.g. O’Carroll & Fisher, Citation2013; Spada et al., Citation2008) reported differences in the MCQ–30 subscales. Younger age was associated with higher reports of the MCQ–30 (Grøtte et al., Citation2016; Spada et al., Citation2008). The inclusion of different and relevant covariates in the network structure of MCQ–30 is important to establish greater validity, robustness, and continued research in the nature and structure of dysfunctional metacognitions for theoretical and empirical harvest. Finally, for our third goal, we examined the connections between the five domains of metacognitive beliefs, the CAS (worry and rumination), and symptoms (anxiety and depression). In this way, we can empirically determine how different sets of metacognitive beliefs are more or less strongly connected with worry, rumination, and symptoms through the so-called bridge connections. Due to the exploratory nature of network analysis, we adopted an atheoretical approach and made no specific hypotheses.

2. Materials and methods

2.1. Participants and procedure

Students at the Norwegian University of Science and Technology were invited to take part in the study. Four hundred and forty students with a mean age of 26 years participated (N = 440; Males = 156, Females = 283). Participation was voluntary, and participants could withdraw their informed consent at any time without consequences. The research project was approved by the Norwegian Ethics committee—project ID 2016/339.

2.2. Measures

The Metacognitions Questionnaire–30 (MCQ–30; Wells & Cartwright-Hatton, Citation2004) assesses maladaptive metacognitive beliefs according to metacognitive theory. Each item is rated on a 4-point Likert-type scale. The MCQ-30 consists of five subscales. Higher scores indicate stronger endorsement of dysfunctional metacognitive beliefs. The Penn State Worry Questionnaire (PSWQ; Meyer et al., Citation1990) was used to assess worry. The brief PSWQ (Topper et al., Citation2014) has five items that assess the degree to which individuals typically perseverate about upcoming life events, rated on a 1 to 5 Likert-type scale. The brief PSWQ has been observed to have adequate psychometric properties in terms of validity and test—retest reliability (Topper et al., Citation2014). The Ruminative Response Scale (RRS; Nolen-Hoeksema & Morrow, Citation1991) includes a brooding subscale comprising five items to assess responses to depressed mood. Respondents rate each item on a 1 to 4 Likert-type scale. Higher scores indicate higher levels of rumination. The RRS has good psychometric properties (Luminet, Citation2004). The Hopkins Symptom Checklist short form (HSCL-10; Nguyen et al., Citation1983) and its two subscales were used to assess levels of anxiety and depression symptoms. Respondents rate each item on a 1 to 4 Likert-type scale. Higher scores indicate higher levels of symptoms.

2.3. Statistical analyses

Network analyses were performed in R version 4.1.3 (R Core Team, Citation2013). All analysis codes and Supplementary Files are available on the Open Science Framework platform at osf.io/s7aq2. To respond to the first goal, a correlation matrix of MCQ–30 items was computed and used as input for a Gaussian Graphical Model (GGM). The network was estimated using qgraph (Epskamp et al., Citation2012), glasso (Friedman et al., Citation2014) and bootnet (Epskamp & Fried, Citation2018) for checking network accuracy and stability. Network components in this study are items from the MCQ–30 questionnaire referred to as nodes. The connections between nodes are referred to as edges, representing partial correlations between two nodes, controlling for all other nodes. The first network structure contained the 30 items of the MCQ–30 questionnaire (30-item network). The one-step Expected Influence (EI) was calculated to determine which metacognitive belief was most central to the network, representing the relative importance of a node in the network (Robinaugh et al., Citation2016). The EI provides a more accurate estimate of node centrality compared to strength centrality, which only takes the absolute values of edges into account, while the EI accounts for both positive and negative values when the network structure contains positive and negative edges (McNally, Citation2016; Robinaugh et al., Citation2016).

To respond to the second goal, two network structures – second and third – were estimated as in a previous study (Armour et al., Citation2017). The second estimated network contained the addition of six covariates (i.e. age, gender, worry, rumination, anxiety, and depression; 36-item network). Then, all six covariates were removed from the adjacency matrix of the 36-item network, resulting in a modified network that contained the connections among 30 items, controlling for the six covariates. To estimate the third network structure, the adjacency matrix of the modified network (controlling for covariates) was subtracted from the adjacency matrix of the first network (not controlling for covariates). The resulting delta (third) network contained the difference between the first and second network structures. If the remaining edges in the delta network are few and weak, then the network structure of metacognitive beliefs (i.e. 30-item network) can be said to be stable and unchanged.

To respond to the third goal, we estimated a domain network of metacognitive beliefs and then computed a combined network comprising the five domains of MCQ–30, i.e. positive beliefs about worry – (POS), negative beliefs about the uncontrollability and corresponding danger of worry(NEG), cognitive confidence – (CC), need for control – (NC) and cognitive self-consciousness – (CSC), the CAS (worry and rumination), and symptoms (anxiety and depression). The bridge expected influence (BEI) was computed to determine how the different domains of metacognitive beliefs are connected to the CAS and symptoms.

3. Results

3.1. The network structure of MCQ–30

The network structure of the MCQ–30 is displayed in Figure (i.e. 30-item network structure). A couple of interesting features emerged from the way items clustered together. A visual inspection shows that the items of the MCQ–30 appeared to cluster in clinically meaningful substructures, which corresponds to the subcomponents of the MCQ–30, namely, (i) cognitive confidence—CC, (ii) positive beliefs about worry—POS, (iii) need for control—NC, (iv) cognitive self-consciousness—CSC, and (v) negative beliefs about the uncontrollability and corresponding danger of worry—NEG. However, MCQ12 appeared to be placed far away from its cluster (i.e. cognitive self-consciousness). Interestingly, the NEG cluster appeared to subdivide into two smaller clusters comprising MCQ9, MCQ21, and MCQ11 as one smaller cluster and MCQ2, MCQ4, and MCQ15 as another smaller cluster. Thus, the network suggests that the negative beliefs about the uncontrollability and corresponding danger of worry subcomponent might consist of two closely related but separate components.

Figure 1. A network of 30 items for the MCQ–30. Green edges (i.e. connections) represent positive associations and red edges represent negative association. The thicker the connection, the stronger the association between nodes.

Figure 1. A network of 30 items for the MCQ–30. Green edges (i.e. connections) represent positive associations and red edges represent negative association. The thicker the connection, the stronger the association between nodes.

The Expected Influence (EI) estimates for all items in the network structure are displayed in Figure . Among the first five highest EI items were MCQ26, MCQ21, MCQ30, MCQ9, and MCQ22. Among the lowest EI items were MCQ5, MCQ14, and MCQ12. This means that these are the first five highly connected (or ‘influential’) nodes in the network with the most influence and the least connected nodes with the least influence.

Figure 2. The expected influence estimates for each item in the network structure of MCQ–30.

Figure 2. The expected influence estimates for each item in the network structure of MCQ–30.

Results for the accuracy of the network structure and tests for significant differences between edges and node strengths are contained in the Supplementary Files (Fig. S1 to S3B).

3.2. The network structure of MCQ–30 with covariates

Figure displays the network structure containing the covariates. The edges between age and MCQ3 (−0.08), MCQ6 (−0.8), MCQ7 (−0.06), MCQ13 (−0.01), MCQ22 (−0.02), and MCQ24 (−0.04) were all negative, which means that when controlling for the other items, older participants scored lower on these MCQ items. Women scored lower on MCQ22 (−0.19), MCQ25 (−0.01), and MCQ27 (−0.087) but higher on MCQ7 (0.03), MCQ15 (0.06), MCQ21 (0.05), and MCQ24 (0.01). Women also scored higher on anxiety (0.05), rumination (0.12), and worry (0.19). Worry was positively connected with MCQ3 (0.02), MCQ4 (0.01), MCQ7 (0.08), MCQ9 (0.07), MCQ10 (0.01), MCQ11 (0.03), MCQ15 (0.08), and MCQ21 (0.10). Worry was also positively connected with rumination (0.15) and anxiety (0.20). Rumination was positively connected with MCQ3 (0.12), MCQ9 (0.09), MCQ15 (0.06), MCQ20 (0.05), MCQ7 (0.03), MCQ22 (0.01), and MCQ30 (0.04). Rumination and depression were positively connected (0.21). Anxiety was positively connected with MCQ2 (0.01), MCQ4 (0.06), MCQ9 (0.08), and MCQ15 (0.05). Anxiety and depression were positively connected (0.39). Depression was positively connected with MCQ4 (0.06), MCQ9 (0.06), MCQ11 (0.01), MCQ17 (0.01), MCQ20 (0.09), MCQ25 (0.03), and MCQ30 (0.07).

Figure 3. (a) The 36-item network structure containing the MCQ–30 and covariates. (b) The delta network for the MCQ–30.

Figure 3. (a) The 36-item network structure containing the MCQ–30 and covariates. (b) The delta network for the MCQ–30.

3.3. The impact of covariates on the network structure of MCQ–30

Figure displays the delta network, which estimates the difference between two network structures (i.e. one without controlling for covariates minus one controlling for covariates). The remaining connections in the delta network structure are few and weak (strongest connection was 0.04). The sum of connections for the network without controlling for the covariates was 14.59, whereas the sum of connections for the network which controls for the covariates was 13.21, with the covariates accounting for only 9.0% of variance when included in the network. The correlation between the two network structures was very high at 0.93, supporting the claim that the network structure of the MCQ–30 did not change much.

3.4. Domain network structure of metacognitive beliefs

The domain network showed positive connections with especially strong connection between need for control and negative beliefs about the uncontrollability and corresponding danger of worry domains. The Expected Influence (EI) in descending order was as follows: need for control, negative beliefs about the uncontrollability and corresponding danger of worry, cognitive self-consciousness, cognitive confidence, and positive beliefs about worry, respectively. Need for control emerged as the most highly connected (or ‘influential’) domain of metacognitive beliefs, and positive beliefs about worry was the least influential domain in the mutual influence between the five domains of metacognitive beliefs. Figure S4A and S4B (in the Supplementary Files) displays the domain network structure of MCQ–30 and values of the EI, respectively. Results for the standardized estimates of betweenness, closeness and node strength centrality indices (Fig. S5), accuracy of network structure (Fig. S6A and B), and differences in edge weights (Fig. S7A) and node strengths (Fig. S7B) are also contained in the Supplementary Files.

3.5. Bridge connections in the combined network

The combined network structure is contained in Figure . Figure shows the Bridge Expected Influence (BEI) values of the domains of metacognitive beliefs with respect to the CAS and symptom communities. Overall, the domains of metacognitive beliefs showed much higher BEI values with respect to CAS than symptoms. Negative beliefs about the uncontrollability and corresponding danger of worry emerged as the domain that strongly connects with worry followed by anxiety. Cognitive self-consciousness showed the strongest domain connection to rumination, while need for control showed the strongest connection to depression. Positive beliefs about worry were the least connected domain, mostly to the symptoms. Connections/edge weights between all components in the combined network are contained in the Supplementary files.

Figure 4. Combined network structure of the domains of metacognitive beliefs, CAS (worry and rumination), and symptoms (anxiety and depression).

Figure 4. Combined network structure of the domains of metacognitive beliefs, CAS (worry and rumination), and symptoms (anxiety and depression).

Figure 5. Bridge expected influence estimates for the domains of metacognitive beliefs with respect worry, rumination, anxiety, and depression. BEI = Bridge expected influence.

Figure 5. Bridge expected influence estimates for the domains of metacognitive beliefs with respect worry, rumination, anxiety, and depression. BEI = Bridge expected influence.

4. Discussion

To the best of our knowledge, only one study (Nordahl et al., Citation2022) has so far investigated the network structure of dysfunctional metacognitions, while no previous study has examined the stability of the network structure when adding relevant covariates in the domains of CAS strategies and symptom measures. Our goals to (i) test the replicability of the network structure of dysfunctional metacognitions, (ii) examine whether the network structure will remain unchanged when adding relevant covariates, and finally (iii) examine the connections between the different domains of metacognitive beliefs, CAS strategies, and symptoms, therefore, seemed fully warranted, adding to the small body of the literature and empirical corpus of the internal structure of dysfunctional metacognitions. There were several similarities between our results and the only previous network analysis of dysfunctional metacognitions (Nordahl et al., Citation2022). Consistent with previous study, the network structure replicated in our sample with some interesting features that correspond to clinically meaningful clusters in the metacognitive model. The metacognitive model hypothesizes five domains of dysfunctional metacognitions to which our findings accord very well. Of note, two novel insights into the clustering of items in the network structure of dysfunctional metacognitions deserve attention.

4.1. The network structure of dysfunctional metacognitions

The first goal of this study sought to replicate the network structure of dysfunctional metacognitions. First, we found that the six items that assess negative beliefs about the uncontrollability and corresponding danger of worry subdivided into two substructures of three items, exactly replicating findings in the previous study (Nordahl et al., Citation2022). MCQ2 (‘My worrying is dangerous for me’), MCQ4 (‘I could make myself sick with worrying’), MCQ15 (‘My worrying could make me go mad’), on the one hand, and MCQ9 (‘My worrying thoughts persists, no matter how I try to stop them’), MCQ11 (‘I cannot ignore my worrying thoughts’), and MCQ21 (‘When I start worrying, I cannot stop’), on the other hand, were the two subdivisions. According to Nordahl et al. (Citation2022), these two subdivisions are separate but closely connected subcomponents in the metacognitive model that corresponds to beliefs about ‘uncontrollability of worrying’ as one subcomponent and beliefs about the ‘danger of worrying’ as another subcomponent that can feature in the metacognitive model. According to metacognitive theory (Wells, Citation2009), uncontrollability beliefs should always be present in psychological disorders as they are the primary influence that prohibits disengagement from the CAS. Danger beliefs are typically present but may also depend on uncontrollability beliefs (e.g. if worry is under your control, how can it be dangerous?) In other words, while these beliefs are tightly connected, they conceptually represent separate belief domains.

Second, MCQ12 (‘I monitor my thoughts’) appeared to be placed further away from its cluster (i.e. cognitive self-consciousness), while in the previous study, this item (i.e. MCQ12) shared strong positive edges with items belonging to need for control cluster, which suggested that beliefs about the need to control thoughts are in some ways related to monitoring thoughts in order to detect and apply control (Nordahl et al., Citation2022). In our study, MCQ12 appeared to share positive edges with both clusters and its placement in a single cluster is therefore unclear. Applications like the Exploratory Graph Analysis (EGA; Golino & Epskamp, Citation2017; Golino et al., Citation2020) – using package ‘EGAnet’ which estimates the number of dimensions using graphical lasso and a weighted network community detection algorithm may share further light in this regard.

Among the most influential items in our network were MCQ26 (‘I do not trust my memory’), MCQ21 (‘When I start worrying, I cannot stop’), MCQ30 (‘I constantly examine my thoughts’), MCQ9 (‘My worrying thoughts persists, no matter how I try to stop them’), and MCQ22 ('I will be punished for not controlling certain thoughts’). Interestingly, MCQ9, MCQ26, and MCQ30 also emerged as very strongly connected items in the previous study. The following items, MCQ1 (‘Worrying helps me to avoid problems in the future’), MCQ14 (‘My memory can mislead me at times’’), and MCQ24 (‘I have little confidence in my memory for places’) were among the least influential, both in the previous and current studies. Our findings thus echo findings from the prior study and could highlight the clinical importance of beliefs about MCQ9 (‘My worrying thoughts persists, no matter how I try to stop them’), MCQ26 (‘I do not trust my memory’), and MCQ30 (‘I constantly examine my thoughts’), which in this case, are potential targets for treatment planning to achieve the largest impact on the whole network structure (or a global and effective metacognitive change) while targeting MCQ1 (‘Worrying helps me to avoid problems in the future‘), MCQ14 (‘My memory can mislead me at times’), and MCQ24 (‘I have little confidence in my memory for places’) may not have as much global impact for effective metacognitive change. This should, however, be interpreted with some caution since our sample in this study are non-clinical participants.

Predictions in the network were relatively good with mean node predictability indicating 83% shared variance on average. Unsurprisingly, MCQ21 (‘When I start worrying, I cannot stop’) accounted for the largest shared variance (97.7%) when considering interconnectedness in the network structure, indicating how much it is influenced by all other items it shares connections. As noted by Nordahl et al. (Citation2022), the intended conceptual overlap within the MCQ–30 scale means that high amounts of shared variance are not surprising, but more importantly, we believe that this finding highlights beliefs about uncontrollability and persistence of worrying as highly connecting different dysfunctional metacognitions that give it a prominent role in psychological disorders. Thus, as is the case with metacognitive therapy (Wells, Citation2009), interventions should seek to address uncontrollability beliefs from the beginning with means to achieve the most impact on all dysfunctional metacognitions.

4.2. The network structure of dysfunctional metacognitions with covariates and conceptual relatedness between metacognition, CAS strategies, and symptoms

The second goal of this study examined whether the network structure will remain unchanged when adding relevant covariates (e.g. in the domains of cognition and symptoms). The results corroborate previous studies which have reported lower scores for older participants and higher scores for younger participants (Grøtte et al., Citation2016; Spada et al., Citation2008). In terms of gender, we found that women compared to men scored lower on items mostly belonging to the need for control but higher on items belonging to negative and positive metacognitive beliefs as well as on the levels of worry, rumination, and anxiety in support of other studies (Johnson & Whisman, Citation2013; Nolen-Hoeksema, Citation2012). According to the Response Style Theory (RST; Nolen-Hoeksema, Citation1987, Citation1991), women have a greater tendency to ruminate, which feeds into higher symptom levels (Nolen-Hoeksema, Citation2012). The question that arises is why do women have a greater tendency to ruminate? We found that women scored higher on items belonging to negative and positive metacognitive beliefs, which from a metacognitive perspective may explain the corresponding higher scores on worry, rumination, and anxiety since endorsing negative and positive meta-beliefs are important for CAS and symptoms. Worry, rumination, anxiety, and depression were positively interrelated, supporting ample evidence in the literature (e.g. Anyan et al., Citation2020a, Citation2020b).

The inclusion of covariates in the form of CAS strategies and symptoms in the network structure of dysfunctional metacognitions also meant that we could examine the conceptual relatedness or overlap between these variables and thus differentiate relations between metacognition, CAS strategies, and symptoms. If metacognitive beliefs are not conceptually different from CAS strategies or symptoms, we would observe greater overlap and variance explained in the network containing the covariates. However, not much variance was explained, and the original network structure of dysfunctional metacognitions was unchanged and highly comparable to the network structure controlling for covariates, supporting the assertion that metacognitive beliefs is related to but not the same as CAS strategies and symptoms and that these components may belong to different levels of the cognitive architecture (i.e. meta-level and cognitive level) (Wells, Citation2019).

4.3. Different sets of dysfunctional metacognitions are related differently to CAS strategies and symptoms

For the third goal of this study, we investigated how domains of dysfunctional metacognitions are linked differently to CAS strategies and symptoms. Domains of dysfunctional metacognitions showed various connections to both CAS strategies (worry and rumination) and symptoms (anxiety and depression), but the strongest connection emerged with the community of items measuring worry followed by anxiety, rumination, and depression, respectively. Specifically, negative beliefs about the uncontrollability and corresponding danger of worry was more robustly connected to both worry and anxiety as evidenced by its nodes’ higher BEI values in reference to those two communities. Cognitive self-consciousness showed the strongest connection to rumination, while need for control showed the strongest connection to depression. These observations are interesting as metacognitive models of worry and generalized anxiety (Wells, Citation1995) place uncontrollability beliefs as the most important belief domain for worry and anxiety, while meta-awareness and cognitive flexibility are formulated and targeted from the start of treatment in metacognitive therapy for depression (Wells, Citation2009). Patients with depression may rely on misdirected control through strategies because beliefs about uncontrollability reduce their efforts to exercise direct mental control over rumination. Cognitive self-consciousness may be specifically linked to perseverative rumination as it is an indicator of heightened self-attention and preoccupation with cognitive events. While these results can help us to understand how dysfunctional meta-domains relate to CAS strategies and symptoms, we cannot generalize our conclusions to mean that they significantly differentiate between symptoms or disorder domains as concluded in a recent meta-analysis (Sun et al., Citation2020) which included clinical samples unlike our study.

In the domain network, need for control showed the highest value across all centrality measures, with the strongest connection to negative beliefs about the uncontrollability and corresponding danger of worry. We observed only positive connections in the domain network structure and especially strong connections between need for control and negative beliefs about the uncontrollability and corresponding danger of worry, supporting the assertion that different sets of dysfunctional metacognitions may coexist and mutually reinforce each other across subcategories. Nordahl et al. (Citation2022) noted that high mutual influence between the domains could mean that if an individual endorses one cluster of dysfunctional metacognitions, there is a chance that the endorsed cluster might interact with neighbouring clusters. As such, an individual presenting with high need for control may also show high levels of negative beliefs about the uncontrollability and corresponding danger of worry. This finding also reemphasises the key role of beliefs about uncontrollability and the failure to disengage negative processing in the metacognitive model (Wells, Citation2019). Broad connections of negative beliefs about the uncontrollability and corresponding danger of worry to the CAS and symptom communities could further stress the ubiquity of uncontrollability and harmfulness of cognition having greater global impacts, not only in the mutual influences between domains of dysfunctional metacognitions but also in the initiation and maintenance of psychological disorders. According to S-REF, CAS involves a universal style of extended and repetitive processing of self-relevant information that is caused by either declarative or procedural metacognitive knowledge (See Wells, Citation1995, Citation2019 for more), with impacts on symptoms (Wells, Citation2019). Our explanation that need for control and negative beliefs about the uncontrollability and corresponding danger of worry highlight greater impact converges with notable propositions in the S-REF, which says that interference with effective control as results of beliefs about uncontrollability and harmfulness of cognition can lead to ‘omnipresent threat from an internal process of cognition itself’ and thus ‘of greater causal significance in [psychological] disorders’ (Wells, Citation1995, p. 3).

4.4. Clinical implications of the findings

The fact that need for control showed the highest index across strength, closeness, and betweenness in both previous and (also highest EI values) current studies may further highlight its clinical importance in effective metacognitive change. Targeting a central domain with high closeness (e.g. need for control) means that it can affect changes in the network quickly and can also be affected quickly by changes in any part of the network, making need for control a potential candidate in targeted interventions for effective metacognitive change. In terms of interventions and treatment planning, our findings align with the goals of Metacognitive therapy (MCT; Wells, Citation2009) and its techniques. Techniques such as Detached Mindfulness (DM; Wells, Citation2005) and Attentional Training Technique (ATT; Wells, Citation1990) are introduced early in the treatment in order to target and modify cognitive inflexibility and beliefs about uncontrollability. In conjunction with our results, need for control and negative beliefs about uncontrollability and corresponding danger of worry along with specific beliefs related to distrusting one’s memory, inability to stop worrying, and the persistence of thoughts as the most influential in the network mean that helping patients to discover flexibility in cognitive control and trust in their minds to self-regulate accords with the existing body of theory and intervention goals in metacognitive therapy (Wells, Citation2009). It has also been evidenced in intervention studies that changes in beliefs about the need for control (Sunde et al., Citation2021) and uncontrollability and danger of worrying (Nordahl et al., Citation2017; Johnson et al., Citation2018; Solem et al., Citation2009) are associated with symptom improvements and with improved functioning and quality of life (Muñoz-Navarro et al., Citation2022) in individuals with common mental disorders.

4.5. Limitations of the study

Several limitations should be considered when interpreting our findings. Cross-sectional network analysis precludes any prospects of causal interpretations in our study. Analysis of the overlap between metacognitive beliefs, cognitive strategies, and symptoms did not include formal testing for statistically significant differences between their network structures. Hence, the results pertaining to their overlap and separability should be taken with caution. Furthermore, only brief scales of measures of cognitive strategies and symptoms were used in this study. Using the full scales could substantially impact the results and interpretations. Different centrality measures can lead to different interpretations under different circumstances with important implications. For example, degree centrality measures the number of connections of an item, whereas the strength centrality is the sum of connections of an item. An item which has a high degree simply means that it has a high number of connections, which does not necessarily make that item the most important in the network if another item has a high strength even with few connections. Thus, an item could have many weak connections, while another item could have few but strong connections, making the latter item more important in the network in terms of the strength of connections.

Furthermore, the strength centrality does not take into account negative connections. As such, in a network which has negative connections, the EI is a better centrality measure. However, the expected influence could also be less relevant when a researcher is interested in an item that both directly and indirectly connects other items via the shortest average distance in the network structure (i.e. closeness). In our study, MCQ26 has the highest EI value, but MCQ22 has the highest closeness value. An item with high closeness means that it can affect changes in the network quickly and can also be affected quickly by changes in any part of the network, making MCQ22 more important, assuming you want to spread activation in the network via the shortest distance. Theoretical interpretability should therefore guide the utility of network analysis for applied practice and the fidelity of findings.

Data were gathered in a convenient sample and does not generalize to clinical populations. The network structure and connectivity may look different in different samples and may potentially vary between diagnostic groups. In addition, the MCQ-30 is a measure of generic metacognitions concerning worry, and it could be that a measure assessing other metacognitive items or domains such as metacognitions about rumination could provide more specific information about relevant metacognitive domains for rumination and depression. These are areas for further investigation.

5. Conclusion

The network structure of dysfunctional metacognitions was very well replicated with the clustering of items corresponding to clinically meaningful substructures and a division of uncontrollability and corresponding danger of worry into beliefs about uncontrollability of worry and danger of worry subcomponents. Comparing the network structures of metacognitive beliefs, CAS and symptoms point to suggest that these components may belong to different levels of the cognitive architecture. Negative metacognitive beliefs and beliefs about (un)controllability might have more prominent functional significance in the mutual connections of dysfunctional meta-domains as well as with CAS and symptoms. For worry and anxiety, negative beliefs about the uncontrollability and corresponding danger of worry were more prominent. For rumination, cognitive self-consciousness was more prominent, whereas for depression, need for control was more prominent. The notion that dysfunctional metacognitions form a network structure provides a new answer to an old question of whether dysfunctional metacognitions are merely reflective of an underlying latent cause whose indicators are functionally independent when conditionalizing on the latent cause. The network structure perspective brings into relief how different sets of metacognitive beliefs interact and reinforce each other, empirically determine the most influential dysfunctional metacognition, and identify which sets of metacognitive beliefs strongly connects with CAS strategies and symptoms in a complex system of combined networks.

Availability of data and material

Data and materials, including codes, will be made available by the authors without undue reservation to any qualified researcher. All analysis codes are available on the Open Science Framework platform at osf.io/s7aq2.

Supplemental material

Supplemental Material

Download PDF (1.1 MB)

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/23311908.2023.2205258

Additional information

Funding

No funds, grants, or other support was received.

Notes on contributors

Frederick Anyan

Fredercik Anyan's research interests are in resilience processes and outcomes for positive mental health and the broad applications of quantitative methods to evaluate the effectiveness of evidence-based treatments inspired by the Metacognitive theory.

Henrik Nordahl

Henrik Nordahl's research interests are in the central mechanisms in the development and maintenance of psychological disorders, and important targets in treatments inspired by the Metacognitve Control System.

Odin Hjemdal

Odin Hjemdal's research interests are in resilience, psychometrics, anxiety, depression, clinical treatment trials, developmental psychopathology and neuropsycholigcal aspects of psychopathology.

References

  • Anyan, F., Morote, R., & Hjemdal, O. (2020a). Prospective relations between loneliness in different relationships, metacognitive beliefs, worry and common mental health problems. Mental Health & Prevention, 19, 200186. https://doi.org/10.1016/j.mhp.2020.200186
  • Anyan, F., Morote, R., & Hjemdal, O. (2020b). Temporal and reciprocal relations between worry and rumination among subgroups of metacognitive beliefs. Frontiers in Psychology, 2378. https://doi.org/10.3389/fpsyg.2020.551503
  • Armour, C., Fried, E. I., Deserno, M. K., Tsai, J., & Pietrzak, R. H. (2017). A network analysis of DSM-5 posttraumatic stress disorder symptoms and correlates in US military veterans. Journal of Anxiety Disorders, 45, 49–15. https://doi.org/10.1016/j.janxdis.2016.11.008
  • Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64(9), 1089–1108. https://doi.org/10.1002/jclp.20503
  • Borsboom, D., & Cramer, A. O. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121. https://doi.org/10.1146/annurev-clinpsy-050212-185608
  • Bouman, T. K., & Meijer, K. J. (1999). A preliminary study of worry and metacognitions in hypochondriasis. Clinical Psychology & Psychotherapy: An International Journal of Theory & Practice, 6(2), 96–101. https://doi.org/10.1002/(SICI)1099-0879(199905)6:2<96:AID-CPP190>3.0.CO;2-G
  • Braithwaite, J. (2022). Complexity science. In F. Rapport, R. Clay-Williams, & J. Braithwaite (Eds.), Implementation Science (pp. 7–12). Routledge.
  • Cartwright-Hatton, S., & Wells, A. (1997). Beliefs about worry and intrusions: The meta-cognitions questionnaire and its correlates. Journal of Anxiety Disorders, 11(3), 279–296. https://doi.org/10.1016/S0887-6185(97)00011-X
  • Epskamp, S., Cramer, A. O., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). Qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48(4), 1–18. https://doi.org/10.18637/jss.v048.i04
  • Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23(4), 617. https://doi.org/10.1037/met0000167
  • Friedman, J., Hastie, T., & Tibshirani, R. (2014). Glasso: Graphical lasso-estimation of Gaussian graphical models. R Package Version, 1(8).
  • Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One, 12(6), e0174035. https://doi.org/10.1371/journal.pone.0174035
  • Golino, H., Shi, D., Christensen, A. P., Garrido, L. E., Nieto, M. D., Sadana, R., Thiyagarajan, J. A., & Martinez-Molina, A. (2020). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods, 25(3), 292. https://doi.org/10.1037/met0000255
  • Grøtte, T., Solem, S., Myers, S. G., Hjemdal, O., Vogel, P. A., Güzey, I. C., Hansen, B., Nordahl, H. M., & Fisher, P. (2016). Metacognitions in obsessive-compulsive disorder: A psychometric study of the metacognitions questionnaire-30. Journal of Obsessive-Compulsive and Related Disorders, 11, 82–90. https://doi.org/10.1016/j.jocrd.2016.09.002
  • Johnson, S. U., Hoffart, A., Nordahl, H. M., Ulvenes, P. G., Vrabel, K., & Wampold, B. E. (2018). Metacognition and cognition in inpatient MCT and CBT for comorbid anxiety disorders: A study of within-person effects. Journal of Counseling Psychology, 65(1), 86. https://doi.org/10.1037/cou0000226
  • Johnson, D. P., & Whisman, M. A. (2013). Gender differences in rumination: A meta-analysis. Personality and Individual Differences, 55(4), 367–374. https://doi.org/10.1016/j.paid.2013.03.019
  • Luminet, O. (2004). Assessment and measurement of rumination. In C. Papageorgiou & A. Wells (Eds.), Rumination: Nature, theory, and treatment of negative thinking in depression (pp. 187–215). Wiley.
  • McNally, R. J. (2016). Can network analysis transform psychopathology? Behaviour Research and Therapy, 86, 95–104. https://doi.org/10.1016/j.brat.2016.06.006
  • Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn state worry questionnaire. Behaviour Research and Therapy, 28(6), 487–495. https://doi.org/10.1016/0005-7967(90)90135-6
  • Morrison, A. P., Wells, A., & Nothard, S. (2000). Cognitive factors in predisposition to auditory and visual hallucinations. British Journal of Clinical Psychology, 39(1), 67–78. https://doi.org/10.1348/014466500163112
  • Muñoz-Navarro, R., Medrano, L. A., Limonero, J. T., González-Blanch, C., Moriana, J. A., Ruiz-Rodríguez, P., & Cano-Vindel, A. (2022). The mediating role of emotion regulation in transdiagnostic cognitive behavioural therapy for emotional disorders in primary care: Secondary analyses of the PsicAP randomized controlled trial. Journal of Affective Disorders, 303, 206–215. https://doi.org/10.1016/j.jad.2022.01.029
  • Nguyen, T. D., Attkisson, C. C., & Stegner, B. L. (1983). Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6(3–4), 299–313. https://doi.org/10.1016/0149-7189(83)90010-1
  • Nolen-Hoeksema, S. (1987). Sex differences in unipolar depression: Evidence and theory. Psychological Bulletin, 101(2), 259. https://doi.org/10.1037/0033-2909.101.2.259
  • Nolen-Hoeksema, S. (1991). Responses to depression and their effects on the duration of depressive episodes. Journal of Abnormal Psychology, 100(4), 569. https://doi.org/10.1037/0021-843X.100.4.569
  • Nolen-Hoeksema, S. (2012). Emotion regulation and psychopathology: The role of gender. Annual Review of Clinical Psychology, 8(1), 161–187. https://doi.org/10.1146/annurev-clinpsy-032511-143109
  • Nolen-Hoeksema, S., & Morrow, J. (1991). A prospective study of depression and posttraumatic stress symptoms after a natural disaster: The 1989 Loma Prieta Earthquake. Journal of Personality and Social Psychology, 61(1), 115. https://doi.org/10.1037/0022-3514.61.1.115
  • Nordahl, H., Anyan, F., Hjemdal, O., & Wells, A. (2022). The network structure of dysfunctional metacognition: Analysis of the MCQ-30. Acta Psychologica, 227, 103622. https://doi.org/10.1016/j.actpsy.2022.103622
  • Nordahl, H., Hjemdal, O., Hagen, R., Nordahl, H. M., & Wells, A. (2019). What lies beneath trait-anxiety? Testing the self-regulatory executive function model of vulnerability. Frontiers in Psychology, 10, 122. https://doi.org/10.3389/fpsyg.2019.00122
  • Nordahl, H., Nordahl, H. M., Hjemdal, O., & Wells, A. (2017). Cognitive and metacognitive predictors of symptom improvement following treatment for social anxiety disorder: A secondary analysis from a randomized controlled trial. Clinical Psychology & Psychotherapy, 24(6), 1221–1227. https://doi.org/10.1002/cpp.2083
  • Nordahl, H., Ødegaard, I. H., Hjemdal, O., & Wells, A. (2019). A test of the goodness of fit of the generic metacognitive model of psychopathology symptoms. BMC Psychiatry, 19(1), 1–9. https://doi.org/10.1186/s12888-019-2266-5
  • Normann, N., & Morina, N. (2018). The efficacy of metacognitive therapy: A systematic review and meta-analysis. Frontiers in Psychology, 9, 2211. https://doi.org/10.3389/fpsyg.2018.02211
  • Normann, N., van Emmerik, A. A., & Morina, N. (2014). The efficacy of metacognitive therapy for anxiety and depression: A meta‐analytic review. Depression and Anxiety, 31(5), 402–411. https://doi.org/10.1002/da.22273
  • O’Carroll, P. J., & Fisher, P. (2013). Metacognitions, worry and attentional control in predicting OSCE performance test anxiety. Medical Education, 47(6), 562–568. https://doi.org/10.1111/medu.12125
  • Papageorgiou, C., & Wells, A. (2003). An empirical test of a clinical metacognitive model of rumination and depression. Cognitive Therapy and Research, 27(3), 261–273. https://doi.org/10.1023/A:1023962332399
  • R Core Team. (2013). R: A language and environment for statistical computing.
  • Robinaugh, D. J., Millner, A. J., & McNally, R. J. (2016). Identifying highly influential nodes in the complicated grief network. Journal of Abnormal Psychology, 125(6), 747. https://doi.org/10.1037/abn0000181
  • Rogier, G., Zobel, S. B., Morganti, W., Ponzoni, S., & Velotti, P. (2021). Metacognition in gambling disorder: A systematic review and meta-analysis. Addictive Behaviors, 112, 106600. https://doi.org/10.1016/j.addbeh.2020.106600
  • Roussis, P., & Wells, A. (2006). Post-traumatic stress symptoms: Tests of relationships with thought control strategies and beliefs as predicted by the metacognitive model. Personality and Individual Differences, 40(1), 111–122. https://doi.org/10.1016/j.paid.2005.06.019
  • Solem, S., Håland, Å. T., Vogel, P. A., Hansen, B., & Wells, A. (2009). Change in metacognitions predicts outcome in obsessive–compulsive disorder patients undergoing treatment with exposure and response prevention. Behaviour Research and Therapy, 47(4), 301–307. https://doi.org/10.1016/j.brat.2009.01.003
  • Spada, M. M., Caselli, G., & Wells, A. (2009). Metacognitions as a predictor of drinking status and level of alcohol use following CBT in problem drinkers: A prospective study. Behaviour Research and Therapy, 47(10), 882–886. https://doi.org/10.1016/j.brat.2009.06.010
  • Spada, M. M., Mohiyeddini, C., & Wells, A. (2008). Measuring metacognitions associated with emotional distress: Factor structure and predictive validity of the metacognitions questionnaire 30. Personality and Individual Differences, 45(3), 238–242. https://doi.org/10.1016/j.paid.2008.04.005
  • Sunde, T., Johnson, S. U., Himle, J. A., Bertelsen, T. B., Haaland, V. Ø., Vogel, P. A., Walseth, L. T., & Haaland, Å. T. (2021). Metacognitions and obsessive beliefs in obsessive–compulsive disorder: A study of within-and between-person effects on long-term outcome. Cognitive Therapy and Research, 45(6), 1105–1119. https://doi.org/10.1007/s10608-021-10210-y
  • Sun, X., Zhu, C., & So, S. (2020). Dysfunctional metacognition across psychopathologies: A meta-analytic review. European Psychiatry, 45, 139–153. https://doi.org/10.1016/j.eurpsy.2017.05.029
  • Team, R. C. (2013). R: A language and environment for statistical computing.
  • Topper, M., Emmelkamp, P. M., Watkins, E., & Ehring, T. (2014). Development and assessment of brief versions of the Penn State worry questionnaire and the ruminative response scale. British Journal of Clinical Psychology, 53(4), 402–421. https://doi.org/10.1111/bjc.12052
  • Wells, A. (1990). Panic disorder in association with relaxation induced anxiety: An attentional training approach to treatment. Behavior Therapy, 21(3), 273–280. https://doi.org/10.1016/S0005-7894(05)80330-2
  • Wells, A. (1995). Meta-cognition and worry: A cognitive model of generalized anxiety disorder. Behavioural and Cognitive Psychotherapy, 23(3), 301–320. https://doi.org/10.1017/S1352465800015897
  • Wells, A. (2005). Detached mindfulness in cognitive therapy: A metacognitive analysis and ten techniques. Journal of Rational-Emotive and Cognitive-Behavior Therapy, 23(4), 337–355. https://doi.org/10.1007/s10942-005-0018-6
  • Wells, A. (2009). Metacognitive therapy for anxiety and depression. Guilford Press.
  • Wells, A. (2019). Breaking the cybernetic code: Understanding and treating the human metacognitive control system to enhance mental health. Frontiers in Psychology, 10, 2621. https://doi.org/10.3389/fpsyg.2019.02621
  • Wells, A., & Cartwright-Hatton, S. (2004). A short form of the metacognitions questionnaire: Properties of the MCQ-30. Behaviour Research and Therapy, 42(4), 385–396. https://doi.org/10.1016/S0005-7967(03)00147-5
  • Wells, A., & Matthews, G. (1994). Attention and Emotion: A clinical perspective. Erlbaum.
  • Wells, A., & Matthews, G. (1996). Modelling cognition in emotional disorder: The S-REF model. Behaviour Research and Therapy, 34(11–12), 881–888. https://doi.org/10.1016/S0005-7967(96)00050-2
  • Wells, A., & Papageorgiou, C. (1998). Relationships between worry, obsessive–compulsive symptoms and meta-cognitive beliefs. Behaviour Research and Therapy, 36(9), 899–913. https://doi.org/10.1016/S0005-7967(98)00070-9