Symptom Checklist-90-Revised: Factor structure and relevance as a measure for the Hierarchical Taxonomy of Psychopathology model

Abstract

The Hierarchical Taxonomy of Psychopathology (HiTOP) is a recently proposed dimensional model of psychopathology that aims to provide a more comprehensive understanding of the organization of mental disorders. The Symptom Checklist-90-Revised (SCL-90-R) is a widely used self-report questionnaire that assesses nine broad dimensions of psychopathologies. The current study reexamined the optimum factor structure of the SCL-90-R (first aim), and its usefulness for the assessment of psychological dimensions from a HiTOP perspective (second aim). These were examined in a community sample of adolescents from Cyprus. A total of 839 adolescents (males = 352, females = 485; no response = 2) with ages ranging from 14 to 18 years [mean (SD) = 15.52 years (0.64 years)] completed the SCL-90-R and the Fünf-Faktoren-Fragebogen für Kinder (FFFK), a personality measure of the Five-Factor Model of personality. Relevant to the first aim, the findings showed the most support for an exploratory structural equal model, with the nine theorized SCL-90-R factors/dimensions. Relevant to the second aim, the SCL-90 dimensions of somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, and phobic anxiety were associated with the five factors in the FFFK as theoretically predicated by the HiTOP model. The SCL-90-R dimensions for paranoid ideation and psychoticism did not. The findings indicated reasonable support for the theorized nine-factor SCL-90 model; and the use of the SCL-90-R dimension scales of somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, and phobic anxiety for measuring the dimensions with similar names in the HiTOP model.

Keywords:

• Adolescents
• Cyprus
• factor structure
• Hierarchical Taxonomy of Psychopathology (HiTOP)
• Symptom Checklist-90-Revised (SCL-90-R)

Conceptualization, taxonomic organization, and measurement are major areas of contention in the field of psychopathology (Simms et al., 2022). Although, traditionally, the dominant clinical conceptualization of psychopathology is categorical, like the Diagnostic and Statistical Manual 4^th edition (DSM-IV; American Psychiatric Association, 1994), dimensional conceptualizations of psychopathology are gaining popularity (e.g., Kotov et al., 2017). One such model is the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017; Ruggero et al., 2019). Given its regency, the HiTOP Clinical Translations workgroup is at present actively engaged in identifying existing measures that can be used for the assessment of psychological dimensions from a HiTOP perspective (Simms et al., 2022; Widiger & Crego, 2019). In this respect, a measure that can be seen as potentially useful for this purpose is the self-report questionnaire called Symptom Checklist-90-Revised (SCL-90-R; Derogatis, 1983; Derogatis & Savitz, 2000). Although its original version was developed forty years ago, its factor structure is still vigorously debated. Despite this, the SCL-90-R continues to be used widely for measuring psychopathology in adolescents and adults (Fan et al., 2024; Pedersen & Karterud, 2010; Preti et al., 2019). It is comprised of nine scales. A cursory examination of these SCL-90-R dimensions raises the possibility that they correspond to different dimensions (at one level or another) in the HiTOP model. The major aim of the current study was to reexamine the factor structure of the SCL-90-R and empirically examine the usefulness of the nine dimensions in the SCL-90-R for assessing conceptually similar dimensions in the current HiTOP model.

The Hierarchical Taxonomy of Psychopathology (HiTOP)

The structure of the HiTOP model

Structurally, in the most recent version, HiTOP organizes psychopathology at six different hierarchical levels, moving upwards from narrow to broader constructs of psychopathology (Kotov et al., 2017; Kotov et al., 2021; Ruggero et al., 2019). Moving upwards, from the bottom to the top, they are signs, symptoms, and maladaptive traits of psychopathology (first level); groups of signs, symptoms, and maladaptive traits called syndromes (second level); closely related syndromes called subfactors (third level); closely related groups of subfactors called spectra (fourth level); closely related groups of spectra called superspectra (fifth level), and a general psychopathology factor. i.e., p factor at the very top (sixth level).

The signs, symptoms, and maladaptive traits of psychopathology at the lowest level of the HiTOP model are the most narrow-band elements of the model, such as impulsivity, dysphoria, insomnia, worry, irritability, and inattention. At the second level, there are currently 11 syndromes. The syndromes are (1) somatic symptom disorder and illness anxiety hypochondriasis; (2) low desire, orgasmic function, and sexual pain; (3) bulimia, anorexia nervosa, and binge eating disorder; (4) social/specific phobia, panic disorder, and obsessive-compulsive disorder; (5) major depression disorder, generalized anxiety disorder, post-traumatic stress disorder, and borderline personality disorder; (6) bipolar I and II; (7) schizophrenia, mood disorder with psychosis, and schizotypal personality disorder; (8) substance-related disorder; (9) antisocial personality disorder, conduct disorder, oppositional defiant disorder, and attention deficit/hyperactivity disorder (ADHD); (10) narcissistic, histrionic, paranoid, and borderline personality disorders; and (11) schizoid, avoidant, dependent, and histrionic personality disorders. Although the HiTOP syndromes do not map onto DSM-5 or ICD-10 disorders, these syndromes form the level that most closely corresponds to them (Smits et al., 2014). The third level contains seven subfactors, and they are (1) sexual problems, (2) eating pathology, (3) fear, (4) distress, (5) mania, (6) substance use, and (7) antisocial behavior. The fourth level comprises six spectra, and they are (1) internalizing, (2) thought disorder, (3) disinhibited externalizing, (4) antagonistic externalizing, (5) detachment, and (6) somatoform (included as a tentative spectrum; Kotov et al., 2017). Superspectra, introduced more recently (Kotov et al., 2021; Krueger & DeYoung, 2020; Watson et al., 2022), are psychosis (combining thought disorder and detachment psychopathologies), externalizing (combining disinhibited and antagonistic psychopathologies), and emotional dysfunction (combining the internalizing and somatoform psychopathologies). The general psychopathology factor or p factor, at the very top, is presumed to reflect the positive covariance of all forms of psychopathology.

Although researchers have noted a number of limitations in the HiTOP model (as discussed in detail by Wittchen & Beesdo‐Baum, 2018), it has been generally argued that the HiTOP has the potential to improve future classificatory models with increased utility for research and practice, and also the development of better psychometric assessment instruments for psychopathology (Wittchen & Beesdo‐Baum, 2018). Thus, continuing research on the HiTOP model would be valuable.

Linking the HiTOP model with the DSM-5 Section III dimensional trait model and five factor model

A feature of the HiTOP model of particular relevance to the current study is that the foundational base for the HiTOP model is the five broad dimensional domains of the DSM-5 Section III dimensional trait model (Widiger & Crego, 2019) The domains are negative affectivity, detachment, antagonism, disinhibition, and psychoticism. More specifically, it has been suggested that the HiTOP spectra for internalizing, thought disorder, disinhibited externalizing, antagonistic externalizing, and detachment are negative affectivity, psychoticism, disinhibition, antagonism, and detachment, respectively (Widiger et al., 2019).

As the DSM-5 Section III dimensional trait model is viewed as “maladaptive variants of the extensively validated and replicated model of personality known as the “Big Five,” or Five-Factor Model of personality (FFM)” (American Psychiatric Association [APA], 2013, p. 773), it can be speculated that the spectra in the HiTOP model could also be related to the FFM. The FFM is a dominant trait personality model (McCrae & Costa, 2008; McCrae et al., 1999; McCrae & Terracciano, 2005). The FFM has broad personality dimensions for extraversion (individual differences in reactivity to positive environmental stimuli); neuroticism (individual differences in reactivity to negative environmental stimuli) or its opposite pole called emotional stability; agreeableness (individual differences in showing and maintaining social harmony with others); conscientiousness (individual differences reflecting how organized, responsible and task-focused an individual is in pursuing goals); and openness to experience (individual differences relating to being “open-minded”). The equivalent FFM dimensions of the DSM-5 Section III dimensions of negative affectivity, detachment, antagonism, disinhibition, and psychoticism are extraversion, emotional stability, agreeableness, conscientiousness, and openness to experience, respectively (Widiger & Crego, 2019). Notwithstanding this, the relationship involving psychoticism and openness to experience has been debated because of weak and/or inconsistent results, probably arising from the alignment of a maladaptive dimension (psychoticism) with what is usually an adaptive dimension (openness), and how these constructs are measured (Widiger & Crego, 2019).

Given how the five DSM-5 trait dimensions could be aligned with the five FFM dimensions, it can be speculated that these HiTOP spectra will be associated negatively with neuroticism, openness to experience, conscientiousness, agreeableness, and extraversion, respectively (see Table 1 in Widiger et al., 2019). Considering this, it can be expected that the relevant signs, symptoms, maladaptive traits, syndromes, and subfactors associated with the relevant spectra, will also be associated similarly with the different FFM personality domains, as proposed for the different spectra.

Development of measures for the HiTOP model

To facilitate and drive research on the HiTOP model, the Measures Development Workgroup of HiTOP has been focusing on the development of HiTOP-based assessment measures. Concurrently, the HiTOP Clinical Translations workgroup has been focusing on identified existing measures that are immediately useful for assessment of psychological disorders from a HiTOP perspective (https://hitop.unt.edu/clinical-tools/hitop-friendly-measures). An example is the Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2011), which is a self-report measure covering at least 9 syndromes that can be aligned to similar syndromes in the HiTOP model. Related to this, it is conceivable that the Symptoms Checklist-90-R (SCL-90-R; Derogatis, 1983; Derogates & Savitz, 2000) although not yet identified as a measure useful for the assessment of psychological disorders from a HiTOP perspective, has the potential to be considered as such.

The SCL-90-R, developed more than four decades ago, is an internationally well-known and used self-report clinical questionnaire (Evers et al., 2012; Hardt & Brähler, 2007). It has nine scales for measuring a range of nine syndromes (henceforth referred to here as dimensions). The dimensions are (1) somatization (for measuring distress arising from bodily perceptions); (2) obsessive-compulsive (for measuring obsessive-compulsive symptoms; (3) interpersonal sensitivity (for measuring feelings of personal inadequacy and inferiority in comparison to others; (4) depression (for measuring depressive symptoms and lack of motivation); (5) anxiety (for measuring anxiety symptoms, including tension); (6) hostility (for measuring negative affect, aggression, and irritability); (7) phobic anxiety (for measuring fears related to specific conditions); (8) paranoid ideation (for measuring symptoms of projective thinking, hostility, suspiciousness, and fear of loss of autonomy); and (9) psychoticism (for measuring symptoms ranging from mild interpersonal alienation to dramatic evidence of psychosis) (Derogatis, 1994; Derogatis & Savitz, 2000).

While our paper was under review, Fan et al. (2024), published a Chinese version of the HiTOP structure, based on ratings of the SCL-90-R. The study examined this in two independent clinical groups of adults and adolescents, using an extended bass-ackwards approach. For both these samples, their model had a general factor at the top, 4 higher-order spectra (Internalizing, Externalizing, Broad Thought Disorder and Somatization and Somatic Anxiety) and 6 subfactors (Distress, Somatoform, Hostility, Fear, Psychosis and Obsessive-Compulsive Disorder). Additionally, the adult sample contained 2 other subfactors: a) Sleep, and b) Suicide and Guilt. Also, at the symptom level, some items were posited to components that diverged from the original SCL-90-R subscales. Based on these findings, the authors concluded that while there is much commonality between their model and the conventional HiTOP model, there are also structural differences between Western and Eastern cultures for the HiTOP model, especially concerning the Somatization spectrum. More importantly, they also concluded that the SCL-90-R can be used to examine the HiTOP structure. However, they did not explain how to align the dimensions in the SCL-90-R with the dimensions in the HiTOP model. In this respect, we propose possible alignments below.

Alignments of the SCL-90-R dimensions with the HiTOP dimensions via the Five-factor model

When the structure and content of the HiTOP model and the dimensions in the SCL-90-R are considered together, it would appear that the SCL-90-R dimensions exist at one level or other in the HiTOP model, thereby alluding to the possibility that the SCL-90-R may have potential for the assessment of psychological disorders from a HiTOP perspective. Indeed, conceptually, the SCL-90-R dimensions can be mapped onto the HiTOP model. Our proposed mapping for this is shown in Supplementary Table 1.

As shown in Supplementary Table 1, the depression and anxiety dimensions can be considered components of the internalizing and distress subfactor; the obsessive-compulsive and phobic anxiety dimensions can be considered components of the internalizing and fear subfactor; hostility can be considered a component of the antagonistic externalizing spectrum and antisocial subfactor; the paranoid ideation and psychoticism dimensions can be considered components of the thought disorder spectrum. In contrast to the dimensions covered above, the relation between interpersonal sensitivity and somatization dimensions with the HiTOP dimensions are not that straightforward. The SCL-90-R subscale for interpersonal sensitivity focuses on feelings of inadequacy and inferiority in comparisons with others and includes symptoms covering self-deprecation, uneasiness, and discomfort during interpersonal interactions. HiTOP does not have either a subfactor or a spectrum that clearly reflects a combination of these symptoms. However, HiTOP has the detachment spectrum that includes schizoid, avoidant, dependent, and histrionic personality disorders as syndromes/disorders, and for this spectrum, the component and maladaptive traits are anhedonia, depressivity, intimacy avoidance, suspiciousness, and withdrawal. Given the inclusion of these features in the detachment spectrum it is probable that SCL-90-R interpersonal sensitivity would be loosely or at least be partially linked with the detachment and the internalizing spectra.

According to HiTOP, the placement of somatoform as a separate spectrum in HiTOP is not that clear and is to be considered provisional for the present moment (Kotov et al., 2017). This is because there is evidence to show that while it can be considered as part of the internalizing spectrum (Krueger et al., 2003; Simms et al., 2012), more recent evidence suggests that with a sufficient number of indicators, it could form its own separate spectrum (Forbes et al., 2017; Kotov et al., 2011; Marek et al., 2020; Sellbom, 2017; Watson et al., 2022) Indeed, Forbes and colleagues (2017) have shown that somatoform and internalizing form the same spectrum at broad (higher) levels of the psychopathology hierarchy, but different factors at more specific (lower) levels of the hierarchy. Therefore, the somatization syndrome can be considered a component of the internalizing spectrum.

Overall, therefore, given these speculated associations it can be argued that the SCL-90-R, although not yet identified as a measure useful for assessment of psychological disorders from a HiTOP perspective, has the potential to be considered as such. However, for this to be credible, empirical support for this possibility has to be demonstrated. One way to accomplish this is by demonstrating that the dimensions of the SCL-90-R have relationships with external correlates (such as the dimensions in the FFM) similar to those predicted for comparable HiTOP dimensions that are related to the same external correlates.

Considering this, when the relations for the SCL-90-R dimensions with the HiTOP factors and dimensions proposed above are considered in relation to the relationships mentioned earlier for the HiTOP factors and dimensions with the FFM, it should be expected that depression and anxiety syndromes would be associated positively with neuroticism (or negatively with emotional stability); obsessive-compulsive and phobic anxiety syndromes should be associated positively with neuroticism (or negatively with emotional stability); hostility should be associated negatively with agreeableness; paranoid ideation should be associated positively with openness to experience. Psychoticism should be associated either positively with openness to experience or have no association with any of the FFM dimensions, including openness to experience; interpersonal sensitivity should be associated negatively with extraversion and positively with neuroticism; and somatization should be associated positively with neuroticism (or negatively with emotional stability). If these relations can be demonstrated, then it can be argued that the SCL-90-R, although not yet identified as a measure useful for the assessment of psychological disorders from a HiTOP perspective, has the potential to be considered as such. Notwithstanding this, for this to have any credibility, the factor structure of the SCL-90-R needs to be robustly established in the first instance. This has been an area of controversy.

Esisting findings for the factor structure of the Symptom Checklist-90-R (SCL-90-R) (Derogatis, 1994; Derogatis & Savitz, 2000)

Corresponding to the nine SCL-90-R dimensions, a number of theoretical models have been proposed for the SCL-90-R (Arrindell et al., 2017; Gomez, Stavropoulos, et al., 2021; Preti et al., 2019; Smits et al., 2014). The more common models include a nine-factor oblique model, where symptoms for each dimension load only on their respective factors; a one-factor model, where symptoms for each dimension, totaling 83 items, load on a single distress factor; and a higher-order model, where symptoms for each of the nine dimensions load only on their respective factors, and these nine dimensions load on a single secondary or higher-order general distress factor. In addition, a bifactor model has also been proposed for the SCL-90-R. In general, for this model there is one general factor and two or more specific factors. Therefore, in the case of the SCL-90-R there is one general factor and 9 specific factors. Typically, in this model, all the items in the measure load on the general latent, and also items belonging to each dimension load on their specific latent factors, and all the latent factors are not correlated with each other (orthogonal model). This specification means that the overall general factor captures all the common (shared) variances between all the items in the measure, and the specific factors capture variances in them that are not shared with the general factor. In general, CFA studies have provided mixed support for these models (Arrindell et al., 2017; Gomez, Liu, et al., 2021; Gomez, Stavropoulos, et al., 2021; Preti et al., 2019; Smits et al., 2014; Urbán et al., 2014).

A recent study by Gomez, Stavropoulos, et al. (2021) of a large general community sample of adolescents and young adults from Greece compared the relative support for the SCL-90-R models mentioned above. Additionally, it also examined the structure of the SCL-90-R in terms of an exploratory structural equation model (ESEM), comprising the theorized nine dimensions, and also a bifactor ESEM model (BESEM), with one general factor and nine specific factors. An ESEM is different from a CFA model with the same number of group factors. The ESEM combines the positives of the exploratory factor analysis (i.e. enabling cross-loadings) and CFA (i.e. being conceptually driven, and allowing the examination of a pre-defined structure) approaches. Subsequently, in an ESEM structure, items are linked to their designated specific factors, as well as all other factors (at rates approximating zero). Studies have shown that ESEM is superior to CFA approaches when testing factor structures (Marsh et al., 2014), Thus as related to the SCL-90-R, this means that in the context of the ESEM model, all items load freely on its predesignated factors (as is also the case with the CFA model), and also all other factors at values close to zero (and not zero as is the case with the CFA model). Results of the Gomez, Stavropoulos, et al. (2021) study revealed more support for ESEM and BESEM models over all the corresponding CFA and BCFA models, with the ESEM model being better supported than the BESEM model. These findings led the researchers to conclude that the ESEM model is the preferred model for the SCL-90-R. However, the support for this model is yet to be replicated. Confirmation of the robustness of this model would be valuable for a better understanding of the factor structure of the SCL-90-R, and also for applying this model in research and clinical settings.

Limitations and omissions of existing data on the SCL-90-R

Overall, based on the literature reviewed above, two areas can be identified that require further research. First, concerning the factor structure of the SCL-90-R, there have been mixed findings (Arrindell et al., 2017; Gomez, Liu, et al., 2021; Gomez, Stavropoulos, et al., 2021; Preti et al., 2019; Smits et al., 2014; Urbán et al., 2014). Although a recent study that compared several currently proposed SCL-90-R models, reported the most support for an ESEM model, with factors corresponding to the nine SCL-90-R scales/dimensions (Gomez, Stavropoulos, et al., 2021), to date no other study has examined and replicated this finding. Confirmation of the robustness of this model would be valuable in terms of the ongoing search for the optimum SCL-90-R factor model. Second, although a recent study by Fan et al. (2024) demonstrated that the SCL-90-R can be used to examine the HiTOP structure, there is no empirical data on how best to align the dimensions in the SCL-90-R with the dimensions in the HiTOP model, although this possibility is conceptually evident.

Aims of the study

Given the limitations and omissions of existing data on the SCL-90-R highlighted here, there were two major aims in the current study. One aim was to evaluate support for the SCL-90-R ESEM model proposed by Gomez, Stavropoulos, et al. (2021) and to compare this model with other popular SCL-90-R models that have been proposed. For this, six different structural models of the SCL-90-R were examined and compared: (1) nine-factor CFA model, with factors corresponding to the nine SCL-90-R scales/dimensions; (2) one-factor CFA model, with symptoms for each scale that makes a total of 83 symptoms loading on single distress factor); (3) a higher order CFA model, with one higher order general factor and nine lower order factors, corresponding to the nine SCL-90-R scales/dimensions; (4) a bifactor CFA model, with one general factor and nine specific factors, corresponding to the nine SCL-90-R scales/dimensions; (5) ESEM model, with factors corresponding to the nine SCL-90-R scales/dimensions nine factors which is the model proposed by Gomez, Stavropoulos, et al. (2021); and (6) BESEM model, with one general factor and nine specific factors corresponding to the nine SCL-90-R scales/dimensions. Based on the findings reported by Gomez, Stavropoulos, et al. (2021), it is expected that the ESEM SCL-90-R will display the most support compared to other structural models.

A second aim was to examine the usefulness of the SCL-90-R for the assessment of psychological disorders from a HiTOP perspective. To achieve this goal, we examined if the nine dimensions in the SCL-90-R were associated with the five factors in the FFM as would be predicted by the HiTOP model. As mentioned earlier, if it can be demonstrated that the SCL-90-R dimensions have relationships with the FFM that are similar to how comparable HiTOP dimensions are related to the FFM dimensions., then it can be taken that the SCL-90-R could be useful for the assessment of psychological disorders from a HiTOP perspective. For this, SCL-90-R depression and anxiety dimensions need to be positively associated with neuroticism (or negatively with emotional stability); obsessive-compulsive and phobic anxiety dimensions need to be associated positively with neuroticism (or negatively with emotional stability); hostility need to be associated negatively with agreeableness; paranoid ideation need to be associated positively with openness to experience; psychoticism need to be associated either positively with openness to experience or have no association with any of the FFM dimensions, including openness to experience; interpersonal sensitivity need to be associated negatively with extraversion and positively with neuroticism; and somatization need to be associated positively with neuroticism (or negatively with emotional stability). In passing, it is important to note that the aim of the study was not to demonstrate the HiTOP model structure of the ratings of the SCL-90-R, as done recently by Fan et al. (2024), but to establish empirical support for the similarity in conceptually comparable dimensions in the SCL-90-R and HiTOP model (for example, HiTOP fear with SCL-90-R phobic anxiety).

Method

Participants

In total, there were 839 respondents. There were 352 (42.0%) males, 485 (57.8%) females, and 2 (0.2%) individuals who did not provide information on their sex. Of the 839 respondents, age was available for 812 participants, (males = 344 males and females = 466). For this group, the age ranged from 14 to 18 years, with a mean age (SD) of 15.52 years (0.64 year). The mean age (SD) of males and females were 15.52 years (0.62 year) and 15.51 years (0.63 years), respectively. There was no significant difference for age, t (df = 808) = .07, ns. Although details are not provided, most participants came from intact families, with both parents having completed at least high school.

Measures

As part of the study, parents of participants provided information on the age and sex of their children targeted for the study. They also provided information on their own marital and educational status. All adolescent participants completed the SCL-90-R (Derogatis, 1994; Derogatis, 1983; Derogatis & Cleary, 1977; Derogatis & Savitz, 2000) and the Fünf-Faktoren-Fragebogen für Kinder, FFFK (Asendorpf & Van Aken, 2003a, 2003b).

The symptom Checklist-90-R (SCL-90-R; Derogatis,1994; Derogates and Savitz, 2000)

Given the Greek speaking population, the Greek adaptation of the SCL-90-R was used in the study (Donias et al., 1991). Psychometrically, it has been claimed that the SCL-90-R has acceptable internal consistency, retest-reliability, and validity (Franke, 2014; Vaurio, 2011). The SCL-90-R was described in some detail in the introduction. Additionally, each item in the SCL-90 is rated on a 5-point scale, ranging from 0 = “not at all” to 4 = “very much”), with higher scores indicating more severity. Examples of items in the different scales are as follows: “Headaches” (somatization scale), “Unwanted thoughts, words, or ideas that won’t leave your mind “(obsessive-compulsive scale), “Feeling that most people cannot be trusted “(interpersonal sensitivity scale); “Feeling low in energy or slowed down “(depression scale); “Nervousness or shakiness inside “(anxiety scale); “Feeling easily annoyed or irritated “(hostility scale); “Feeling afraid in open spaces or on the streets “(phobic anxiety scale); “Feeling others are to blame for most of your troubles “(paranoid ideation scale); and “Having thoughts that are not your own” (psychoticism subscale). For this study, the item scores were used as observable indicators in the CFA/ESEM analyses. The internal consistency (Cronbach’s α) values for the subscales in this data were somatization = .90; obsessive-compulsive = .86, interpersonal sensitivity = .85, depression = .89, anxiety = .88, hostility = .84, phobic anxiety = .82, paranoid ideation = .77, and psychoticism = .87.

Fünf-Faktoren-Fragebogen für Kinder, FFFK (Asendorpf & Van Aken, 2003b)

A Greek version of the Five Factor Questionnaire for Children or the Fünf-Faktoren-Fragebogen für Kinder (FFFK; (Asendorpf & Van Aken, 2003a, 2003b) was used to measure the FFM personality traits of conscientiousness, emotional stability, extraversion, agreeableness and openness (C, ES, E, A, O, respectively). The FFFK consists of a series of bipolar adjective pairs, balanced for positive and negative stems in the first position (Asendorpf & van Aken, 1999). Each personality dimension has eight adjective pairs. Each pair is rated on a five-point Likert scale (i.e., very, somewhat, neither/nor, somewhat, and very) anchored by its adjectives. The internal consistency values of the E, ES, C, and A in the present study were adequate with Cronbach’s values of .73, .65, .61, and .73, respectively. O was low at .37.

Procedure

This current study was conducted in Cyprus. Data was collected in 2012 after receiving approval from the ethical committees of the Cypriot Youth Organization. Prior to data collection, written permission to conduct the study came from (i) the Ministry of Education; (ii) the Teachers’ Council; and (iii) parents and/or guardian of all non-adult participants. Consequent to this, based on the location of the schools, and type of school (academic vs vocational track schools), study participants were preselected randomly by lottery. There were no exclusion criteria as the aim was to collect an inclusive and representative sample. The original database comprised 1373 respondents. Parent consent and adolescent response rate was over 95%. The estimated maximum sampling error was 2.70% at the 95% level of confidence (z > 1.96). A specially trained research team of 13 undergraduates, postgraduates, and PhD students of psychology collected the ratings in the participants’ classrooms during the first two or the last two school hours of a school day, in accordance with the permission provided by the Ministry of Education. This process took approximately 45 min. Participation was anonymous and voluntary, with no incentives offered for participation. Nor were they penalized if they decided to discontinue.

Statistical analysis

Missing values

This current study was part of a larger project that required participants to complete not only the SCL-90-R and the FFFK, but also the Differentiation of Self Inventory (Skowron & Friedlander, 1998), and Family Adaptability and Cohesion Evaluation Scales IV (Olson, 2006) Only the SCL-90-R and FFFK are relevant for the current study. Data for the entire group of respondents was examined for missing values. Of the initial group of 1373 respondents, there were 534 respondents with missing values on the SCL-90-R and/or FFFR. More importantly, missing values were not completely random. Thus, we used listwise deletion to deal with missing values (Kang, 2013). Listwise deletion involves removing from the analysis any participant with a missing value in any one of the variables. The total number of remaining respondents (with fully completed scores for the SCL and FFFK) was 839. Only these participants were included in the analyses.

Evaluation of the optimum model

For all the factor models tested in the study, the Mplus software 7.3 (Muthén & Muthén, 2012) with robust maximum likelihood estimator (MLR), was used. MLR corrects for non-normality in the data set (Muthén & Muthén, 2012). Given that the first aim of the current study was to replicate the earlier Gomez, Stavropoulos, et al. (2021), in particular, support for the ESEM SCL-90-R model, we followed the same statistical steps as in the Gomez, Stavropoulos, et al. (2021) study. As done in that study, to select the optimum CFA models we followed a sequence of four steps: global model fit (step 1); factor clarity (step 2); factor reliability (step 3), and; support for external validity (step 4). These steps are summarized in Supplementary Table 2. The reader is referred to the Gomez, Stavropoulos, et al. (2021) study for more details on steps 1, 2 and 3. Step 4 (i.e., support for the external validity) in the current study involved how the factors in the preferred SCL-90-R model were correlated with the factors in the FFM model, as predicted by the HiTOP model. We used Cohen’s (1988) guidelines for interpreting the effect sizes of correlations (r = 0.10 is small, r = 0.30 is medium, and r = 0.50 is large).

Examination of the usefulness of the SCL-90-R for the assessment of psychological dimensions from a HiTOP perspective

Given that in Step 4, support for the external validity of the preferred SCL-90-R model was tested by examining how the factors in the preferred SCL-90-R were correlated with the factors in the FFM model, as predicted by the HiTOP model, the findings for this evaluation are relevant for addressing the second aim of the study (i.e., examination of the usefulness of the SCL-90-R for the assessment of psychological disorders from a HiTOP perspective), as discussed earlier.

Results

Evaluation of the optimum SCL-90-R model

Step 1: Comparison of global fit of all SCL-90-R models

Table 1 presents the global fit values for all the SLC-90-R models tested in the study. As shown, all model tested had sufficient fit in terms of their RMSEA values. The CFI and TLI values showed a poor fit for all the CFA models. Thus, all the CFA models showed mixed fit. The CFI and TLI values of both the ESEM models (ESEM and BESEM) showed average fit. However, there was no difference between these models in terms of ΔCFI and ΔRMSEA. Therefore, it can be taken that the support for the ESEM and BESEM were somewhat comparable, and better than all the CFA models. Thus, in Step 2, we examined the clarity of the factors in the ESEM and BESEM models.

Table 1. Fit of all the SCL-90-R models tested in the study.

	Fit values
Models	χ^2 (df)	CFI	TLI	RMSEA (90% CI)	AIC	BIC
CFA 1-factor	10,568.81 (3320)	.72	.722	.05 (.05–.05)	175,020	176,198
CFA 9-factor	8643.92 (3284)	.80	.793	.04 (.04–.04)	172,104	173,453
Second order CFA with 9 primary factors	9032.46 (3311)	.78	.780	.04 (.04–.04)	172,661	173,882
BCFA with 9 specific factors	No convergence
ESEM 9-factor	4916.90 (2692)	.91	.89	.03 (.03–.03)	167,029	171,179
BESEM with 9 specific factors	4969.60 (2618)	.91	.88	.03 (.03–.03)	166,825	171,325

Note. CI = confidence interval; χ² = chi-square; df = degrees of freedom; CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; BCFA = bifactor confirmatory factor analysis; BESEM = bifactor exploratory structural equation modeling. RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis Index; AIC = Akaike Information Criterion; BIC = Bayesian information criterion.

Step 2: Factor loadings in the ESEM and BESEM SCL-90-R models

The factor loadings for the ESEM and BESEM models are shown in Supplementary Table S3. Table 2 provides a summary of the number of designated salient and non-designated salient factor loadings (including cross-loadings where applicable) in the ESEM and BESEM models. As shown in Table 2, for the ESEM model, for all factors, except the anxiety and paranoid ideation factors, the designated symptoms loaded saliently on with their respective factors. In contrast, none of the ten designated symptoms loaded saliently on the anxiety factor, and only two of the six designated symptoms loaded saliently on the paranoid ideation factor. Generally, non-designated symptoms did not load saliently on non-designated factors. With the exception of one symptom on the anxiety factor, there was no other significant negative loading by non-designated symptoms on the different factors. For the BESEM model, all 83 symptoms loaded saliently on the general factor. With the exception of the somatization and hostility symptoms, the majority of the symptoms for the other seven specific factors did not load saliently on their designated factors. Only one symptom showed salient cross-loading, and there was no negative loading. The inter-correlations of the factors in the ESEM model are shown in Supplementary Table S4. As shown, anxiety did not show significant correlations with any other factor, except with psychoticism. All other factors were correlated significantly and positively. Taken together, these findings can be interpreted as indicating that although the ESEM factors was not completely clearly defined, they were more clearly defined than the factors in the BESEM model. This model was therefore deemed tentatively as our preferred model and it was examined further for the reliabilities and validities of its factors.

Table 2. Summary of the number of targeted and non-targeted factor loadings in the ESEM and BESEM models.

Number of	G	SO	OC	ÌS	DE	AN	HO	PA	PI	PY
ESEM model
Targeted items		12	10	9	13	10	6	7	6	10
Salient targeted items		12	7	8	8	0	6	6	2	6
Salient targeted items (negative)		0	0	0	0	1	0	0	0	0
Nontargeted items		71	73	74	70	73	77	76	77	73
Salient nontargeted items		4	2	2	8	1	4	3	4	4
Salient negative nontargeted items		0	0	0	0	1	0	0	0	0
BESEM model
Targeted items	83	12	10	9	13	10	6	7	6	10
Salient targeted items (positive)	83	9	1	1	4	1	5	3	0	2
Salient targeted items (negative)	0	0	0	0	0	0	0	0	0	0
Nontargeted items	–	71	73	74	70	73	77	76	77	73
Salient nontargeted items	–	0	0	2	0	1	0	0	0	2
Significant negative nontargeted items	–	0	0	0	0	0	1	0	0	0

Note. G = General; SOM = Somatization; OC = Obsessive-Compulsive; IS = Interpersonal Sensitivity; DEP = Depression; AN = Anxiety; HOS = Hostility; PANX = Phobic Anxiety; PI = Paranoid Ideation; PSY = Psychoticism.

Reliabilities of the factors in the SCL-90-R model (Step 3)

Table 3 presents the omega coefficient reliability values for the nine factors in the ESEM SCL-90-R model. As shown in Table 3, for the ESEM model, the ω value values for all factors, except anxiety and paranoid ideation ranged from .41 to .83. The values of seven of them were either very close (.49) or above the adequacy threshold of .50, and were substantial based the guidelines proposed by Smits et al. (2014). For anxiety and paranoid ideation, they were .05 and .22, i.e., anxiety was low and paranoid ideation was moderate (Smits et al., 2014). The SCL-90-R factor reliabilities were also computed based on the factor loadings of the designated symptoms in the CFA 9-factor oblique model. As shown in Table 3, the model-based reliability (ω value) for all factors was high, ranging from .79 to .91, thereby further supporting the reliabilities of all nine SCL-90-R factors, including anxiety and paranoid ideation.

Table 3. Omega coefficient reliabilities for the factors in the SCL-90-R ESEM and CFA models, with nine group factors.

	SOM	O-C	IS	DEP	ANX	HOS	PANX	PI	PSY
ESEM model	.83	.49	.49	.67	.05	.79	.61	.22	.41
CFA 9-factor	.91	.87	.88	.91	.89	.86	.82	.79	.86

Note. SOM = Somatization; OC = Obsessive-Compulsive; IS = Interpersonal Sensitivity; DEP = Depression; AN = Anxiety; HOS = Hostility; PANX = Phobic Anxiety; PI = Paranoid Ideation; PSY = Psychoticism.

External validities of the factors in the ESEM model (Step 4)

Table 4 shows the correlation coefficients for the associations of the factors in the FFFK and the nine factors in the ESEM SCL-90-R model. Somatization was correlated negatively with emotional stability. Obsessive-compulsive was also correlated negatively with emotional stability. Both these correlations were of small effect sizes. Interpersonal sensitivity was correlated negatively with emotional stability, with a medium effect size, and positively with agreeableness, with a small effect size. Depression was correlated negatively with extraversion, with a small effect size, and emotional stability, with a medium effect size. Anxiety was correlated negatively with emotional stability, with a medium effect size, and positively with conscientiousness, with a small effect size. Hostility was correlated negatively with emotional stability and agreeableness, and positivity with openness to experience. They were all of small effect sizes. Phobic anxiety was correlated negatively with extraversion and emotional stability, with small effect sizes. Paranoid ideation was correlated positively with extraversion, and psychoticism was correlated negatively with extraversion. Both were of small effect sizes. Supplementary Table S5 shows the unique associations of the factors in the FFFK with the nine factors in the SCL-90-R ESEM model from the SEM regression analysis in which all the FFM factors were regressed on all the SCL-90-R factors in the ESEM model. As shown, all the significant associations reported for the correlations, except the association for obsessive-compulsive with emotional stability showed significant unique associations in the expected directions.

Table 4. Correlations for the FFFK factors with the SCL-90-R subscale factor scores in the ESEM model.

	FFFK factors
SCL-90-R dimensions	E	ES	A	C	O
Somatization	.09	−.26***	−.01	.03	−.03
Obsessive-compulsive	−.12	−.20*	−.07	−.02	.11
Interpersonal sensitivity	−.10	−.37***	.19**	.14	.01
Depression	−.13*	−.33***	.00	−.02	.01
Anxiety	.13	−.44***	.01	.24*	.03
Hostility	.09	−.27***	−.25***	−.06	.15**
Phobic anxiety	−.11*	−.25**	.00	.13	−.02
Paranoid ideation	.15*	−.10	−.13	.03	−.04
Psychoticism	−.15*	−.12	−.03	.02	−.03

Note. E = extraversion, Es = emotional stability; A = agreeableness; C = Conscientiousness; O = openness to experience; FFFK = Fünf-Faktoren-Fragebogen für Kinder. ***p < .001, **p < .01, *p < .05.

Overall, therefore, seven (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility and phobic anxiety) of the nine SCL-90-R factors were associated with FFM personality dimensions, as would be expected by predictions from the HiTOP model, with six of them showing unique associations (the exception being obsessive-compulsive). Paranoid ideation and psychoticism were not associated with openness to experience as predicted by the HiTOP model. Instead, paranoid ideation was associated positively with extraversion, and psychoticism was associated negatively with extraversion. Notwithstanding this, as seven SCL-90-R (out of nine) dimensions showed the expected associations with FFM personality dimensions, the overall findings can be interpreted as indicating reasonable support for the external validity of the ESEM SCL-90-R factors.

Examination of the usefulness of the SCL-90-R for the assessment of psychological disorders from a HiTOP perspective

As mentioned previously, the analyses conducted in Step 4 to evaluate the external validities of the factors in the ESEM SCL-909-R model are also relevant for evaluating the associations of the SCL-90-R factors with the FFM factors as predicted by the HiTOP model. As already noted, the SCL-90-R somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility and phobic anxiety dimensions showed association with the FFM factors as predicted by the HiTOP model. Additionally, the correlations for somatization, obsessive-compulsive, and phobic anxiety with their theoretical expected personality dimensions (neuroticism in all instances) were all .20 (the cutoff level used in the study to infer meaningful correlations; Mukaka, 2012) or higher, thereby indicating that they were all meaningful. Although interpersonal sensitivity, depression, anxiety and hostility were also associated with personality dimensions not predicted by the HiTOP model, this is unlikely to be problematic as their correlations were less than .20. Therefore, as the associations for seven SCL-90-R dimensions with their expected personality dimensions were meaningful, the overall findings can be interpreted as showing potential support for seven of its scales (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility and phobic anxiety) for assessing psychological disorders from a HiTOP perspective.

Discussion

The primary aims of the study were to examine the factor structure of the SCL-90-R, and the usefulness of the SCL-90-R for the assessment of psychological disorders from a HiTOP perspective. In relation to the factor structure, the findings showed that the ESEM (symptoms loading on their own designated nine factors as well as all other factors at values close to zero) and BESEM (symptoms loading on the general factor as well as their own designated nine factors as well as all other factors at values close to zero) models had sufficient global fit, whereas the CFA one-factor (all symptoms loading on a single distress factor), nine-factor (symptoms for each of the nine scale loading only on their respective factors), bifactor (all symptoms loading on the general latent, and also symptoms belonging to each specific factor loading on their own specific factors), and higher order (symptoms for each of the nine scales loading only on their respective factors, and the nine factors loading on a single secondary/higher order general/distress factor) models had mixed fit. Such findings have been reported in past studies (Arrindell et al., 2017; Preti et al., 2019; Smits et al., 2014; Urbán et al., 2014). For the ESEM model, for all factors, except the anxiety and paranoid ideation factors, most of the designated symptoms loaded saliently on their respective factors. For the BESM model, all symptoms loaded saliently on the general factor. However, with the exception of the somatization and hostility symptoms, the majority of the symptoms for the other seven specific factors did not load saliently on their designated factors. For the ESEM model, the reliabilities (ω) for all factors, except anxiety and paranoid ideation were adequate or very close to adequate. Additionally, there was support for externality validities of seven (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility and phobic anxiety) of the nine SCL-90-R factors (the exception being paranoid ideation and psychoticism).

As will be noticed, when the findings presented in the previous paragraph, are considered in the context of the HiTOP model, somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility and phobic anxiety showed associations with the FFM factors as predicted by the HiTOP model. More specifically, as expected, somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, and phobic anxiety were correlated negatively with emotional stability; and hostility was correlated negatively with agreeableness. All these associations, except the association for obsessive-compulsive with emotional stability, showed significant unique associations in the expected directions, as predicted by the HiTOP model. Our findings also showed that although not expected, interpersonal sensitivity was correlated positively with agreeableness; depression was correlated negatively with extraversion, anxiety was correlated positively with conscientiousness; hostility was correlated positively with openness to experience. Although interpersonal sensitivity, depression, anxiety and hostility were associated with personality dimensions not predicted by the HiTOP model, this is not considered problematic as the correlations were low (i.e., < .20). Inconsistent with predictions from the HiTOP model, our findings indicated that paranoid ideation was correlated positively with extroversion and had no association with openness to experience; psychoticism was correlated negatively with extroversion and had no association with openness to experience. However, these findings were not that surprising. Regarding the lack of association between psychoticism and openness to experience, an updated version of the personality dimensions underlying the HiTOP model has pointed out that the empirical literature has shown weak and/or inconsistent results, probably arising from alignment of a maladaptive dimension (psychoticism) with what is usually an adaptive dimension (openness), and how these constructs are measured (Widiger & Crego, 2019). It is conceivable that the positive association for paranoid ideation with extroversion rather than openness to experience found in the current study can be explained by the robust positive association often found between extroversion and openness to experience (Costa & McCrae, 1992; Gomez, Liu, et al., 2021; Gomez, Stavropoulos, et al. (2021).

Overall, related to our first aim, our findings were highly comparable and replicated the findings reported by Gomez, Stavropoulos, et al. (2021), and they reconfirm the support for the ESEM model as the preferred model for the SCL-90-R (Gomez, Liu, et al., 2021; Gomez, Stavropoulos, et al., 2021). Although this could appear to support the theorized nine-factor for the SCL-90-R (Derogatis, 1983; Derogatis & Savitz, 2000), this needs to be viewed with caution as none of the designated anxiety symptoms loaded saliently on the anxiety factor and one non-designated symptom loaded saliently on this factor. Thus, the anxiety factor was not clearly defined. Additionally, it lacked sufficient reliability. The study by Gomez, Stavropoulos, et al. (2021) also reported that the anxiety factor for the ESM SCL-90-R scale was not clearly defined and lacked adequate reliability. Taken together, these findings indicate that the SCL-90-R anxiety scale may not be useful for screening anxiety, highlighting the need for revising the scale, as was done during the revision from SCL-90 to SCL-90-R (Vaurio, 2011). Our findings suggest that the preferred scoring method for the SCL-90-R is to assign scores to each scale, except for the anxiety, paranoid ideation, and psychoticism scales. This recommendation aligns with that of Gomez, Stavropoulos, et al. (2021), which also advises against scoring for anxiety. Additionally, our findings do not support deriving an overall general distress score using all the symptoms, as proposed for the SCL-90-R (Derogatis, 1983; Derogatis & Savitz, 2000).

In relation to our second aim, as our findings indicated that somatization, obsessive-compulsive, and phobic anxiety are associated clearly with personality dimensions as predicted by the HiTOP model, the scales for these dimensions can be considered useful for understanding their corresponding psychological dimensions in the HiTOP model. Expressed differently, they can be considered pure indicators of the corresponding HTOP dimensions. While interpersonal sensitivity, depression, anxiety and hostility were associated with personality dimensions as predicted by the HiTOP model, they were also associated with personality dimensions not predicted by the HiTOP model. Thus, the scales for depression, anxiety and hostility are not pure indicators of their corresponding HiTOP dimensions. This means that although we have suggested earlier that they can be used to assess their corresponding psychological disorder from a HiTOP perspective, they need to be used with caution. In this regard, there are serious reservations over the use of the anxiety scale as it is poorly defined and lacks sufficient reliability. As paranoid ideation and psychoticism failed to show any meaningful associations with any of the FFM personality dimensions, it is proposed that they are not useful for consideration as scales that can assess comparable psychological dimensions from a HiTOP perspective.

Although we have provided valuable new psychometric findings for the SCL-90-R, the findings and interpretations in this study need to be considered with several limitations in mind. First, as it is possible that background factors, such as age, gender and ethnicity could influence ratings of SCL-90-R items, the failure to control for these effects may have confounded our findings. Second, as there is no information about those who knew of the study but did not respond to the invitation to participate, or those omitted from the study because of missing values, we do not know how these impacted our results. Third, as this study examined a community sample that was not truly random, the findings may not be generalizable. Also, the findings may not be applicable to clinically diagnosed adolescents and young adults. Fourth, as the SCL-90-R and the other measures used in the study were self-report questionnaires, the ratings may have been influenced by the common method variance effect. Fifth, the conclusions made in this study on the associations of the SCL-90-R and FFM dimensions are based on a single sample and need confirmation. Sixthly, as were no exclusion criteria used to select participants, it is conceivable that participants affected by medication, drug addiction, intellectual disability, psychological problems, and literacy problems, may have provided data that could potentially confound our findings. Seventh, the study examined empirical support for the alignment of the SCL-90-R dimensions with the HiTOP dimensions indirectly via the FFM model and not directly in terms of how the SCL-90-R dimensions were associated with HiTOP dimensions. Thus, our methodology may be criticized. However, as mentioned in the method section, the data was collected in 2012, as part of a larger project. Thus, the data collection predated the first articulation of the HiTOP modal (Kotov et al., 2017). Consequently, the current project was not conceptualized at that time, and no data was collected for HiTOP specific measures. Notwithstanding all these limitations, our findings do support the use of the SCL-90-R in clinical practice and in research related to the HiTOP model. At a more general level, our study provides a methodological approach for identifying existing measures that can be used for the assessment of psychological disorders from a HiTOP perspective, and in that way, impetus for more studies in this area, taking into consideration the methodologies used in the current study, controlling for the limitations highlighted here.

In summary, the current study reexamined the optimum factor structure of the SCL-90-R, and its usefulness for the assessment of psychological disorders from a HiTOP perspective in a community sample of adolescents from Cyprus. The findings indicated reasonable support for the theorized nine-factor SCL-90 model; and also using the SCL-90 scales of somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, and phobic anxiety as measures for these constructs in the HiTOP model. In conclusion, we hope the findings and interpretations made in these studies could be taken into consideration by the HiTOP Clinical Translations workgroup as they continue to search for existing measures that can be used for the assessment of psychological disorders from a HiTOP perspective. This is especially so with regard to the scales for somatization, obsessive-compulsive, interpersonal sensitivity, depression, hostility and phobic anxiety, but not anxiety, paranoid ideation and psychoticism.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data are available upon request from the corresponding author due to the ethics application received.