Psychometric properties of the Children’s Revised Impact of Event Scale (CRIES-8) among refugee adolescents from Afghanistan, Syria, and Somalia

ABSTRACT

Background: High levels of post-traumatic stress are well documented among refugees. Yet, refugee adolescents display high heterogeneity in their type of trauma and symptom levels.

Objective: Following the recurrent plea for validated trauma screening tools, this study investigated the psychometric properties of the Children’s Revised Impact of Event Scale (CRIES-8) among refugee adolescents from Afghanistan (n = 148), Syria (n = 234), and Somalia (n = 175) living in Europe.

Method: The model fit for the confirmatory factor structures was tested, as well as measurement invariance between the three groups. The robustness of results was evaluated by testing measurement invariance between recently arrived and settled adolescents, and between different response labelling options. Reliability (α, ω, and ordinal α), criterion validity, and prevalence estimates were calculated.

Results: The intrusion subscale showed a better stable model fit than the avoidance subscale, but the two-factor structure was mainly supported. Configural measurement invariance was achieved between Afghan and Somali adolescents, and strong measurement invariance between Syrian and Somali adolescents. The results were robust considering the time living in the host country and response labelling styles. Reliability was low among Afghan and Syrian adolescents (.717−.856), whereas it was higher among Somali adolescents (.831−.887). The total score had medium-sized correlations with emotional problems (.303−.418) and low correlations with hyperactivity (.077−.155). There were statistically significant differences in symptom prevalence: Afghan adolescents had higher prevalence (55.5%) than Syrian (42.8%) and Somali (37%) adolescents, and unaccompanied refugee minors had higher symptom prevalence (63.5%) than accompanied adolescents (40.7%).

Conclusions: This study mostly supports the use of the CRIES-8 among adolescents from Afghanistan, Syria, and Somalia, and even comparative analyses of group means. Variation in reliability estimates, however, makes diagnostic predictions difficult, as the risk of misclassification is high.

HIGHLIGHTS

• We investigated the psychometric properties of the 8-item Children’s Revised Impact of Event Scale (CRIES-8) among refugee adolescents from Afghanistan, Syria, and Somalia living in Europe.

• We found support for the CRIES-8 as a suitable assessment tool for Afghan, Syrian, and Somali adolescents.

• The reliability of the CRIES-8 was low among Afghan and Syrian adolescents, whereas among Somali adolescents, reliability was higher.

Antecedentes: Niveles altos de estrés postraumático en refugiados están bien documentados. Sin embargo, los adolescentes refugiados muestran una alta heterogeneidad en el tipo de trauma y sus niveles de sintomatología.

Objetivo: Siguiendo la petición recurrente de herramientas validadas para pesquisa de trauma, este estudio investigó las propiedades psicométricas de la Escala Revisada del Impacto de Eventos en Niños (CRIES-8 por sus siglas en inglés) en adolescentes refugiados de Afganistán (n = 148), Siria (n = 234) y Somalia (n = 175) que viven en Europa.

Método: Se probó el ajuste del modelo para las estructuras factoriales confirmatorias, así como la invarianza de medición entre los tres grupos. La solidez de los resultados también se evaluó probando la invarianza de medición entre adolescentes recién llegados y los ya establecidos, y entre las diferentes opciones de respuestas etiquetadas. Se calculó la confiabilidad (α, ω y α ordinal), la validez de criterio y las estimaciones de prevalencia.

Resultados: La subescala de intrusión mostró un mejor ajuste estable del modelo que la subescala de evitación, pero la estructura de dos factores fue principalmente respaldada. Se logró una invarianza de medición configural entre adolescentes afganos y somalíes, y una fuerte invarianza de medición entre adolescentes sirios y somalíes. Los resultados fueron robustos considerando el tiempo vivido en el país anfitrión y los estilos de respuestas etiquetadas. La confiabilidad fue baja entre los adolescentes afganos y sirios (.717–.856), mientras que fue mayor entre los adolescentes somalíes (.831–.887). La puntuación total tuvo correlaciones de tamaño mediano con problemas emocionales (.303–.418) y correlaciones bajas con hiperactividad (.077–.155). Hubo diferencias estadísticamente significativas en la prevalencia de los síntomas: los adolescentes afganos tuvieron una alta prevalencia (55.5%) en comparación con los adolescentes sirios (42.8%) y somalíes (37%), y los refugiados menores de edad no acompañados (URMs por sus siglas en inglés) (63.5%) en comparación con los adolescentes acompañados (40.7%).

Conclusiones: Este estudio respaldó mayoritariamente el uso de la CRIES-8 en adolescentes de Afganistán, Siria y Somalia, e incluso el análisis comparativo de medias grupales. Sin embargo, la variación en las estimaciones de confiabilidad dificulta las predicciones diagnósticas, ya que el riesgo de clasificación errónea es alto.

KEYWORDS:

• Post-traumatic stress symptoms
• CRIES-8
• psychometric properties
• refugees
• adolescents

PALABRAS CLAVE:

• Síntomas de estrés postraumático
• CRIES-8
• propiedades psicométricas
• refugiados
• adolescentes

1. Introduction

Almost half of the world’s forcibly displaced population are minors, many of them originating from Syria, Afghanistan, and Somalia. Significant numbers of them flee without the presence of their parents or caregivers, as unaccompanied refugee minors (URMs). Refugee children and adolescents are considered particularly vulnerable, as the displacement happens at a crucial time in their physical, emotional, social, and cognitive development (World Health Organization, 2018). In this article, the term ‘refugee adolescent’ refers to internationally (forcibly) displaced adolescents regardless of their legal status (e.g. asylum applicants, those with a recognized refugee status, or unaccompanied minors with a temporary protection status).

Exposure to war and migration-related trauma are well-documented risk factors for mental health problems (Bogic et al., 2015). Yet, refugee children and adolescents are a heterogeneous group in terms of exposure to adversities and the effect of these adversities on their mental health (Dangmann et al., 2022). Reviews also describe a wide range of prevalence rates for different types of mental distress, such as post-traumatic stress disorder (PTSD), depression, and anxiety, that vary with age, gender, measurement type, country of origin, and country of settlement (Blackmore et al., 2020; Kien et al., 2019). For example, reviews indicated that the overall PTSD prevalence estimates among refugee minors were 22.7% (Blackmore et al., 2020), or varied between 19.0% and 52.7% (Kien et al., 2019).

Exposure to forced migration-related trauma and the development of PTSD have been associated with a range of consequences for health and well-being, such as for decreased cognitive skills (Mirabolfathi et al., 2022; Mueller et al., 2021; Scharpf et al., 2021), higher risks for psychiatric care and mortality (Dunlavy et al., 2023), and difficulties in education and employment (Borsch et al., 2019; Manhica et al., 2019). Owing to the severe consequences that post-traumatic stress symptoms (PTSS) may have on an adolescent’s life, it is important to identify the most vulnerable adolescents and treat them without delay. Many tools are being used for the assessment of trauma and mental health in refugee adolescents, even though their validity has not been confirmed in these populations. Recent reviews on mental health screening and assessment tools for forcibly displaced adolescents have concluded that, overall, the psychometric evidence was relatively weak (Gadeberg et al., 2017; Verhagen et al., 2022). There is, therefore, an urgent need for validated trauma symptom and mental health screening tools for refugee adolescents (Gadeberg & Norredam, 2016) and for researchers to critically evaluate the utility and validity of the most used screening tools, including their cross-cultural validity (Verhagen et al., 2022). In this study, we focus on the widely used 8-item Children’s Revised Impact of Event Scale (CRIES-8), which builds on the Impact of Event Scale (IES), originally developed by Horowitz et al. (1979) and later revised for use in children by Yule (1992). The CRIES-8 has been recommended for use in assessing PTSS by an international consortium of experts on PTSD among children and adolescents (Krause et al., 2021).

The CRIES-8 was originally developed as a screening tool based on two large data sets, including over 2000 children in one school area in Bosnia. The independent factor analyses on the two data sets produced remarkably close factor structures (Smith et al., 2003). Over the years, many studies from various countries and with various languages have made use of the CRIES-8 from a screening perspective (Bhushan & Kumar, 2007; Hassan et al., 2018). It is essential to emphasize that the CRIES-8 was never intended as a diagnostic instrument. Instead, aligning with the pressing need for user-friendly assessments following disasters and in other scenarios where a rapid evaluation is paramount, its primary purpose is to provide a valuable screening tool that aids in identifying individuals who may require further, more in-depth assessments by professionals.

In the development of the IES, two major response sets, describing symptoms of intrusion and avoidance, were abstracted from in-depth evaluation and psychotherapy interviews (Horowitz et al., 1979). Intrusion was characterized by unbidden thoughts and images, troubled dreams, strong pangs or waves of negative feelings, and repetitive behaviour. Avoidance responses included ideational constriction, denial of the meanings and consequences of the event, blunted sensation, behavioural inhibition or counterphobic activity, and awareness of emotional numbness.

In the original study by Horowitz et al. (1979), the internal consistency of the subscales (Cronbach’s α) was high (intrusion = 0.78, avoidance = 0.82). Also, a correlation of 0.42 (p < .01) between the intrusion and avoidance subscale scores indicated that the two subsets were associated but did not measure the same dimension. Later, Yule (1992) found that some items of the original IES were being misinterpreted by children, and they developed the shortened version with eight items that best reflected the underlying two-factor structure. The CRIES-8 consists of four items measuring intrusion and four items measuring avoidance. Developed in 1979/1992, it does not contain items pertaining to the remaining diagnostic criteria for PTSD, in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (American Psychiatric Association, 2022) (negative changes in mood and cognition, and increased arousal and reactivity) or in the 11th revision of the International Statistical Classification of Diseases (ICD-11) (World Health Organization, 2019) (persistent perceptions of heightened current threat), launched in 2013 and 2018, respectively.

So far, findings on the psychometric properties of the CRIES-8 have been positive. For instance, Deeba et al. (2014) examined the psychometric properties of the CRIES-8 among 1342 Bangladeshi children and adolescents aged 9–17 years. They found that confirmatory factor analysis (CFA) supported factor structures (intrusion, avoidance), although multigroup confirmatory factor analysis (MGCFA) showed significant differences in the factor structures between males and females, and between younger and older children. Another study, by de Sousa Magalhães et al. (2018), found that the two latent constructs of intrusion and avoidance were replicated among a sample of Brazilian children and adolescents, but the study did not provide further information on construct validity.

A third study investigated the factorial structure and internal consistency of the CRIES-8 among recently arrived URMs (Salari et al., 2017). The sample consisted of 208 asylum seekers who were 9–18 years old. Almost all participants were male (97.6%) and the vast majority was from Afghanistan (81.4%). All fit indices suggested satisfactory model fit for the two factors model of intrusion and avoidance. Internal consistency (Cronbach’s α) for intrusion was 0.74, for avoidance 0.65, and for the PTSD total score (a combination of the intrusion and avoidance scales) 0.76. Of the 208 participants, 159 (76.4%) scored above the cut-off of 17 or above (at risk of PTSD). The number of participants from countries other than Afghanistan was too small to allow for separate analysis.

Some other studies have also reported findings on the psychometric properties of the CRIES-8: for instance, Fängström et al. (2020) found that among unaccompanied refugee adolescents, the CRIES-8 showed good internal consistency and its factor structure was confirmed, but test–retest reliability and longitudinal invariance indicated potential instability. The supplement does not report exact estimates. Another study, by Smith et al. (2003), among early adolescent children from Bosnia, reported that the CRIES-8 had an α coefficient of 0.75, and that principal component and exploratory factor analyses showed two-factor solutions with similar loadings for each item.

Evidently, the CRIES-8 instrument has been validated with only a few groups and mainly non-refugees, despite being frequently used to assess PTSD among refugee children and adolescents. There is no psychometric evidence of the cross-cultural measurement invariance, although cross-cultural heterogeneity in prevalence and symptomatology is known (Solberg et al., 2020). The translated items may be conceptually or psychometrically problematic if some ethnic groups do not ascribe the same meaning to the item and the appertaining response category. Specifically, there is scarce psychometric evidence for the CRIES-8 among some of the most common recent refugee groups in Europe; namely, those from Afghanistan, Syria, and Somalia (UNHCR, 2022). Furthermore, measurement invariance has not been tested, so it is not known whether the instrument functions similarly in these groups. Since the time spent in a new host country may have a substantial effect on adolescents’ language skills and cultural understanding (Borsch et al., 2021; Kartal et al., 2019), the measurement invariance between recently arrived refugee adolescents and those who have already settled should be considered. Finally, reliability has been estimated using only the α coefficient, which has a strict requirement for the essential τ-equivalence of indicators (McNeish, 2018). According to modern psychometric evidence recommendations, in this study, we test these assumptions and provide reliability estimates that can account for error-related variance in indicators (Flora, 2020).

2. Research aims

The aim of this study was to investigate the psychometric properties of the CRIES-8 among adolescents from Afghanistan, Syria, and Somalia. First, we tested the model fit of the two-factor structure of the CRIES-8 within each of the three country groups, and also the measurement invariance between groups. In this way, we could evaluate whether and to what extent the CRIES-8 seems to assess the same dimensions of post-traumatic stress, namely intrusion and avoidance. Also, we could evaluate whether the respondents report symptoms similarly between groups, meaning that indicators have similar loadings on their assumed factors. For robustness of the results, we decided to control for two other invariance tests. We investigated whether the time spent in the host country changed the way adolescents responded to the CRIES-8 by testing measurement invariance between recently arrived and settled adolescents. In addition, owing to unexpected country-specific differences in the response labelling options for the CRIES-8 (see Section 3.2), we decided to test the measurement invariance between different response labelling options. Secondly, we estimated reliability for the intrusion and avoidance scales, and for the PTSD total score among the three country groups. Thirdly, we estimated the symptom prevalence among the three groups. Fourthly, we calculated correlations between the CRIES-8 score and the emotional symptoms, such as depression and anxiety, and the hyperactivity symptoms, such as problems concentrating. These correlations are reported as indicators of criterion validity for the CRIES-8.

3. Methods

3.1. Data

In this study, we use baseline data from the RefugeesWellSchool (RWS) study, conducted in 2019. The study included six European countries: Belgium, Denmark, Finland, Norway, Sweden, and the UK. This study involves data from Belgium, Denmark, Finland, Norway, and Sweden, since the UK sample for the three groups had fewer than five cases. The aim of the RWS was to investigate the effectiveness of psychosocial interventions for refugee and migrant young people in secondary schools. Self-reported survey data were collected in schools. The data set for the present study included responses from a total of 148 refugee adolescents from Afghanistan, 234 from Syria, and 175 from Somalia. The three groups were the largest country of origin samples in the study. Since the host country samples were too small for multigroup analysis (total sample size of the three groups in Belgium n = 218, Denmark n = 117, Finland n = 51, Norway n = 72, Sweden n = 99), data from the five countries were merged. Participation was voluntary, and surveys were translated and back-translated so that participants could consent and respond in their mother tongue. When available, official translations for the questionnaires were used. For instance, the CRIES-8 is available in Arabic, Farsi, and Kurdish/Sorani, and the Strengths and Difficulties Questionnaire (SDQ) is available in Arabic, Farsi, Kurdish, Pashto, and Urdu. Participants completed the survey on paper (Belgium, Denmark) or online, using LimeSurvey (Denmark, Finland, Norway, Sweden) (Schmitz, 2012). When needed, interpreters or teachers aided with filling in the questionnaire. For detailed information on recruitment and data collection, see Spaas et al. (2022).

The data analysed in this study are subject to the following licences/restrictions. The participants provided their informed consent to allow only certain researchers to use the data. This was based on the requirements for data usage of the ethical boards of the participating organizations. Requests concerning data should be directed to the corresponding author. The study was not preregistered. All analytical code can be accessed at https://osf.io/zmcfb/.

3.2. Measures

PTSD symptoms were assessed with the CRIES-8. The CRIES-8 has two subscales, namely intrusion and avoidance, and it is designed to be used in children who can read independently, aged 8–18 years. The items are scored on a four-point scale: 0 = not at all, 1 = rarely, 3 = sometimes, 5 = often. Intrusion is the sum of the items ‘Do you think about it even when you don’t mean to?’, ‘Do you have waves of strong feelings about it’, ‘Do pictures about it pop into your mind?’, and ‘Do other things keep making you think about it?’; and avoidance is the sum of the items ‘Do you try to remove it from your memory?’, ‘Do you stay away from reminders of it (e.g. places or situations)?’, ‘Do you try not talk about it?’, and ‘Do you try not to think about it?’. The total score is the sum of scores from the two subscales, ranging between 0 and 40. A total score of 17 or above indicates a likely PTSD diagnosis (Perrin et al., 2005). The instrument is freely available to clinicians and researchers through the website www.childrenandwar.org, and it is available in 28 different languages.

In the RWS study, some of the translations accidentally used slightly different response option labelling. In the data collected in Finland and Denmark, the response scale was 0 = not at all, 1 = sometimes, 3 = often, 5 = always. In Belgium, Norway, and Sweden, the data were collected using the original response labels. This originally unplanned setting offered us a possibility to test measurement invariance between scale response labelling styles, to check whether the different labelling affects the interpretation of items.

The survey included self-reported information on gender, age, country of origin, reason for migration, time in host country, separation from family during migration, and whether the respondents arrived unaccompanied. PTSS were reported using the CRIES-8. Respondents were categorized as refugees based on their responses to reason for migration (fleeing war or persecution) or country of origin (based on European Statistics) if they did not know or respond to the reason for migration. Respondents were categorized as ‘settled’ if they had spent 2 years or more in the host country.

To assess criterion validity, we used the emotional problems and hyperactivity subscales from the SDQ (Goodman, 2001). These subscales contain five statements with three response options: not true, somewhat true, or certainly true. The emotional problems subscale contains statements such as ‘I worry a lot’ (worries) and ‘I am often unhappy, downhearted, or tearful’ (unhappy). The hyperactivity subscale contains statements such as ‘I am restless, I cannot stay for long’ (restless) and ‘I am constantly fidgeting or squirming’ (fidgety).

3.3. Analytical strategy

The statistical analyses were conducted with R software (R Core Team, 2021), using the R packages lavaan (Rosseel, 2012) and semTools (Jorgensen et al., 2022). Using CFA, we examined the dimensionality and measurement invariance of the two subscales called intrusion and avoidance, and the total scale in the CRIES-8. CFA models were estimated using the diagonal weighted least squares (DWLS) estimator, suitable for ordered variables. Since the missing observations are in the so-called response variables and we did not have suitable auxiliary variables for predicting missingness, only complete data were used in the analysis.

In CFA, a sample size between 100 and 200 is small but still acceptable when the population size is restricted and the number of parameters is small (Kline, 2016). The downside of a small sample size is that the χ²-test statistic does not necessarily follow the χ²-distribution and may overreject models that fit well (Bentler & Yuan, 1999; Herzog & Boomsma, 2009; McNeish, 2017). Several correcting functions have been developed, with Swain’s (1975) method showing the best results in terms of most power to detect whether the model fits poorly (Herzog & Boomsma, 2009; McNeish, 2017; McNeish & Harring, 2017). Furthermore, as a sensitivity analysis, we provide the comparative analyses using estimators for continuous variables in Table S1 in the Appendix (see supplementary material). The problems encountered in recruiting sufficient samples among refugee populations are well known (Enticott et al., 2017).

Model fit was evaluated using the absolute fit index χ², a parsimony-corrected index called the root mean square error of approximation (RMSEA), and comparative fit indices such as the comparative fit index (CFI) and Tucker–Lewis index (TLI). The χ²-value and RMSEA approach 0 when the model fit is good. CFI and TLI approach or exceed 1 in the case of a well-fitting model. When we refer to commonly used cut-offs in Section 4, we use guidelines from Schreiber et al. (2006). These are p-values of ≤ .01 for χ², ≥ .95 for CFI, ≥ .96 for TLI, ≤ .06 for RMSEA, and ≤ .08 for the standardized root mean square residual (SRMR). The lavaan package produces two types of index: standard and robust. All reported model fit indices refer to robust indices in this study. The Swain correction (Herzog & Boomsma, 2009; McNeish & Harring, 2017; Swain, 1975) was applied as a function, and the model fit estimates were transformed after fitting the models and calculating the original fit estimates. Models were also evaluated on their parameters, including the inspection of residuals.

The measurement invariance was evaluated stepwise as configural, metric/weak, and scalar/strong invariance. Measurement invariance was evaluated using the following criteria: in the weak compared with the configural invariance, the fit should not decrease by more than .01 in CFI, or increase by more than .015 in RMSEA, or increase by more than .03 in SRMR (Chen, 2007). In the strong compared with the weak invariance, the fit should not decrease by more than .01 in CFI, or increase by more than .015 in RMSEA, or increase by more than .01 in SRMR. We decided not to analyse measurement invariance between genders or age groups, or between URMs and accompanied adolescents, because of the small group sizes, and to avoid an overabundance of study aims.

We estimated reliability using α, ordinal α, and ω coefficients for intrusion and avoidance, and higher order ω for PTSD second order factor (the total score). We used the cut-off of ≥ 0.8 to indicate acceptable reliability (Raykov & Marcoulides, 2011). For the commonly used CRIES-8 sum score where the eight items are simply summated, we calculated the α coefficient to enable comparisons with previous studies on the psychometric properties of the CRIES-8. The sum score model was not tested for its construct validity. Therefore, the assumptions of the Cronbach’s α coefficient were not tested, and the α coefficient may be a biased estimate for reliability.

We evaluated the criterion validity of the CRIES-8 by estimating Pearson’s correlations between the CRIES-8 total score and the emotional problems and hyperactivity symptoms indicated by the SDQ (Goodman, 2001). Lastly, we estimated the prevalence of PTSS using the sum scores of the CRIES-8 and the recommended cut-off of 17 or more points to indicate a likely PTSD diagnosis (Perrin et al., 2005). We recoded the CRIES values so that all adolescents had the same scale [0 = not at all (A, B), 1 = rarely (A) or sometimes (B), 3 = sometimes (A) or often (B), 5 = often (A) or always (B)]. Except for the comparisons of different response labelling, we used the original CRIES values.

4. Results

Table 1 presents demographic characteristics of the adolescents by their country of origin. The distribution of the time in the host country was right-skewed, meaning that most adolescents had arrived only a few years ago. The majority (60.3%) of the adolescents from Afghanistan had stayed in the host country for less than 2 years. Of the adolescents from Syria, nearly two-thirds (62.5%) had stayed for 2 years or more. Half of the adolescents from Somalia had stayed in the host country for at least 2 years. The majority (58%) of the adolescents from Afghanistan had been separated from their family during migration, more than the adolescents from Syria (29%) or from Somalia (39%). Finally, almost half (46.2%) of the adolescents from Afghanistan had arrived unaccompanied, whereas for adolescents from Syria and Somalia the proportions were 4.8% and 5.2%, respectively. The level of emotional problems was low among all three groups, but Somali adolescents reported about half as many symptoms as Syrian or Afghan adolescents. The level of hyperactivity was also low among all three groups, but Syrian adolescents reported slightly more symptoms than the other groups.

Table 1. Demographic characteristics of adolescents from Afghanistan, Syria, and Somalia (N = 557).

	Min.	Max.	Afghanistan	Syria	Somalia
Gender, female (%)	0	1	25.6	45	57.1
Age (years)	11	17	15.4 (1.4)	15.4 (1.3)	14.6 (1.6)
Time in country (years)	0	17	1.9 (1.9)	2.4 (1.9)	3.3 (3.2)
Time in country: ≥ 2 years (%)	0	1	39.7	62.5	50
Separated (%)	0	1	58	29	39
Came unaccompanied	0	1	46.2	4.8	5.2
Emotional problems score	0	10	3.95 (2.43)	3.39 (2.37)	1.77 (1.97)
Hyperactivity score	0	10	2.82 (1.85)	3.33 (2.00)	2.28 (1.85)

Note: Data are presented as percentage (%) or mean (SD).

Separated = responded yes to the question: ‘Were you ever separated from family members during your migration to this country?’; Came unaccompanied = responded ‘No one, I came alone’ to the question: ‘Which family member did you come with?’; Emotional problems score = SDQ subscale ‘Emotional problems’ sum score; Hyperactivity score = SDQ subscale ‘Hyperactivity’ sum score.

4.1. Fit of the factor structure by country of origin

Table 2 presents the results for testing the model fit of the factor structure of the CRIES-8 among adolescents from Afghanistan, Syria, and Somalia. We tested the model fit for one-dimensionality of the intrusion and avoidance factors, and the two-factor structure. The intrusion factor was one-dimensional among adolescents from Afghanistan and Somalia. In the Syrian group, the RMSEA indicated problems in one-dimensionality of the intrusion factor, while other estimates evidenced good fit. The avoidance factor did not fit well in any of the groups based on the RMSEA, but CFI and TLI showed excellent fit. The χ²-test was acceptable for the Afghan and Somali groups, but showed significance for the Syrian group. The two-factor structure seemed to have perfect fit with the data among Somalian adolescents. Among Syrians, only the RMSEA upper limit was above the limit, and among Afghan adolescents, the χ²-test was significant and the RMSEA was above the limit, indicating a bad fit to the data. The CFI and TLI indices, which favour a low number of degrees of freedom, showed good fit for all models and all groups. Probably as a result of the small sample sizes, the RMSEA confidence intervals were very wide and the upper limit was not at an acceptable level in any of the models. The factor loadings are reported in the Table S2 in the Appendix (see supplementary material).

Table 2. Factor structure of the 8-item Children’s Revised Impact of Event Scale (CRIES) among adolescents from Afghanistan, Syria, and Somalia: Swain-corrected model fit estimates.

Group	Factor model	Obs.	χ^2	df	p	CFI	TLI	RMSEA	RMSEA CI
Afghanistan	Intrusion	130	0.794	2	.672	1	1.016	0	.001−.133
	Avoidance	131	3.707	2	.157	0.988	0.964	.081	.001−.209
	PTSD 2F	124	3.241	19	.049	0.977	0.966	.069	.005−.114
Syria	Intrusion	201	3.884	2	.143	0.995	0.986	.069	.001−.171
	Avoidance	196	6.638	2	.036	0.991	0.974	.109	.024−.206
	PTSD 2F	187	29.759	19	.055	0.988	0.982	.055	.000−.092
Somalia	Intrusion	165	3.036	2	.219	0.998	0.995	.056	.001−.175
	Avoidance	170	3.948	2	.139	0.997	0.991	.076	.001−.187
	PTSD 2F	164	13.482	19	.813	1	1.005	0	.000−.044

Note: Obs. = observations; df = degrees of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; PTSD 2F = two-factor model of intrusion and avoidance; factors are allowed to correlate.

4.2. Measurement invariance by country of origin, time in host country, and response labels

Table 3 presents the results from measurement invariance tests. We tested measurement invariance between adolescents from Afghanistan, Syria, and Somalia; between recently arrived and settled adolescents; and between different response labels. Between Afghan and Syrian adolescents, even the two-factor structure did not fit well, as the RMSEA was above the limit, the TLI was low, and the χ²-test was significant, indicating bad fit. Between Afghan and Somali adolescents, the two-factor structure fitted well with the data. However, the factor loadings differed, and thus the weak invariance showed bad fit based on the significant χ²-test and the above-the-limit increase in RMSEA. Between Syrian and Somali adolescents, only the χ²-test was significant in the strong invariance test. Other indices indicated that this comparison had strong invariance. The three-way comparison showed an above-the-limit RMSEA value in the configural and weak invariance test, and a significant χ²-test in the weak and strong invariance test, while the other indices indicated good fit even for the strong invariance. This finding gives some support for the group mean comparisons. Again, in the measurement invariance test, the RMSEA upper limit was above the limit in all but one case, possibly because of the small sample sizes.

Table 3. Measurement invariance by country of origin, response labelling style, and time in host country.

Group	vs	vs	Obs.	Invariance	χ^2	df	p	CFI	ΔCFI	TLI	RMSEA	ΔRMSEA	RMSEA CI	SRMR	ΔSRMR
Afghanistan	Syria		124 | 187	Configural	61.512	38	.009	.952		.929	.091		.055−.124	.050
				Weak	61.969	44	.038	.954	.002	.941	.082	−.009	.048−.114	.055	.005
				Strong	96.485	58	.001	.972	.018	.973	.066	−.016	.041−.088	.051	−.004
Afghanistan	Somalia		124 | 164	Configural	45.309	38	.193	.995		.993	.033		.000−.092	.041
				Weak	63.144	44	.031	.988	−.007	.985	.048	.015	.000−.095	.053	.012
				Strong	84.324	58	.014	.988	0	.988	.056	.008	.026−.081	.042	−.011
Syria	Somalia		187 | 164	Configural	43.924	38	.235	.992		.988	.044		.000−.093	.037
				Weak	57.292	44	.086	.987	−.005	.983	.052	.008	.000−.093	.046	.009
				Strong	78.317	58	.039	.992	.005	.992	.045	−.007	.011−.069	.038	−.008
Afghanistan	Syria	Somalia	124 | 187 | 164	Configural	75.412	57	.052	.981		.973	.061		.000−.097	.043
				Weak	98.711	69	.011	.975	−.006	.970	.064	.003	.024−.095	.055	.012
				Strong	146.854	97	.001	.984	.009	.986	.057	−.007	.037−.075	.044	−.011
< 2 years	≥ 2 years		208 | 187	Configural	35.596	38	.581	.999		.998	.018		.000−.069	.027
				Weak	43.093	44	.510	.998	−.001	.997	.021	.003	.000−.066	.033	.006
				Strong	62.829	58	.309	.998	0	.999	.021	0	.000−.050	.027	−.006
Labelling A	Labelling B		341 | 134	Configural	54.33	38	.042	.984		.977	.058		.017−.089	.033
				Weak	58.005	44	.077	.985	.001	.981	.052	−.006	.007−.082	.037	.004
				Strong	139.703	58	.000	.975	−.01	.976	.077	.025	.061−.094	.033	−.004

Note: Obs. = observations; df = degrees of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; < 2 years = arrived in the host country less than 2 years ago; Labelling A = response labels 0 ‘Not at all’, 1 ‘Rarely’, 3 ‘Sometimes’, 5 ‘Often’; Labelling B = response labels 0 ‘Not at all’, 1 ‘Sometimes’, 3 ‘Often’, 5 ‘Always’.

The results were robust considering the time in the host country and labelling style. The response labelling style did not affect the factor loadings of the items (labelling style had weak measurement invariance, except for the significant χ²-test in configural invariance), and the time in the host country did not affect even the item intercepts (time in host country had strong measurement invariance).

4.3. Reliability

Table 4 depicts reliability estimates for intrusion and avoidance and the PTSD total score among adolescents from Afghanistan, Syria, and Somalia. It is important to note that the fit of the two-factor structure with factor loadings fixed to equal size was not tested. Therefore, the assumptions of the Cronbach’s α coefficient are not shown, and the α coefficient may be a biased estimate for reliability. It is reported only to enable comparisons between previous studies, and should not be considered as a valid reliability estimate. Among adolescents from Afghanistan, intrusion and avoidance had low reliability, as judged by α, ordinal α, and ω coefficients. Among adolescents from Syria, intrusion had α values that exceeded the recommendation of 0.8 (Raykov & Marcoulides, 2011). However, the ω coefficient was low, indicating low reliability for intrusion. On avoidance, all coefficients were above the recommendation, showing high reliability. Among the Somalian adolescents, the group had rather high reliability on intrusion and avoidance. The PTSD second order ω coefficients were acceptable for Afghan and Somali adolescents, but unacceptable for Syrian adolescents.

Table 4. Reliability estimates by country of origin.

		α	Ordinal α	ω
Afghanistan	Intrusion	.754	.790	.746
	Avoidance	.706	.766	.717
	PTSD 2nd order			.817
	CRIES-8 sum score	.825
Syria	Intrusion	.800	.825	.786
	Avoidance	.801	.856	.817
	PTSD 2nd order			.686
	CRIES-8 sum score	.826
Somalia	Intrusion	.818	.887	.832
	Avoidance	.819	.885	.831
	PTSD 2nd order			.876
	CRIES-8 sum score	.881

Note: CRIES-8 = 8-item Children’s Revised Impact of Event Scale; PTSD = post-traumatic stress disorder.

4.4. Criterion validity of the CRIES-8

Table 5 presents the Pearson’s correlations between the CRIES-8 sum score and emotional problems, and the CRIES-8 sum score and hyperactivity for Afghan, Somali, and Syrian adolescents. In emotional problems, the CRIES-8 sum score correlated significantly with the worries, unhappy, and fears statements, and the emotional problems score in all groups. Correlations varied between .175 and .418. Among Syrian adolescents, also somatic statement correlated with the CRIES-8 sum score. In hyperactivity, the distractible statement (.183–.188) correlated significantly with the CRIES-8 sum score among Afghan and Syrian adolescents, and the fidgety statement (.210) correlated significantly with the CRIES-8 sum score among Somali adolescents.

Table 5. Pearson’s correlation r between the 8-item Children’s Revised Impact of Event Scale (CRIES-8) sum score and emotional problems, and the CRIES-8 sum score and hyperactivity.

Afghanistan

Somatic

Worries

Unhappy

Clingy

Fears

Emotional problems score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

.022

.794

141

.352

< .001

145

.332

< .001

144

.08

.341

143

.176

.035

144

.303

< .001

139

Restless

Fidgety

Distractible

Reflective

Persistent

Hyperactivity score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

.139

.1

141

.085

.314

142

.183

.03

142

.067

.426

145

−.079

.347

143

.155

.07

138

Syria

Somatic

Worries

Unhappy

Clingy

Fears

Emotional problems score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

.345

< .001

223

.338

< .001

223

.266

< .001

223

.175

.009

223

.291

< .001

224

.418

< .001

221

Restless

Fidgety

Distractible

Reflective

Persistent

Hyperactivity score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

.011

.876

224

.056

.407

224

.188

.005

221

−.04

.553

224

.004

.95

222

.098

.151

217

Somalia

Somatic

Worries

Unhappy

Clingy

Fears

Emotional problems score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

.145

.059

171

.230

.002

172

.183

.016

172

.139

.069

172

.277

< .001

171

.325

< .001

170

Restless

Fidgety

Distractible

Reflective

Persistent

Hyperactivity score

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

r

p

N

CRIES-8 sum score

−.021

.783

171

.210

.006

171

.1

.197

169

.084

.271

172

.012

.872

171

.077

.327

166

4.5. Prevalence of symptoms

Table 6 presents the symptom prevalence estimates with χ² significance tests for the following groups: Afghan, Syrian, and Somalian adolescents; recently arrived and settled adolescents; original response labelling and alternative labelling; and URMs and those adolescents who arrived accompanied. The three country groups, different response labelling style, and URM status showed statistically significant differences. More than half of the Afghan adolescents were above the cut-off, which indicates a likely PTSD. The respondents in the original response labelling style group showed higher prevalence than those in the alternative response labelling style group. Since the scoring was unchanged, the alternative response labelling style is more conservative and requires symptoms with higher frequency to achieve the total score of the original response labelling style. This may reflect that the response labelling does not have an impact on the tendency to report symptoms.

Table 6. Prevalence estimates for post-traumatic stress symptoms assessed with the 8-item Children’s Revised Impact of Event Scale (CRIES-8) in different groups.

	Below cut-off		Above cut-off		χ^2	p
	n	%	n	%
Afghanistan	65	44.5	81	55.5	11.345	.003
Syria	131	57.2	98	42.8
Somalia	109	63	64	37
< 2 years	115	53	102	47	1.198	.274
≥ 2 years	136	58.1	98	41.9
Labelling A	146	52	135	48	4.7405	.029
Labelling B	105	62.5	63	37.5
URM	31	36.5	54	63.5	15.184	< .001
Accompanied	270	59.3	185	40.7

Note: ≤ 2 years = arrived in the host country less than 2 years ago; Labelling A = response labels 0 ‘Not at all’, 1 ‘Rarely’, 3 ‘Sometimes’, 5 ‘Often’; Labelling B = response labels 0 ‘Not at all’, 1 ‘Sometimes’, 3 ‘Often’, 5 ‘Always’; URM = unaccompanied refugee minors; accompanied = accompanied refugee minors; Below cut-off = total score < 17 on CRIES-8; Above cut-off = total score ≥ 17.

5. Discussion

This study investigated the psychometric properties of the CRIES-8 among adolescents from Afghanistan, Syria, and Somalia. We tested the model fit of the factor structures and measurement invariance between the country groups. We evaluated the robustness of the results by testing whether the measurement model was invariant between recently arrived and settled adolescents, and between different response labelling styles. In addition, we estimated reliability for the subscales and the PTSD total score, and we evaluated the criterion validity of the CRIES-8 by estimating correlations between the CRIES-8 sum score and emotional problems and hyperactivity indicators. Finally, we estimated the prevalence of symptoms. This study supported the CRIES-8 as a suitable assessment tool for Afghan, Syrian, and Somali adolescents and for comparing group means between some of the groups, although reliability estimates varied from low to high. The results also showed significant differences in the symptom prevalence.

In general, the results showed better model fit for the intrusion scale than for the avoidance scale, and reasonably good fit for the two-factor structure. This is partly in line with previous findings about problems with the avoidance factor (Rasmussen et al., 2014). On the other hand, the poor fit of the avoidance factor was mainly based on RMSEA, which penalizes for model simplicity, small sample size, and high factor loadings (McNeish et al., 2018). The fit indices which favour a low number of degrees of freedom indicated that the model fitted well with the data. This ambiguous finding requires further investigation with a much bigger sample size. Similarly, the good fit of the two-factor structure supports findings from previous studies that PTSD symptomatology has globally cross-cultural validity (Hinton & Lewis-Fernández, 2011; Michalopoulos et al., 2020). The variation in indicators responded to the hypothesized model of two latent variables, and the indicators were conditionally independent. Nevertheless, the good fit is not a proof of a true model (Kline, 2016).

The results for the measurement invariance were ambiguous. Comparisons between Afghan and Syrian adolescents were not supported, as the groups did not even seem to have a similar factor structure. This finding could motivate future studies to focus more on qualitative approaches to understand how these groups may have different understanding of the items. However, between Syrian and Somali adolescents, the mean comparisons were mostly supported. When controlling for the possible variance between recently arrived and settled adolescents, and different response labelling styles, the results were robust. Both comparisons were reasonably measurement invariant.

In general, the CRIES-8 seemed to have low reliability among Afghan and Syrian adolescents, whereas among Somali adolescents, reliability was high. Somali adolescents seem to have responded to symptoms of intrusion and avoidance more consistently than Afghan and Syrian adolescents. Somali adolescents also had the lowest prevalence of symptoms, whereas more than half of the Afghani adolescents had high levels of symptoms, suggesting a likely PTSD. However, the high levels of symptoms may be due to the high proportion of URMs among these adolescents, and not to their Afghan background as such. Especially among URMs, there was a high prevalence, which is in line with previous research (Bean et al., 2007). Yet again, there is previous evidence that (adult) Somali refugees may underreport symptoms, whereas Middle-Eastern refugees may overreport symptoms (Jakobsen et al., 2011). Afghan adolescents differed from Syrian and Somali adolescents in their male majority, shorter time in the host country, high proportion of URMs, low subscale reliability, and high proportion of a likely PTSD diagnosis. However, because the reliability estimates were low among Afghan adolescents, the prevalence estimates should be interpreted with a high degree of caution, as the risk of misclassification is greatly increased (Charter & Feldt, 2001).

The CRIES-8 sum score had medium-sized correlations with emotional problems, such as the worries, unhappy, and fears statements, and low correlations with hyperactivity, such as the distractible and fidgety statements. These associations show that the CRIES-8 sum score is related to emotional problems and hyperactivity but still distinct from them. The associations also indicate some comorbidity of PTSS and emotional problems among adolescents.

Among other revisions, the changes to the diagnostic criteria from the DSM-IV to DSM-5 include three new symptoms added to the PTSD criteria in the DSM-5: persistent negative emotional state, persistent distorted cognitions about the cause or consequences of the trauma, and reckless or self-destructive behaviour. The addition of the last symptom (reckless or self-destructive behaviour) has been criticized (see, for example, Pai et al., 2017, for a summary), but it is true that in its current form the CRIES-8 does not fulfil these new criteria. In that sense, we recommend adding at least one question reflecting these new symptom categories, together with the hyperarousal item (as present in CRIES-13). Further research is needed to decide which of the questions should be added. It is, however, notable that the new diagnosis of complex post-traumatic stress disorder (C-PTSD) overlaps these symptoms and has an emphasis on guilt and shame, as well as other cognitive and social changes in the child’s life after trauma. In future, it will be important to develop measures that can discriminate between PTSD and C-PTSD in children.

5.1. Strengths and limitations

The CRIES-8 is a short screening tool for easy and widespread use. The CRIES-8 does not capture the breadth of DSM-5 or ICD-11 PTSD diagnostic criteria, but is a first measure for screening to detect symptoms of post-traumatic stress. It cannot replace a valid diagnosis by a specialist in adolescent psychiatry. This study focused on evaluating the psychometric properties of a mental health screening tool in a sample of a hard-to-reach population. Evidently, the sample size was small, and it is hard to say how well the country samples represented their populations. The data were collected in schools, and all those who were going to these schools were given an opportunity to participate, although participation was voluntary. Therefore, it is possible that more disadvantaged adolescents may have not participated and thus were not included in the analyses. Afghanistan, Syria, and Somalia were some of the top countries of origin for refugees during the 2010s. A strength of this study is that, for the first time, the country groups were compared for their measurement invariance. This provided preliminary evidence on the validity of group mean comparisons.

Author contributions

RK, with the support of KP and MV, conceived of the presented research idea and hypotheses. RK formulated the initial drafts of the manuscript and revised them in collaboration with KP, MV, MO, CS, ID, SSJ, MS, ND, and FO. RK, KP, MV, MO, CS, SSJ, MS, ND, FO, AA, PKH, SA, and RL contributed towards collecting the empirical data. RK conducted all data preparation and statistical analyses. All of the authors contributed to this study, and read and approved the final manuscript.

Acknowledgements

We are grateful to all the adolescents who participated in this study, and to the teachers and other school authorities who helped us with the data collection in schools.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability

The data analysed in this study are subject to the following licences/restrictions. The participants provided their informed consent to allow only certain researchers to use the data. This was based on the requirements for data usage of the ethical boards of the participating organizations. Requests concerning data should be directed to the corresponding author.

Additional information

Funding

This study is part of the Refugees WellSchool (RWS) study Horizon 2020 research project [grant number 754849 (https://cordis.europa.eu/project/id/754849)].