Validity and reliability of the Turkish version of the knowledge about childhood autism among health workers questionnaire

ABSTRACT

Objective: The aim of this study is to present the psychometric properties of the Turkish version of the knowledge about childhood autism among health workers (KCAHW) questionnaire.

Methods: Three hundred fifteen primary health care facility workers and 28 child and adolescent mental health professionals were enrolled in this study. Participants filled out socio-demographic data forms and the KCAHW. Reliability analyses consisted of internal consistency and test–retest reliability. For validity analysis, construct validity (confirmatory factor analysis -CFA) and criterion validity were used.

Results: The mean KCAHW questionnaire score was 13.83 ± 2.55. The floor effects in all domains were below 15%, the ceiling effects were over 15% in overall score and in Domain 4. Intraclass correlation coefficient and the Kuder Richardson 21 values were 0.83 and 0.683, respectively; All goodness of fit indices generated by CFA were found satisfactory (Comparative fit index = 0.79; Root mean square error of approximation = 0.056, and chi-square/degree of freedom = 1.91). Being a physician, being a CAMH specialist, having mental health clinic experience, having done a child psychiatry internship, knowing someone diagnosed with autism, follow-up experience of a patient with autism, having previous autism training, and the perception of adequate knowledge about autism, significantly increased the KCAHW scores (p < 0.001).

Conclusion: The Turkish version of the KCAHW questionnaire is reliable (in terms of test-retest and internal consistency) and valid (sensitive some known/expected external criteria). Due to the insufficient internal consistency in Domain 4, the scores received from Domain 4 should be evaluated with caution.

KEYWORDS:

• Autism Spectrum Disorder
• reliability and validity
• knowledge
• health worker
• primary care physicians
• general practitioners

Introduction

Autism Spectrum Disorder (ASD) is one of the most frequently seen childhood neurodevelopmental disorders. While the frequency of ASD in the 1990s ranged from 1 in 2500 children to 1 in 1000, it was stated to be 1 in every 110 children in 2006 [1,2]. According to April 2018 data from the U.S. Center for Disease Control and Prevention (CDC), the frequency of ASD has been reported to be 1 in 59 children [3]. Possible reasons for this increase in frequency have included changes in the ASD diagnosis criteria, expansion of the scope of the spectrum, subjects diagnosed with retardation can now be diagnosed with ASD, an increase in the awareness of society and health workers about ASD, and patients being able to reach health institutions in an easier manner [4,5]. Early diagnosis and early treatment are factors which allow for a positive course for ASD [6,7].

The first step of guiding a child who has ASD symptoms toward a diagnosis is to suspect ASD. Therefore, health workers who have contact with children need know about ASD. It is reported in the literature that although ASD symptoms appear before 2 years of age, the average age of ASD diagnosis is 4–6 years [8,9]. In Erden et al.’s study, it is stated that only 11.2% of the 125 subjects diagnosed with ASD were recognized by paediatricians [10]. In light of these data, it is apparent that the knowledge of health workers about autism is insufficient.

The Knowledge about Childhood Autism among Health Workers questionnaire (KCAHW) was developed by Bakare et al. to evaluate the ASD knowledge of health workers, particularly in countries where this knowledge is limited [11]. This questionnaire has been used in various studies to evaluate the knowledge of health workers about autism [12–14]. For example, Harrison et al. published a review that included studies using this questionnaire, concluding that although there was no strong evidence to support the global usability of the questionnaire, it was nonetheless usable in evaluating knowledge and educational needs about ASD [15]. In Turkey, studies have been conducted on autism knowledge and awareness across different groups of subjects, including college of education students, family medicine residents, preschool educators, and parents [16–20]. However, in none of these studies, a valid and reliable measurement tool used. There is no validity and reliability scale in Turkish that evaluates the level of knowledge of autism in health care workers. The aim of this study is to present the psychometric properties of the Turkish version of the knowledge about childhood autism among health workers (KCAHW) questionnaire.

Methods

Ethical considerations

Ethical board approval was received from the Muğla Sıtkı Koçman University’s Human Researches Ethical Board (180086/72) and the Muğla Provincial Directorate of Health. We complied with the principles of the Declaration of Helsinki in conducting the study. Permission was obtained from the developer of the scale (M. O. Bakare) to adapt and validate the KCAHW into Turkish. We obtained written consent from all volunteers participating in the study.

Subjects

The study sample consisted of a total of 315 physicians and family health workers (i.e. midwives, nurses) who worked at the family health centres in the Muğla city centre and were involved in the autism training programme in Muğla. Additionally, Children and adolescent mental health (CAMH) specialists (n = 28) were recruited from the Muğla and the other regional provinces (Aydın and Izmir). Sample size was calculated based on the 10:1 case to parameter ratio, that means a sample size at least 190 subjects for the 19 item KCAHW. Additionally, adequacy of the sample size for factor analysis was confirmed with the Kaiser–Meyer–Olkin (KMO) (>0.5) together with Bartlett’s test of sphericity. Child and adolescent mental health specialists (n = 28) were included in the study as a reference group.

The unique inclusion criterion for the voluntary subjects was given as currently working in a primary health care facility, whereas the exclusion criteria were determined as giving incomplete answers to the questions (n = 4). Any item answered with more than one choice (n = 3) was considered as missing and replaced with the mean.

Data collection tools

All subjects were self administered the KCAHW questionnaire in addition to the socio-demographic data form, while another 10 lay subjects were underwent cognitive debriefing interviews for the KCAHW questionnaire in between May 2018 and August 2018.

KCAHW questionnaire

As developed by Bakare et al., the KCAHW questionnaire consists of 19 items across four domains of autism [11]. The first domain consists of eight items and concerns the impairment in social interaction observed in children with autism. The second domain consists of a single item, the symptom related to communication and language development. The third domain consists of four items and indicates the obsessive and compulsive, repeating, and stereotypical symptoms observed in autism. The fourth domain consists of six items and questions whether or not autism is a neurodevelopmental disorder, examines possible comorbid conditions, and explores the ages at which it emerges. The possible total score that can be received from the questionnaire is between 0 and 19. Each item is answered as “yes”, “no”, or “I don’t know”. Correct answers receive 1 point, and the other answers, 0. The last item questions the onset age of autism and is scored as 0 for neonatal age or infancy, and as 1 for childhood. On average, it takes 10 min to complete the KCAHW questionnaire.

Socio-demographic/work experience questionnaire

In the study, socio-demographic variables of the health professionals were asked such as gender, education, income, marital status, number of children, and type of medical profession and specialty as well as work-related variables such as work duration at a mental health hospital, internship experience in child psychiatry during their medical education, having a child diagnosed with autism or with a chronic disease, work experience in following-up with patients with autism, and their perceived knowledge about autism.

Stages procedure

Cultural/language adaptation

Turkish forward and backward translations

The linguistic adaptation process followed internationally accepted procedures [21–24]. The KCAHW questionnaire was translated into Turkish by two native academicians with a very good level of English proficiency who were independent of each other. The translations were then evaluated by a third native academician with an advanced level of English proficiency, who prepared a consensus version of the scale. The consensus version was then back translated into English by a bilingual expert and approved by the developer.

Cognitive debriefing of the Turkish KCAHW questionnaire (pilot application)

Ten health professionals (six general practitioners and four nurses) underwent cognitive debriefing interviews independently of each other. Each of the respondents were asked to give feedback for each of the instruction/item/response choices: “what does it mean for him/her; whether he/she has difficulty understanding or not; are there words that he/she finds difficult to understand and, if so, how would he/she change the wording?”

The items were rewritten where necessary, in line with the changes suggested by the participants and agreed upon. The items (such as items 16 and 17 of Domain 4) considered problematic in the interviews were further discussed with the developer of the original KCAHW questionnaire and revised accordingly. The final “field trial Turkish version” was prepared following the cognitive interviews and back-translated to English by a bilingual expert; the backward translation report was sent to and approved by the developer of the scale.

Psychometric analysis of the field trial Turkish version

The Turkish field version was administered to the 343 participants. Confirmatory approach was used in the psychometric analyses.

Distribution characteristics of the Turkish KCAHW and reliability analyses

Prior to reliability analysis, the scale’s distribution characteristics and measurement skill were assessed. The distribution characteristic was evaluated through skewness and kurtosis analyses, whereas the measurement capacity of each of the domains was evaluated through floor and ceiling effects. The limit values were accepted as 1.0 for skewness, 2.0 for kurtosis, and 15% for the floor and ceiling effects [25,26].

Reliability of the scale was tested with the test–retest reliability and internal consistency approaches. The test–retest reliability of the scale was assessed by giving the questionnaire to the 22 primary health care workers. The interval between test–retest assessments was 15 days. Intraclass correlation coefficient (ICC) was used to assess the test-retest results; an ICC closer to 1.0 indicates better consistency.

Internal consistency was assessed by using two methods: Kuder–Richardson 21 (KR-21) coefficient and scale success. Kuder–Richardson shows the internal consistency of the dichotomous items in a scale and identically gives the same coefficient as Cronbach’s alpha. Scale success, on the other hand, is a method based on item–domain correlations and refers to producing a greater and meaningful correlation coefficient with its own domain compared to its correlation to the other domains. So, scale success refers to the percentage of the number of items producing a greater and meaningful correlation coefficient with its own domain divided by the number of all items.

Additionally, “if item deleted/removed” internal consistency coefficients were calculated for each of the items of the scale. An increase in the value of the internal consistency coefficient following an item removal shows that the removed item has low contribution to the variability of the domain and may be considered a problematic item. Similarly, if the item-domain correlation coefficient is lower than 0.30, this item might be problematic item as well.

Validity analyses

In validity analysis, construct validity and criterion validity methods were used.

Construct validity. Exploratory (EFA) and Confirmatory factor analysis (CFA) was used for the assessment of construct validity. Exploratory factor analysis was done by Principal Components Analysis for informative purpose such as testing the sample size adequacy and for item analyses. KMO values over 0.5, refers to sample size adequacy. Confirmatory factor analysis is based on structural equation modelling and has been used to show to what extent the original domain structure of the scale was preserved in the Turkish version. We used comparative fit index (CFI), root mean square error of approximation (RMSEA), and chi-square/degree of freedom (ChiSq/df) among the fit statistics generated by CFA. Root mean square error of approximation is a parsimony-adjusted index with a cut-off good fit smaller than 0.08 and CFI compares the fit of a target model to the fit of an independent model and cut-off god fit for CFI is suggested as ≥0.90; The limit which should be accepted for CFA is at least 0.9. Since chi-square alone is sensitive to sample size, chi-square/degree of freedom (ChiSq/df), referred to as relative/normed chi-square, is used as another fit criterion in this paper [27]. A ChiSq/df value smaller than 2.0 indicates a good fit. The adequacy of sample size was evaluated with the KMO method; a KMO value over 0.5 was accepted as sufficient [28].

Criterion validity. Criterion validity analysis compares the test with the criterion already held to be valid. For criterion validity, several hypotheses were tested in this study and follow below.

Scale points were higher in

• CAMH specialists (n = 28) compared to general practitioners,

• physicians compared to other health workers (nurses, midwives),

• those who had work experience in a mental health hospital compared to those who did not,

• those who did their internship in child psychiatry compared to those who did not,

• those who have a child diagnosed with autism or a chronic disease compared to those who do not,

• those who have received in-service training on autism compared to those who have not,

• those whose perception of professional knowledge about autism is adequate compared to those with poor perception of knowledge,

• those who have a person diagnosed with autism in their immediate environment compared to those who do not, and

• those who did follow-up of a patient diagnosed with ASD compared to those who did not.

Statistical hypothesis tests used in these comparisons

Subject data were evaluated using Windows SPSS 17.0 software (SPSS Inc., Chicago, IL, USA) and Lisrel v. 8.05. Normality distributions were evaluated using the Shapiro–Wilk test. Student’s t-test is used to determine if there is a significant difference between the means of two groups when parametric conditions are satisfied, and Mann–Whitney U test when parametric conditions are not satisfied. One-way analysis of variance (ANOVA) test is employed in the comparison of more than the means of two groups in parametric conditions, and Kruskal–Wallis ANOVA in non-parametric conditions. Tukey B test was applied in case of statistically significant ANOVA results for post-hoc group comparisons. The correlation analysis of two numerical variables was evaluated with the Spearman’s rho. The predefined type 1 error level was accepted as less than 0.05 in the analyses.

Results

Results on descriptive variables and the distribution characteristics of the scale

The socio-demographic data of the 343 health workers who participated in the study are given in Table 1. The mean age of the participants was 44.96 ± 8.85 years. The mean duration of working in the health sector was 21.78 ± 8.81 years for the respondents. Nine family physicians had a history of working at a mental health clinic. The mean KCAHW questionnaire score the participants who worked in the primary health care facilities was 13.83 ± 2.55. The mean scores for Domains 1, 2, 3, and 4 were 7.07 ± 1.21, 0.89 ± 0.30, 2.69 ± 1.02, and 3.16 ± 1.30, respectively.

Table 1. The sociodemographic characteristics of the participants.

	n (%)
Gender	Female	196 (57.1)
	Male	147 (42.9)
Occupation	Physician	201 (58.6)
	Nurse	40 (11.7)
	Midwife	100 (29.2)
	Other	2 (0.6)
Specialty	General practitioner	159 (46.4)
	Family physician specialist	14 (4.1)
	CAMH specialist	28 (8.2)
Marital status	Single/lives alone	35 (10.2)
	Married/has a partner	291 (84.8)
	Separated/widow(er)	16 (4.7)
Education level	High-school	17 (5.0)
	Undergraduate	60 (17.5)
	Graduate	70 (20.4)
	Postgraduate	153 (44.6)
	Specialty in medicine/doctorate degree	43 (12.5)
Income/expense level	Income much lower than the expenses	40 (11.7)
	Income a little lower than the expenses	65 (19.0)
	Equal income and expenses	131 (38.2)
	Income a little higher than the expenses	94 (27.4)
	Income much higher than the expenses	8 (2.3)
Number of children	0	43 (12.5)
	1	89 (25.9)
	2	187 (54.5)
	3	23 (6.7)
Perception of sufficient level of knowledge	Quite insufficient	31 (9.0)
	Sort of insufficient	124 (36.2)
	Neither sufficient nor insufficient	110 (32.1)
	Sort of sufficient	53 (15.5)
	Quite sufficient	20 (5.8)

The floor-ceiling effects and skewness-kurtosis values of the domains of the scale are presented in Table 2. While the floor effect in all domains was below 15%, the ceiling effect was over 15% in overall scale score and in Domain 4 score. Skewness and kurtosis were particularly high in the Domain 2.

Table 2. The floor-ceiling, skewness-kurtosis values of the domains of KCAHW^† questionnaire.

Scale domains	Floor effect %	Ceiling effect %	Skewness	Kurtosis
Domain 1	1.5	53.6	1.545	1.810
Domain 2^‡	9.6	90.4	2.751	5.599
Domain 3	2.9	29.2	0.619	0.140
Domain 4	1.7	6.4	0.022	0.374
Total	0.3	3.2	0.379	0.320

^†KCAHW: Knowledge about childhood autism among health workers questionnaire

^‡Domain 2 consists of only one item

Reliability

Scale reliability analyses are presented as a whole in Table 3. The scale success was 100%. Each item had a higher correlation coefficient with its own domain compared with the other domains. The item-domain correlation ranged between 0.25 and 1.00. In Domain 1, the item-domain correlation value of the first item was below 0.3. The domain–overall score correlations were determined as 0.717 for Domain 1; 0.356 for Domain 2; 0.723 for Domain 3; and 0.726 for Domain 4.

Table 3. Item-domain correlations, internal consistency^† and scale success^‡.

Items	Domain 1 r (KR-21 when the item is deleted)	Domain 2 r ( KR-21 when the item is deleted)	Domain 3 r ( KR-21 when the item is deleted)	Domain 4 r ( KR-21 when the item is deleted)
Item 1	0.25 (0.61)	−0.06	0.10	0.00
Item 2	0.30 (0.60)	−0.01	0.12	−0.02
Item 3	0.66 (0.52)	0.15	0.27	0.19
Item 4	0.63 (0.53)	0.16	0.20	0.12
Item 5	0.39 (0.59)	0.13	0.07	0.01
Item 6	0.49 (0.58)	0.12	0.18	0.09
Item 7	0.48 (0.57)	0.14	0.16	0.13
Item 8	0.64 (0.55)	0.22	0.33	0.13
Item 9	0.25	1.00 (na)	0.16	0.14
Item 10	0.08	0.10	0.49 (0.48)	0.13
Item 11	0.30	0.26	0.69 (0.43)	0.25
Item 12	0.30	0.04	0.57 (0.46)	0.12
Item 13	0.27	0.01	0.72 (0.39)	0.22
Item 14	0.02	−0.02	0.06	0.47 (0.35)
Item 15	0.17	0.08	0.13	0.54 (0.30)
Item 16	0.08	0.11	0.19	0.35 (0.41)
Item 17	0.20	0.19	0.28	0.58 (0.27)
Item 18	0.17	0.16	0.34	0.63 (0.21)
Item 19	0.00	−0.07	−0.21	0.30 (0.48)
KR-21 (Overall)	0.61	(na)	0.52	0.39
Domain-Overall scale score	0.72 (na)	0.35 (na)	0.72 (na)	0.73 (na)

na: not applicable.

^†Measured with Kuder–Richardson 21.

^‡A percentage of the items in one domain gives a high correlation coefficient with their domain, not with the other domains.

Kuder–Richardson 21 values were 0.61 for the Domain 1; 0.52 for Domain 3; 0.39 for Domain 4, and 0.68 for the whole scale. Internal consistency for the domain 4 is under the acceptable limits. Based on the comparisons of the “if item deleted KR-21 values” and the KR-21 values of each of the domains, the only domain that may have problematic items was the Domain 4. The overall KR-21 values were greater than the “if item deleted KR-21 values” indicating that all of the items are contributing to the variance of the domains 1 and 3. When the 16th and 19th questions were removed, the KR-21 values increases to 0.41 for item 16 and 0.48 for the item 19. Since there was only one question in Domain 2, the KR-21 value was not calculated.

For the test–retest analysis, 22 participants out of 315 primary health care workers in the study were given the questionnaire again 2 weeks later without any interference (training). While the mean test score was 14.81 ± 1.81, the mean retest score was 15.04 ± 2.33 (p > 0.05). Test-retest ICC value for the overall scale was 0.83 (95% CI = 0.60–0.93).

Validity

Construct validity

In the Principal Components Analysis, KMO value was 0.699 and the Bartlett’s test of sphericity was significant (p < 0.001). Total variance explained for the EFA is 63.7% and the communalities of each of the items are higher than 0.40.

The goodness of fit findings of the CFA are presented in Table 4 and Figure 1. Comparative Fit Index (CFI) was determined as 0.79; RMSEA as 0.056 (95% CI: 0.046–0.066), and ChiSq/df as 1.91. The fit indexes were within acceptable limits except of the CFI. When the items are analysed one by one, item 19 in Domain 4 is problematic in its CFA consistent with the results of the reliability analysis. When the 19th item was recalculated assuming “infancy” and “childhood” answers were both accepted as correct answers, the improvement observed in internal consistency and in CFA indices. The covariance value for item 19 increased from 0.049 to 0.091.

Figure 1. Confirmatory factor analysis for KCAHW questionnaire.

Table 4. Results of confirmatory factor analysis.

Fit indices	By conventional scoring of item 19	By revised scoring of item 19
CFI^†	0.79	0.81
RMSEA^‡	0.056	0.060
Chi square / df^§	2.42	1.91
Covariance between item 19-Domain 4	−0.049	0.091

^†CFI = Comparative Fit Index.

^‡RMSEA = Root mean square error of approximation.

^§Degree of freedom.

Criterion validity

The results related to the variables used in the testing of criterion validity (sensitivity) are given in Table 5. When the total scores of the physicians were compared in terms of their specialty, it was observed that the lowest score was received by general practitioners and the highest score by the CAMH specialists (one-way ANOVA, df = 2, F = 35.03, p < 0.05; Tukey B test). There was a weak correlation between number of working years and KCAHW questionnaire total scores (r = –0.213). As the perception of sufficiency in knowledge on autism increases, the total scores received from the scale also increases (df = 4, F = 20.35, p < 0.001). In our study, KCAHW questionnaire total score was not significantly affected by the health workers’ gender (p = 0.95), working years (r = –0.213), or having a child with chronic diseases/handicaps (p = 0.358) or autism (n = 7; p = 0.87). There was a difference between Domain 1, Domain 4, and the overall scores of the scale in terms of occupation groups. It was seen that the differences were significant between physicians–nurses for Domain 1; physicians–midwives for Domain 4, and both physicians–midwives and physicians–nurses (one-way ANOVA, df = 2, F = 9.75, p < 0.001; Tukey B test).

Table 5. Known groups and criterion validity results of KCAHW^† questionnaire.

Known groups/Criterion variables	Domain 1		Domain 2		Domain 3		Domain 4		KCAHW total score
	Mean ± sd	p	Mean ± sd	p	Mean ± sd	p	Mean ± sd	p	Mean ± sd	p
Gender*
Female	7.09 ± 1.24	0.309	0.92 ± 0.26	0.154	2.82 ± 1.02	0.559	3.32 ± 1.32	0.991	14.15 ± 2.71	0.950
Male	7.22 ± 1.12		0.87 ± 0.32		2.75 ± 1.05		3.31 ± 1.46		14.17 ± 2.72
Follow-up experience of ASD*
Yes	7.56 ± 0.85	<0.001	0.92 ± 0.26	>0.05	3.23 ± 0.88	<0.001	3.85 ± 1.38	<0.001	15.58 ± 2.55	<0.001
No	7.03 ± 1.25		0.89 ± 0.30		2.67 ± 1.04		3.15 ± 1.35		13.75 ± 2.64
CAMH internship*
Yes	7.56 ± 0.85	<0.001	0.92 ± 0.26	>0.05	3.23 ± 0.88	<0.001	3.85 ± 1.38	<0.001	15.58 ± 2.55	<0.001
No	7.03 ± 1.25		0.89 ± 0.30		2.67 ± 1.04		3.15 ± 1.35		13.75 ± 2.64
Work experience in mental health area/hospital*
Yes	7.69 ± 0.75	<0.001	0.98 ± 0.13	<0.001	3.48 ± 0.75	<0.001	4.26 ± 1.25	<0.001	16.42 ± 2.26	<0.001
No	7.05 ± 1.23		0.88 ± 0.31		2.66 ± 1.03		3.13 ± 1.33		13.74 ± 2.58
Got any training on ASD*
Yes	7.62 ± 0.78	<0.001	0.90 ± 0.30	>0.05	3.10 ± 0.88	<0.001	3.88 ± 1.44	<0.001	15.51 ± 2.55	<0.001
No	6.97 ± 1.28		0.90 ± 0.29		2.67 ± 1.07		3.09 ± 1.30		13.64 ± 2.63
Having children with ASD**
Yes	7.57 ± 0.53	0.531	0.85 ± 0.90	0.684	2.42 ± 0.97	0.271	3.57 ± 1.61	0.529	14.42 ± 2.07	0.870
No	7.14 ± 1.2		0.37 ± 0.29		2.79 ± 1.04		3.3 ± 1.38		14.14 ± 2.79
Have any children with a chronic diseases/handicapped**
Yes	6.78 ± 1.2	0.041	0.78 ± 0.42	0.045	2.56 ± 1.07	0.271	3.43 ± 1.23	0.570	13.56 ± 2.96	0.358
No	7.17 ± 1.19		0.91 ± 0.28		2.8 ± 1.03		3.29 ± 1.39		14.18 ± 2.7
Any patients with ASD in the immediate environment**
Yes	7.33 ± 1.1	0.037	0.95 ± 0.2	0.017	3.05 ± 0.97	0.001	3.46 ± 1.36	0.138	14.8 ± 2.6	0.001
No	7.06 ± 1.22		0.87 ± 0.32		2.65 ± 1.04		3.23 ± 1.38		13.83 ± 2.69

^†KCAHW: Knowledge about childhood autism among health workers questionnaire.

*Independent groups t test, **Mann Whitney U test.

Discussion

In this study, we aimed to investigate the validity and reliability of the Turkish version of the KCAHW questionnaire developed by Bakare et al. In the psychometric evaluations of the questionnaire, we used the validity methods consisting of the distribution characteristics of the items, reliability, and validity approaches. The main findings of this study were: (1) the floor effect in all domains was below 15%, the ceiling effect was over 15% in total and in Domain 4; ICC = 0.83; KR-21 = 0.683; CFA indicators were found as CFI = 0.79, RMSEA = 0.056, and ChiSq/df = 1.91. Fit indices other than CFI were within acceptable limits. The scale was found to be sensitive to various variables according to criterion validity.

Although ASD is a frequently seen neurodevelopmental disorder in society, studies show that health workers have a limited amount of knowledge about ASD. Raising the awareness and knowledge of health workers about autism may contribute to the diagnosis of children with autism at earlier ages. Therefore, it is important to measure objectively the level of knowledge of the health worker. Bakare et al. have determined the four domains of KCAHW as Domains 1, 2, 3, and 4 [11]. In our study, we made some suggestions for naming these domains, and we believe that designating Domain 1 as “social interaction”, Domain 2 as “communication”, Domain 3 as “behavioural pattern”, and Domain 4 as “characteristics of disorder” embodies the concepts represented by the domains. In addition, these name suggestions were made in parallel with the explanations of the domains used in studies involving this questionnaire [12,13]. The reason we did not add “impairment” to the names of the domains was to avoid creating bias in the people filling out the questionnaire.

In the original article in which the questionnaire was developed, the mean score of knowledge of the nurses who work at community mental health services was determined as 9.6 ± 5.25 [11]. Bakare et al. [12] have reported the mean score of paediatric and psychiatry nurses as 12.35, Igwe et al. [29] reported the mean score of paediatric nurses as 11.78 and psychiatry nurses as 13.35, and Sampson et al. [30] reported the average score of paediatric nurses as 11.37 and psychiatry nurses as 12.11. In our study, the mean score of the KCAHW questionnaire of 315 participants who work at primary health care facilities was 13.83 ± 2.55. The mean score of KCAHW being high in our study compared with that of other studies might be related to the fact that a majority of our participants were physicians. In our study, the mean score for Domain 1 was determined as 7.07; for Domain 2, 0.89; for Domain 3, 2.69; and for Domain 4, 3.16. In other studies, the average score for Domain 1 ranged between 5.24 and 6.71; for Domain 2, between 0.63 and 0.94; for Domain 3, between 1.1 and 2.62; and for Domain 4, between 1.3 and 3.53 [12–14,29–31]. According to these results, we conclude that the score obtained from Domains 1 and 3 increased the total mean score in our study.

When the floor-ceiling effects of the scale were analysed, it was determined that the ceiling effect was over 20% in Domains 1 and 3. It was considered that floor-ceiling effect was high in Domain 2 because there was only one item in Domain 2. In line with this, the skewness and kurtosis values being high show that the answers given to these questions do not display normal distribution. This may due to participants’ lack of understanding of the questions. Therefore, we determined that because of the high ceiling effects of the measurement skill in Domains 1 and 3 may not be sufficient and that the scores obtained from these domains should be evaluated in a more careful manner.

The item-domain correlation was found to be between 0.25 and 1.00. Although the correlation value was below 0.3 in the question in the first item of Domain 1 (“Marked impairment in use of multiple non-verbal behaviours such as eye to eye contact, facial expression, body postures and gestures during social interaction?”), the question’s correlation with its own domain is higher compared with that of the other domains. According to the scale’s item-domain correlation results, it can be stated that the scale’s success is 100% since each item has a high correlation with its own domain. These values show that each item of the scale is in harmony with its own domain and the whole of the scale. On the other hand, since there were no data related to the item-domain correlation values of the scale in the original article, a comparison could not be made. However, it was determined that the results of the obtained domain-total score correlation analysis were similar to the results of the original article [11]. Bakare et al. applied the questionnaire to the 50 psychiatry nurses in their study again 2 weeks later and analysed the test–retest correlation of the questionnaire [11]. They have concluded that both the domain and total score test–retest correlation values had a highly positive correlation level [11]. In the findings obtained from the test–retest results of our study, the scale’s ICC value was determined to be good, and we therefore consider the test–retest results to be reliable.

As for internal consistency, KR-21 value was 0.68 and was considered to be sufficient. In the study by Bakare et al., the internal consistency was determined as 0.97 [11]. Although the internal consistency result obtained in our study is not at a desired level, it is considered to be a more reasonable value compared to the result in the original article. The internal consistency results of the questionnaire’s domains were also analysed in our study; however, the KR-21 value was found to be low in Domain 4. The possible reason for this might be negative contributions of the 16th and 19th items in this domain on internal consistency. When the answers “infancy” and “childhood” were accepted as correct answers in the question in the 19th item “Onset of Autism is usually in … ” and the internal consistency analysis was performed again, the KR-21 value increased a little and reached 0.48. In the study by Eseigbe et al. in which the KCAHW questionnaire was used to measure the knowledge of 175 physicians on autism, it was reported that insufficient knowledge was seen the most in the question about autism onset and the mental retardation or epilepsy accompanying autism in Domain 4; in the question about social smiling in Domain 1 and in the question about abnormal eating habits in Domain 3. The authors determined that the highest scores received from the question about impairment in the area of communication and language development was in Domain 2 [13].

In the validity analysis of the scale, CFA and criterion validity were used. According to the results of CFA, it was determined that ChiSq/df and RMSEA values were within acceptable limits. However, it was determined that the CFI value (0.79) was not over the suggested value of 0.90 (Table 4). When the calculation of the question related to the onset of autism in item 19 was changed, it was seen that the covariance value of this item increased. According to these results, due to item 19 lowering the KR-21 value as well being found problematic in confirmatory factor analysis and achieving partial improvement in internal consistency after the calculation method of that item was changed, item 19 is considered as being problematic.

In our study, according to the results of criterion validity used in validity analysis, being a physician, being a CAMH specialist, having worked at a mental health clinic before, having completed an internship in child psychiatry, having a person who has autism in the immediate surroundings, having followed-up on, a patient with autism, having received training on autism, and the perception of sufficient knowledge about autism significantly increased the knowledge scores received from the scale (p < 0.001). Eseigbe et al. have reported that, while the KCAHW questionnaire scores were high for paediatricians or psychiatrists working at tertiary health care institutions, it was significantly lower for general practitioners. However, they have also determined that the knowledge is significantly higher in the male gender, in those who are familiar with autism cases, in those who work in tertiary health care institutions, and in experts of internal branches [13]. Bakare et al. have reported that the most insufficient knowledge was seen in Domain 3, in the group involving people aged 40 years and over, whereas those who followed-up patients with ASD had a high level of knowledge in their study in which they have used the questionnaire and had a sample consisting of paediatric and psychiatry nurses. They have determined that mean scores received by psychiatry and paediatric nurses from the questionnaire were close to each other and high [12]. It was determined that senior year medicine students who have attended to the evaluation process of patients with ASD have received higher scores in KCAHW questionnaire Domains 1, 2, 3, and 4 compared to students who have not attended this process, and that there was a statistically significant difference between the groups for each domain [31].

Children and adolescent mental health specialists, who constitute the reference group, have received the highest score from the scale, followed by family physician specialists and general practitioners. In addition, the physicians had a higher level of knowledge compared to midwives and nurses. These data can also indicate the sensitivity reflected by the scale concerning knowledge. In a study using the KCAHW questionnaire, it was reported that the knowledge of psychiatry nurses in total scores and in all of the domains were higher compared to paediatric nurses [29]. In addition, there are other studies which showed that the questionnaire scores of both paediatric and psychiatry nurses who have followed-up children diagnosed with autism were significantly higher compared to those nurses who have not [29,30]. Health workers’ gender, years of work experience and having a child with chronic diseases/handicaps or autism did not make any significant change in the scale scores. In the literature, there are studies which have reported, similar to the results of our study, that there is no difference between age, gender, working years, and questionnaire scores [29,30]. Igwe et al. showed in their study that the highest level of knowledge was found in medicine students, followed by nursing and psychology students. Between the groups, the difference between Domains 1, 3, and 4 and total score was significant and was found to be the highest in medicine students. They also reported a positive correlation between the scores received from the questionnaire and the time spent in psychiatry and paediatric internship [14].

The main limitation of this study is the composition of the study sample, in which child psychiatrists are a minority in comparison to both nurses and general practitioners. General practitioners and nurses are rarely faced with autism disorder in their professional practice since Turkey has yet to establish a compulsory referral system, and this absence of a referral system has encouraged people to routinely by-pass primary health care to seek services at specialists [32]. Thus, some poor results for the construct validity of this study may be attributable to the minority of child psychiatrists in the study sample. We are encouraged, however, by the success of the criterion analysis based on the comparison of child psychiatrists and others.

According to the results of the analysis in our study, we can conclude that, the Turkish version of the KCAHW questionnaire is a reliable (in terms of test–retest and internal consistency) and valid (sensitive to external criteria) tool for use in Turkish health professionals. We think that this questionnaire can be used to measure health workers’ knowledge on autism and objectively evaluate the results of interventions for increasing their level of knowledge. However, due to the insufficient internal consistency in Domain 4 (KR-21 = 0.39), the scores received from Domain 4 should be evaluated carefully. In addition, there is a need to put forward more findings with further studies related to item 19, which was found to be problematic. We suggest to name the four domains of KCAHW as: “social interaction-D1”, “communication-D2”, “behavioural pattern-D3”, and “characteristics of disorder-D4” respectively, because we believe that these designations (instead of “Domains 1, 2, 3, and 4”) could be of benefit to future studies which use this questionnaire.

Authors contributions

Conception and design of the study: B.G.Ö., H.A., E.E. Acquisition and analysis of data: B.G.Ö., E.E. Drafting the manuscript: B.G.Ö., H.A., E.E. Writing the article: B.G.Ö. All authors contributed to the critical revision of the manuscript.

Acknowledgements

Authors would like to thank to Hande Aytaç, Ogün Özkaya and Ceyda Öztabak for data collection and all participants (physicians, child and adolescent mental health specialists, midwifes and nurses). The authors have no funding to report.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Börte Gürbüz Özgür http://orcid.org/0000-0002-9176-7359

Hatice Aksu http://orcid.org/0000-0001-6882-1129

Erhan Eser http://orcid.org/0000-0002-2514-0056