Diagnostic usefulness of the Alcohol Use Disorders Identification Test (AUDIT) questionnaire for the detection of hazardous drinking and dependence on alcohol among Spanish patients

Abstract

Objective: To check the validity of the Alcohol Use Disorders Identification Test (AUDIT) among Spanish adult citizens. Methods: This is a descriptive observational study. The surveyed group comprised patients aged 18–80 years who went to their doctors’ surgeries at two primary care centres located in Cordoba (Spain). We examined the psychometric properties of AUDIT and its capacity to correctly diagnose alcohol abuse or dependence, as defined by DSM-IV, ICD-10, and hazardous drinking. Results: Six hundred and fourteen patients were studied (mean age 43±1.43 years). At a cut-off value of 7 points, the sensitivity of AUDIT in detecting hazardous drinking was 91.7%, and its specificity 91.9%; the area below the curve was 0.95 (95% confidence interval [CI] 0.937–0.975). To detect possible dependence, the optimum cut-off value was 6 points. According to ICD-10, sensitivity was 81.6%, specificity 82.3%, and the area under the curve 0.885 (95% CI 0.848–0.923; p<0.001), whereas according to DSM-IV criteria, sensitivity was 88.3%, specificity 83.1%, and the area under the curve 0.918 (95% CI 0.885–0.951).

Conclusion: The high criterion-related validity of AUDIT was proven, regardless of the gold standard used.

• AUDIT test
• alcoholism
• screening

Introduction

According to the World Health Organization (WHO) 2002 report on world health [1], alcohol consumption is the third most important risk factor in developed countries. Spain is sixth in the world ranking of alcohol intake per capita [2]. The US Preventive Services Task Force as well as other international organizations recommends screening and health advice to reduce alcohol abuse among adults [3]. There are several screening tools, among which the WHO recommends the Alcohol Use Disorders Identification Test (AUDIT) questionnaire [4], due to its numerous advantages, such as its being the only test specifically designed to be used internationally and its being short, effective, and well suited for primary healthcare personnel. This tool was developed after an extensive validation study had been carried out in six countries [5], [6]; also, in recent decades, additional studies have been performed to assess its precision and accuracy in different contexts and populations [7–12], including among adult Spanish citizens. Nevertheless, the WHO itself suggests that, in order for this validation process to be more successful, AUDIT should be used to answer some as yet unresolved research questions. One of these research questions concerns identifying the sensitivity, specificity, and predictive strength of this test depending on the different validation criteria used [4].

The aim of our study was to check the criterion-related validity of AUDIT according to different criteria of reference (hazardous drinking, ICD-10, and DSM-IV), and to set the most suitable cut-off value for the detection of hazardous drinking and alcohol dependence.

Methods

We designed an observational descriptive study for the assessment of a diagnostic test on a sample of adults living in the province of Cordoba (Spain) who, for whatever health problem, attended two primary healthcare centres (one urban, the other rural), one county centre for drug addicts (rural), and one association for the rehabilitation of alcoholics (in an urban setting).

Participants were selected through consecutive sampling among patients ranging from 18 to 80 years of age who gave their informed consent to take part in the study. Those who were excluded either had cognitive disorders that could prevent them from answering the questionnaire adequately or had been previously diagnosed as having alcoholism (new cases). Fieldwork was carried out between June 2005 and May 2006. This study was approved by the Ethics and Clinical Investigation Committee of the Queen Sofia University Hospital of Cordoba (Spain). To calculate the sample size, we used the formula used for estimating a proportion in infinite populations and the C4-Study Design Pack program (version 1.1, GlaxoSmithKline). We estimated that the required sample should consist of 603 patients, considering an AUDIT specificity of 83% (expected proportion), based on a value obtained in a previous study [8], an alpha error of 5% (confidence level of 95%), and a precision of±3%.

Personal interviews were conducted prior to handing out the questionnaires. Four family doctors took part, all of whom had been trained previously, so that they carried out the interviews with the greatest possible homogeneity. After that, final diagnosis was made by another family doctor who was an expert in alcoholism issues and who was blinded to the results of the AUDIT.

We obtained a series of socio-demographic data concerning the study participants, quantified their weekly alcohol consumption, and gave them the AUDIT [5], a questionnaire comprising 10 questions about their alcohol consumption in the previous year. In this study, we have focused on the Spanish version of the questionnaire, as validated by Rubio et al. in 1998 [7]. This version includes three questions on the consumption of alcoholic drinks (amount, frequency), four questions related to dependence, and, finally, three other questions aimed at the analysis of its consequences. The questions score from 0 to 4 points, except questions 9 and 10, which can score 0, 2, or 4 points. The scoring ranges from 0 to 40 points. A total score equal to or higher than 8 points is recommended as an indicator of hazardous drinking and harmful consumption as well as possible alcohol dependence [5].

The diagnostic criteria used as a reference standard (gold standard) were based on the Schedule for Clinical Assessment in Neuropsychiatry (SCAN) interview. This standardized interview consists of a battery of instruments developed by the WHO to evaluate, measure, and classify the psychological problems and behaviours associated with the main psychiatric disorders of adults [13]. SCAN is a semi-structured standard clinical interview, validated in Spain for the diagnosis of alcohol abuse and dependence [14], whose information allows one to make diagnoses according to DSM-IV criteria (alcohol abuse, dependency) and the ICD-10 criteria of the WHO (harmful consumption, dependency). Harmful alcohol consumption is an alcohol intake that provokes health problems [15]. Hazardous drinking is defined as consumption behaviour that increases the risk of suffering any physical, psychological, or social damage in the future. The threshold for the level of risk is arbitrary, but in Spain hazardous drinking is currently considered to be a weekly alcohol intake of over 280 g (28 or more standard drink units [SDU]) for men and 170 g (17 SDU or more) for women. These are the criteria proposed by the Spanish Preventive Activities and Health Promotion Programme (Programa de Actividades Preventivas y de Promoción de la Salud [PAPPS]), which is an important reference for many providers of primary care in Spain [16]. One SDU amounts to 10 g of pure ethanol [5], [16–18]. Final diagnosis was made by a specialist who worked independently to assess the results obtained from the SCAN interview.

To assess criterion-related validity, we calculated the sensitivity and specificity indices as well as the area under the receiver operating characteristic (ROC) curve by using hazardous drinking criteria and those provided by DSM-IV and ICD-10 as gold standards. The best cut-off value was the one that maximized the sum of sensitivity and specificity. For the selected cut-off value, we also calculated positive and negative predictive values and the likelihood ratio. Finally, to measure the convergent validity of AUDIT with the MALT questionnaire, we used the Pearson correlation coefficient. The MALT survey was designed in Germany by Feuerlain to detect alcoholic patients [19]. The Validation process for the Spanish population was carried out in 1984 [20] We consider relevant to know the convergence between this survey and the AUDIT. All this was done with the help of the SPSS (version 12.0) and EPIDAT (version 3.1) statistical programs.

Results

Of the 614 subjects studied, 59.3% were recruited at the primary healthcare centres and 40.7% at the centres for drug dependence. Their average age was 43±1.43 (range 18–80) years; 67.4% were women (Table I), most of whom were married (69%) and had primary education (40.6%); 61.1% lived in rural areas; and 63.2% consumed alcoholic drinks. Their average alcohol consumption was 34.2±2.4 SDU/week (95% confidence interval [CI] 29.48–38.92, range 1–350): 26.2% (95% CI 22.7–29.7%) were hazardous drinkers, 3.1% showed alcohol abuse, and 17.4% dependence, according to the DSM-IV criteria; according to ICD-10, 4.7% showed harmful consumption and 17.4% alcoholism.

Table I.  Socio-demographic characteristics of the studied population.

Characteristics	N	%
Age, years
≤ 24	55	9.0
25–34	145	23.7
35–44	159	26.0
45–54	100	16.3
55–64	104	17.0
65 +	49	8.0
Total	614	100.0
Gender
Men	200	32.6
Women	414	67.4
Total	614	100.0
Marital status
Married	429	69.8
Single	114	18.5
Divorced	36	5.8
Widower/widow	35	5.7
Total	614	100.0
Residence
Urban	239	38.9
Rural	375	61.1
Total	614	100.0
Level of education
Illiterate	74	12.1
Primary	249	40.6
Secondary	117	19.1
Upper secondary	106	17.3
University	68	11.1
Total	614	100.0

AUDIT produced an average score of 5.6±7.6 (95% CI 5.0–6.2, median 3, range 0–40). The area under the ROC curve (Figure 1) for hazardous drinkers was 0.95 (95% CI 0.937–0.975; p<0.001), whereas for alcohol dependence the area was 0.918 (95% CI 0.885–0.951; p<0.001) when using DSM-IV as the gold standard (Figure 2) and 0.885 (95% CI = 0.848-0.923; p<0,001) when using ICD-10 (Figure 3). Table II shows the different cut-off values of sensitivity and specificity according to the criteria used. As can be seen, the most efficient cut-off values (the point at which the sum of sensitivity and specificity reaches the maximum score) are 6–7 points for excessive drinkers and 6 points for alcohol-related disorders. Table III shows the results of the AUDIT indices of criterion-related validity in accordance with the reference criteria used.

Figure 1.  Diagnostic performance of the AUDIT as to the reference criteria hazardous drinker.

Figure 2.  Diagnostic performance of the AUDIT as to the reference criteria DSM-IV.

Figure 3.  Diagnostic performance of the AUDIT as to the reference criteria ICD-10.

Table II.  Sensitivity and specificity values of AUDIT for different cut-off scores, according to different gold standards.

	Hazardous drinker, %		According to DSM-IV, %		According to ICD-10, %
Cut-off value	Sensitivity	Specificity	Sensitivity	Specificity	Sensitivity	Specificity
4	98.6	72.8	94.5	66.8	91.2	66.7
5	93.8	82.2	93.6	76.7	86.8	75.9
6	92.4	89.9	88.1	83.1	81.6	82.4
7	91.7	91.9	85.3	84.5	78.1	83.5
8	84.8	94.4	78.0	87.3	74.6	87.1

Table III.  Indices of criteria validity of AUDIT based on the reference criteria used and most efficient cut-off values.

	Hazardous drinker		DSM-IV	ICD-10
Parameters	Six points	Seven points	Six points	Six points
Sensitivity	92.4% (87.7–97.1)	91.7% (86.9–96.5)	88.1% (81.5–94.6)	81.6% (74.0–89.1)
Specificity	89.9% (87.1–92.8)	91.9% (89.3–94.5)	83.1% (79.8–86.5)	82.4% (79.0–85.8)
Global efficacy	90.5% (88.2–92.9)	91.9% (89.6–94.1)	84.0% (81.1–87.0)	82.2% (79.1–85.3)
Positive predictive value	74.0% (67.4–80.7)	77.8% (71.2–84.3)	53.0% (45.5–60.6)	51.4% (43.8–58.9)
Negative predictive value	97.5% (95.9–99.1)	97.3% (95.7–98.9)	97.0% (95.3–98.7)	95.1% (93.0–97.3)
Likelihood ratio of positive test	9.22 (7.07–12.17)	11.32 (8.31–15.41)	5.23 (4.26–6.43)	4.64 (3.76–5.71)
Likelihood ratio of negative test	0.08 (0.05–0.15)	0.09 (0.05–0.15)	0.14 (0.09–0.24)	0.22 (0.15–0.33)

Figures in parentheses represent 95% confidence intervals.

Discussion

To date, most studies that have assessed AUDIT have used different reference criteria in isolation. Some have considered the consumption of a particular amount of alcohol, while others have used the presence of alcohol-related disorders. Yet, few studies like our own have borne in mind different reference criteria at the same time. This allows us to compare and check—in one and the same sample—the usefulness of AUDIT for the different aims it was created for. The results of this study not only reveal that AUDIT is highly valid for the detection of alcohol abuse and dependence (the likelihood ratios show very good values), but also that it is even more useful in identifying hazardous drinking (91.9% of global efficacy for a cut-off value of 7 points).

On the whole, studies aimed at detecting hazardous drinking are barely comparable, since the criteria used to consider a person as a hazardous drinker vary considerably and the results depend on the time when the study was carried out, the definition of drinking units by the authors or authorities in individual countries, and the cut-off value used. For example, while in the United Kingdom an SDU amounts to 8 g of pure ethanol, in the USA it is 14 g and in Spain 10 g—it was 8 g until recently. In AUDIT, in questions 2 and 3, it is assumed that a standard drink unit also equals 10 g [4], [16]. Table IV shows the results of the validity rates of the studies with which we have compared our data. The original study carried out by the WHO was designed to detect a consumption of 60 g/week in men and 40 g/week in women. A similar sensitivity (92%) and a higher specificity (94%) were obtained compared to our study, which is logical since a higher reference value was used [5]. A systematic review of Medline for works published between 1966 and 1998 on the role of AUDIT in detecting excessive consumption (considered positive from 8 points on) showed a sensitivity of 57–59% and a specificity of 91–96% [21]. In two studies that were performed in primary healthcare settings in Spain, the results were slightly lower for sensitivity (81.4% and 89.1%) and very close to ours as regards specificity (91.1% and 94.6%) [11], [22]. In one other similar study that was carried out in three French-speaking areas, the results concerning detection of hazardous drinkers were lower than ours (AUDIT ≥7 for males: sensitivity 83.5%, specificity 79.9%; AUDIT ≥6 for females: sensitivity 81.2%, specificity 93.7%) [23].

Table IV.  Statistical values from studies concerning accuracy of AUDIT.

		At-risk, harmful, or hazardous drinking, %
Studies	Cut-off score	Sensitivity	Specificity
Saunders et al., 1993 [5]	≥8	92.0	94.0
	≥10	80.0	98.0
Rubio et al., 1998 [7]	≥9 for females	82.0	90.0
	≥6 for females	80.0	87.0
Bradley et al., 1998 [26]	≥8	57.0	96.0
Fiellin et al., 2000 [28]*	≥8	51.0–97.0	78.0–96.0
Gache et al., 2005 [23]	≥7 for males	83.2	79.9
	≥6 for females	81.2	93.7
Gómez et al., 2005 [22]	≥8	81.4	94.6
Berner et al., 2007 [27]*	≥8	31.0–89.0	83.0–96.0

*Systematic review.

Studies that are carried out in different settings—primary healthcare centres as well as hospitals—to detect alcohol-related disorders obtain highly variable results, as their sensitivities range from 32% to 96% and their specificities somewhere between 84% and 96% [5], [7–10], [12], [22–27]. Improvements have been made in terms of detection capacity by increasing or diminishing the cut-off value according to the population and the objective of the screening programme. In the samples of the development of the test [5], Saunders et al. suggested two cut-off values, obtaining a sensitivity of 92% and a specificity of 94% when the cut-off value was at 8 points, and a sensitivity of 80% and specificity of 98% when this was set to 10 points. In the validation study performed by Rubio et al. in Spain [7], the most efficient cut-off values were 6 for women (sensitivity 80%, specificity 87%) and 9 for men (sensitivity 82%, specificity 90%). We have not found any differences in the optimum cut-off value between genders, and propose a cut-off value of 6 for both. A systematic review of the literature has come to the conclusion that AUDIT is the best screening instrument for all alcohol-related problems in primary healthcare, in contrast to other questionnaires such as CAGE and MAST [28]. However, it must be taken into account that MAST was designed only for screening patients suffering from alcohol dependence.

One of the limitations of our study is that its conclusions cannot be fully extrapolated to the target population (patients at primary healthcare surgeries). The reason for this is that, in order to increase the number of patients that were likely to be diagnosed as having alcoholism and therefore obtain more robust estimators, we had to recruit people who were being seen at centres specialized in drug dependence; and, naturally, the prevalence of alcohol-related disorders is higher in such centres. However, although this undoubtedly affects the predictive values obtained with AUDIT, in no way does it affect the sensitivity and specificity indices, which are intrinsic properties of the measurement tool. Therefore, these indices should not vary greatly depending on where the questionnaire is administered, provided it is done so under similar conditions [29]. When performing a study to validate measurement tools, it is recommended that the sample should include a wide clinical range of participants, from patients who do not drink a single drop of alcohol to patients with a high level of dependence, so as not to overestimate the validity of the study [29]. Furthermore, we used personal interviews in both settings and prior training of the doctors involved in order to reach criteria uniformity among those in charge. These professionals were family doctors, which undoubtedly helped to establish an environment of trust and sincerity within the framework of a good doctor–patient relationship. In addition, in order to try to avoid information bias—since interviewers might be influenced by the answers given in AUDIT and therefore overestimate its validity—the final diagnosis was made a posteriori by another doctor who was specialized in alcoholism. This physician was blinded to the AUDIT results.

In short, this study shows that the AUDIT questionnaire is a highly effective and useful instrument for the detection of hazardous drinking and, although the results are poorer, for the diagnosis of alcohol-related disorders. It has been confirmed that the best cut-off value for our population is 6–7 points for the detection of hazardous drinking and 6 points for the detection of abuse and alcohol dependence. Despite the high number of studies concerning AUDIT, further research is needed in different contexts (especially in less developed countries) [5] and in other population groups (teenagers, old people) in order to obtain a clearer picture of its psychometric properties, such as its construct validity, convergent validity, or sensitivity to the change of test.

Acknowledgements

This project received funding from the Spanish Society for Family and Community Medicine, the Andalusian Society for Family and Community Medicine, and the Health Department of the Autonomous Andalusian Government (Spain). Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.