ABSTRACT
The new A-B-C-D Global Initiative for Chronic Obstructive Lung Disease (GOLD) classification of severity of chronic obstructive pulmonary disease (COPD) is based on combined symptoms and exacerbation risk assessment. The assumed equivalence between dyspnoea modified Medical Research Council (mMRC) grade ≥2 and COPD Assessment Test (CAT) score ≥ 10 to identify more symptoms has been questioned. Whether the exacerbation risk assessment criteria, old GOLD spirometry staging and frequency of exacerbations, are equivalent has not been examined. We evaluated the extent of agreement between these alternative criteria and whether it improved by redefining the equivalence between mMRC grade and CAT score. CAT scores, mMRC grades of dyspnoea, frequency of exacerbations and spirometry stages were computed in 400 patients with COPD. Receiver operating characteristic curve was analysed to determine the best CAT score to identify more symptoms. CAT scores across mMRC grades and the frequency of exacerbations across spirometry stages showed substantial overlaps. The symptoms criteria gave discordant classification in 88 (22%) patients (kappa 0.62) and the exacerbation risk assessment criteria in 181 (45%) patients (kappa 0.12). A CAT score of ≥10 had 82% sensitivity but 24% specificity to identify mMRC grade ≥ 2, while a score of 17 had 98% specificity but a low sensitivity of 52% and did not improve the agreement. We conclude that symptoms and exacerbation risk assessment criteria of the new GOLD classification yield discordant group categorisations. Lack of any satisfactory equivalence between CAT score and mMRC grades implies that the former cannot be used alone. Using the higher of mMRC ≥ 2 and CAT score ≥ 17 to identify more symptoms would avoid discordant categorisation.
Introduction
Systemic inflammation and extrapulmonary involvement, and co-morbidities compound the pulmonary manifestations to determine the overall morbidity and mortality due to chronic obstructive pulmonary disease (COPD) Citation(1–3). Acute exacerbations of COPD hasten the decline in lung function and worsen the quality of life besides posing immediate mortality risk Citation(4–7). Traditionally, the severity of COPD has been assessed using the forced expiratory volume in 1 second (FEV1). The 2006 Global Initiative for Chronic Obstructive Lung Disease (GOLD) strategy document for COPD classified the severity into four stages based on FEV1 Citation(8), a practice also adopted by other guidelines Citation(9,10).
Although associated with mortality Citation(11,12), FEV1 has only modest relationship with several patient-centred outcomes such as exercise capacity, dyspnoea, health-related quality of life, or the frequency of exacerbations Citation(13–15). Therefore, the GOLD 2011 revision introduced a multidimensional assessment scheme for COPD based on evaluation of symptoms and risk of exacerbation. This classification is consensus based and lacks supporting evidence in literature. It yields four groups, A to D, and serves as a guide for the initial treatment of the patient. For symptoms, it recommends using the modified Medical Research Council (mMRC) grading of dyspnoea Citation(16) or, alternatively, the more comprehensive COPD Assessment Test (CAT) Citation(17). It assumes an equivalence between mMRC grade ≥ 2 and a CAT score ≥ 10 to categorise patients into more symptoms category Citation(18). This equivalence has however been questioned as group assignment by these criteria has substantial discordance Citation(19–23). For grading the exacerbation risk as high or low, the higher of the assignment by GOLD spirometry stages or the frequency of exacerbations/hospitalisations in the past 12 months has been proposed Citation(18). The discordance in group assignments by these two criteria has not been examined yet.
Discordance in classification using these different symptoms and exacerbation risk criteria is a negative attribute as it creates confusion, affects management and may even act as a barrier to wider application of the guidelines. Therefore, we addressed the agreement between the different criteria for symptoms and exacerbation risk assessment, and whether redefining the equivalence between mMRC grade and CAT score removed the discordance.
Methods
Study subjects and design
The study was carried out in the outpatient setting after approval by the Institutional Ethics Committee. We included four hundred consecutive stable patients with COPD, aged >40 years, with a history of smoking (>10 pack-years) or biomass fuel exposure (>10 years), presenting with exertional dyspnoea and productive cough and with a post-bronchodilator FEV1/FVC (forced expiratory volume in 1 second/forced vital capacity) ratio below 0.70, in the study Citation(18). Those with a current or recent acute exacerbation, other concurrent acute illness, and an inability to comprehend the study questionnaires or perform spirometry were excluded. Written informed consent was obtained from the patients. Following a detailed history including dyspnoea assessment by mMRC scale Citation(16) and the frequency of exacerbations and hospitalisations in the preceding 1 year, the CAT questionnaire Citation(17) was administered. The patients then underwent spirometry.
Study tools
The mMRC is an ordinal scale of five grades (0–4), reflecting limitation of activity due to breathlessness Citation(16). The CAT questionnaire measures the health status of COPD patients Citation(17). It has eight statements related to symptoms, sleep, confidence and exercise limitation, and the patients rate their response on a scale of 0–5 (best to worst) for each statement. The total score range is 0–40. Patients were classified into less and more symptoms categories based on mMRC grade 0–1 or ≥2, and a CAT score <10 and ≥10, respectively Citation(18). Based on the frequency of acute exacerbations in the previous year, the patients were categorised into low exacerbation risk (0–1) or high exacerbation risk (≥2). A history of ≥1 hospitalisations for COPD exacerbations in the last 12 months also qualified for high-risk category Citation(24). Exacerbation risk assessment by using severity of airflow limitation labelled GOLD stages 1–2 as low risk and stages 3–4 as high risk. Spirometry was carried out using a standardised protocol Citation(25) on a dry rolling-seal spirometer (Benchmark lung function equipment, P.K. Morgan, Kent, the United Kingdom). Prediction equations of Chhabra et al. for the local population were used Citation(26).
The patients were assigned to groups A to D by the combination of symptoms and exacerbation risk assessment Citation(18). The classification obtained using the mMRC grading or the CAT scores was labelled as ‘GOLD 2011 mMRC’ and ‘GOLD 2011 CAT’, respectively. For exacerbation risk assessment, the higher of the assignment by GOLD stage or frequency of exacerbations was used as recommended by the GOLD strategy document. Similarly, classifications obtained with the two exacerbation risk assessment criteria were labelled as ‘GOLD 2011 Stage’ and ‘GOLD 2011 Exacerbations’, respectively. For this categorisation, symptom assessment was done using the higher of the assignment by mMRC or CAT scores.
Statistical analysis
Statistical analysis was carried out using SPSS 20.0 (IBM Corporation, New York, USA) and GraphPad Prism 6.05 (GraphPad Inc., California, USA) software. Data are presented as mean ± SD with 95% confidence intervals (CIs). The frequency distribution of the patients in four groups by GOLD 2011 mMRC and GOLD 2011 CAT, and by GOLD 2011 Stage and GOLD 2011 Exacerbations classifications was compared using the chi square test, and the kappa coefficient was used for evaluating the extent of agreement Citation(27). Quantitative data across multiple groups were compared using one-way analysis of variance (ANOVA) with post hoc Tukey's multiple comparison test to identify significantly different groups. If normality of distribution of data and homoscedasticity of variance were not confirmed by Bartkett's test, the nonparametric Kruskal–Wallis test was used followed by post hoc Dunn's multiple comparison test.
Receiver operating characteristic (ROC) curves were analysed to determine the optimal CAT score for identifying mMRC cut-off of ≥2. The area under curve (AUC) was computed with 95% CIs. For each CAT score, the sensitivity, specificity and the likelihood ratio were determined. A conventional value of p < 0.05 (two-tailed) was used for statistical significance.
Results
shows the characteristics of the study population. All the men (n = 330) were smokers, of whom 95 were current smokers (29%); and of the women (n = 70), 41 were smokers (59%), of whom 7 were current smokers (17%) and 29 had biomass fuel exposure (41%).
Table 1. Characteristics of the study population.
CAT scores across the mMRC grades of dyspnoea revealed significant differences (Kruskal–Wallis test, p < 0.0001, ). Post hoc paired comparisons among different grades showed significant differences (p < 0.0001) except between grades 1 and 2 where the scores were, respectively, 11.82 ± 2.71, 95% CI 11.01–12.64 and 12.64 ± 4.64, 95% CI 11.86–13.42, (p > 0.05). Fifty-three (13.2%) patients had CAT scores<10. The proportion of patients with low (< 10) and high (≥10) CAT scores among different mMRC groups showed significant differences (, p < 0.0001). Low CAT scores were observed in 43 (30.9%) of grade 2 patients but only in 5 (3.2%) mMRC grade-3 patients and none of grade-4 patients.
Figure 1. Scatter of CAT scores with mean ± SD in patients with different mMRC grades of dyspnoea (Kruskal–Wallis analysis of variance, p < 0.0001). For post hoc paired comparisons, see text.
Figure 2. Proportion of patients with low (<10) and high (≥10) CAT scores among groups by mMRC grades (chi square test, p < 0.0001).
The assignment to groups using the combined assessment recommended by the GOLD strategy document with both the GOLD mMRC and CAT criteria for symptoms is shown in . Both the classifications showed group D to be dominant followed by group B. However, there were significant differences in the categorisation by the two criteria with 88 (22%) patients being classified differently (p < 0.0001). The kappa coefficient of agreement was 0.61, indicating a substantial but less than perfect agreement Citation(27). Shifts occurred among patients in both groups when reclassified with the other criteria. Of the 116 patients in group B and 36 in group A by GOLD 2011 CAT, 25 (21.6%) were classified under group A and 33 (91.7%) under group B, respectively, by GOLD 2011 mMRC. Similarly, of the 231 patients in group D by GOLD 2011 CAT, 14 (6%) moved to group C under GOLD 2011 mMRC.
Table 2. Distribution of patients using mMRC or CAT scores.
provides the results of the ROC analyses, and the curve is shown in . The AUC was 0.76 (95% CI 0.71–0.81, p < 0.001), indicating that the CAT score had fairly good discriminating power for mMRC grade ≥2. The cut-off of 10 for the CAT score had a sensitivity of approximately 82% but a specificity of only 24% to identify patients with mMRC ≥ 2. At a CAT score of 16.5, the sensitivity decreased to 54.9 but with marked improvement in specificity to 97.8, while at 17.5, these were 52.1 and 97.8, respectively. Thus, a score of 17 and above would yield the optimal combination of sensitivity and specificity to identify mMRC grade ≥2.
Table 3. Receiver operator characteristic analysis results.
Figure 3. Receiver operator characteristic curve of different cut-offs of CAT score to identify patients with mMRC ≥ 2. The optimal combination of sensitivity and specificity was obtained with a CAT score of 17 and is shown by an arrow.
The allocation to groups with a CAT score cut-off of 17, labelled as GOLD 2011 CAT17, and GOLD 2011 mMRC is shown in . A substantial proportion of patients classified under higher categories with mMRC shifted to lower categories with CAT 17. With GOLD 2011 mMRC and GOLD 2011 CAT17, 171 (42.7%) patients were classified differently, resulting in only a fair agreement with a kappa coefficient of 0.40.
Table 4. Distribution of patients using mMRC or modified CAT score cut-off of 17.
shows the frequency of exacerbations and hospitalisations across the old GOLD stages. ANOVA indicated significant differences for both exacerbations and hospitalisations (p < 0.0001). The frequency of exacerbations was the highest in stage IV [1.63 ± 1.41 (95% CI 1.19–2.08)]. Paired comparisons showed that stage II [0.72 ± 1.09 (95% CI 0.55–0.89)] did not differ from stage I [0.27 ± 0.63 (95% CI 0.05–0.49)] or III [0.93 ± 1.25 (95% CI 0.74–1.11)] (p > 0.05). Other paired comparisons revealed significant differences: stage IV vs stages III (p < 0.01), II (p < 0.001) and I (p < 0.0001), and stage III vs stage I (p < 0.05). Similarly, the frequency of hospitalisation was the highest in stage IV [0.77 ± 1.02 (95% CI 0.43–1.08)]. The frequency of hospitalization did not differ significantly (p > 0.05) among stages I (zero counts), II [0.26±0.58, 95% CI 0.17–0.35) and III (0.31±0.67, 95% CI 0.21–0.41); it was significantly greater in stage IV compared to stages III (p < 0.001), II (p < 0.001) and I (p < 0.0001).
Figure 4. Frequency of exacerbations (Exac.) and hospitalisations (hosp.) across the old GOLD stages (ANOVA: F 9.718, p < 0.0001 and F 9.017, p < 0.0001, respectively). For post-hoc paired comparisons, see text.
Categorisation of patients with GOLD 2011 Stage and GOLD 2011 Exacerbations classifications was significantly different (, p < 0.0001) with only 219 (54.7%) patients being classified in the same groups. GOLD 2011 Exacerbations criteria resulted in 200 (50%) patients being assigned to group D compared to 119 (29.7%) with GOLD 2011 Stage. The agreement in classification between the two criteria was poor with kappa coefficient of only 0.12.
Table 5. Distribution of patients using GOLD stage and frequency of exacerbations.
Discussion
Our study showed a wide scatter of CAT scores across mMRC grades of dyspnoea with lack of significant differences between grades 1 and 2. While most patients with grade-1 dyspnoea had low CAT scores, i.e. <10, nearly one-third of patients with grade-2 dyspnoea also had similarly low CAT scores. Due to this overlap, there was only a moderate agreement in group assignment between mMRC grading and CAT scores. Nearly one-fifth of the patients were classified differently with shifts occurring both upwards and downwards on applying the alternative criteria. Similarly, there were substantial overlaps in the frequency of exacerbations/hospitalisations across GOLD stages though these tended to increase with increasing stage. Consequently, the categorisation of patients with these criteria also had poor agreement with only about half the patients being classified in the same groups. Choice of exacerbations criteria resulted in more patients being assigned to group D than the use of stages to classify the risk.
Jones et al. Citation(19) have recently shown a similar moderate agreement in group assignment by the mMRC or CAT score. Kim et al. Citation(21) obtained similar results, and Rieger-Reyes et al. Citation(23) found that more than 25% of patients were classified into different categories by the two criteria. Rhee et al. Citation(28) corroborated these observations and more recently, Han et al. Citation(29) have observed that the choice of criteria for symptom assessment is an important confounder affecting group assignment. The use of CAT resulted in more patients being placed into groups B and D. However, the patients assigned to these groups using the mMRC scale of dyspnoea had a higher burden of cardiac co-morbidities than patients assigned by CAT. Our results are consistent with these studies and confirm that the equivalence of symptoms criteria assumed in the new GOLD classification does not appear to be valid. Furthermore, the ROC analysis of our data showed that a CAT score of 10 had a sensitivity of approximately 82% but a specificity of only 24% to identify patients with mMRC ≥ 2.
Attempts to redefine the equivalence between mMRC and CAT scores have been without success. Jones et al. Citation(19) suggested that mMRC grade ≥ 1 and CAT ≥ 10 were approximately similar in identifying less symptoms patients although this too did not yield a perfect agreement. Price et al. Citation(22) suggested that a CAT score ≥ 26 was equivalent to mMRC ≥ 2 to achieve a more consistent classification. Rhee et al. Citation(28) using an ROC analysis found a CAT score of 15 to yield the best combination of sensitivity and specificity but without substantial improvement in agreement between classifications. In the present study, a CAT score of 17 and above obtained from the ROC analysis yielded the best combination of specificity and sensitivity, i.e. 98% and 52%, to identify patients with mMRC ≥ 2. However, it yielded an even poorer agreement with mMRC grading than the CAT score ≥ 10.
A failure to establish an acceptable equivalence between the mMRC grade of dyspnoea and CAT score in the previous and current studies is not unexpected given that these are fundamentally different tools. The mMRC is a unidimensional ordinal grading of activity limitation due to dyspnoea, while CAT is a multidimensional tool that measures health status by quantifying the impact of COPD on a person's life. These are therefore different through related constructs and therefore may not be seen as equivalent alternatives for symptoms assessment as recommended in the new GOLD strategy document.
Dyspnoea of mMRC grade ≥ 2 indicates interference with activities of daily living and thus can be said to be a reasonable threshold for ‘more’ symptoms. With a specificity approaching 100% in the present study, a CAT score of ≥17 would identify all true positives with more symptoms. This is also a meaningful cut-off as it has recently been shown to be the best threshold to predict mortality Citation(29). However, because of a modest sensitivity, it would yield unacceptably high false negatives. Seventy-nine patients classified into group D with mMRC in the present study were classified into group C on applying the CAT 17 criteria. This downgrading would have major clinical implications, as it would lead to undertreatment. The GOLD strategy document recommends the use of a comprehensive symptom assessment using CAT over mMRC but retains the latter as an alternative because of its widespread use globally. Although a comprehensive symptom assessment is desirable, the CAT instrument alone does not appear to be an appropriate tool to identify more symptoms because no cut-off can be defined with an acceptable sensitivity and specificity to identify more-symptom patients. On the other hand, using ‘the higher of mMRC grade ≥2 dyspnoea or CAT score ≥ 17’ would avoid false negatives and also yield a single group assignment. Such an approach would also be consistent with the GOLD strategy document recommendation for exacerbation risk assessment, that is, ‘higher of the GOLD stage or the frequency of exacerbations and hospitalizations’ that also yields a single categorisation and circumvents the discordance between these. Any discordance in categorisation on using the proposed alternative criteria would be seen as a major limitation of the new GOLD classification. Besides implications in management, it introduces an ambiguity that may limit its wider acceptance, especially among non-pulmonologists.
A limitation of our study is that the proportions of patients in groups A and C was small. This was the result of the study being hospital based in a tertiary care referral centre; therefore, only patients with clinically significant symptoms were available. Our inclusion criteria limited the study to patients with exertional dyspnoea and productive cough. It may not therefore be possible to comment on possible category shifts when the observations are applied to the wider clinical spectrum of COPD in the general population. Secondly, COPD among women in developing countries like India is largely due to the exposure to biomass fuels. CAT remains to be validated in this population. However, the GOLD strategy document makes no distinction between COPD in smokers and in non-smokers due to biomass fuel exposure in management.
The strength of the study is the discordance brought out in exacerbation risk assessment criteria, which is a new contribution. We have also suggested a reasonable alternative schema for symptoms evaluation that avoids discordance in group assignment. The suggested criteria however need to be validated against other patient-centred measures of severity in cross-sectional studies as well as prognostication of morbidity and mortality in longitudinal studies.
Conclusions
Substantial lack of agreement exists between the alternative criteria recommended for symptoms and exacerbation risk assessment in the new GOLD strategy document for COPD. This discordance may limit wider adoption of the GOLD strategy for management of COPD. In the absence of a cut-off with an acceptable sensitivity and specificity to identify more-symptom patients, the CAT score does not appear to be appropriate for use alone but needs to be combined with mMRC. Using the higher of mMRC ≥2 and CAT score ≥ 17 to identify more symptoms would yield single group assignment avoiding discordance, which is also consistent with the schema for exacerbation risk assessment.
Declaration of interest
The authors state that they have no financial, consulting and personal relationships with other people or organisations that could influence (bias) the authors' work. There are no conflicts of interest.
Funding
No financial support was received from any source, and the study was carried out with the available resources of the Institute.
Acknowledgments
The intellectual property rights in the COPD Assessment Test are owned by the GlaxoSmithKline group of companies.
References
- Gan WQ, Man SFP, Senthilselvan A, Sin DD. Association between chronic obstructive pulmonary disease and systemic inflammation: a systematic review and a meta-analysis. Thorax 2004; 59:574–580.
- Wouters EF, Groenewegen KH, Dentener MA, Vernooy JH. Systemic inflammation in chronic obstructive pulmonary disease: the role of exacerbations. Proc Am Thorac Soc 2007; 4:626–634.
- Barnes PJ, Celli BR. Systemic manifestations and comorbidities of COPD. Eur Respir J 2009; 33:1165–1185.
- Spencer S, Calverley PMA, Burge PS, Jones PW. Impact of preventing exacerbations on deterioration of health status in COPD. Eur Respir J 2004; 23:698–702.
- Halpin DM, Decramer M, Celli B, Kesten S, Liu D, Tashkin DP. Exacerbation frequency and course of COPD. Int J Chron Obstruct Pulmon Dis 2012; 7:653–661.
- Tashkin DP. Variations in FEV1 decline over time in chronic obstructive pulmonary disease and its implications. Curr Opin Pulm Med 2013; 19:116–124.
- Soler-Cataluña JJ, Martínez-García MA, Román Sánchez P, Salcedo E, Navarro M, Ochando R. Severe acute exacerbations and mortality in patients with chronic obstructive pulmonary disease. Thorax 2005; 60:925–931.
- Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD). 2006 revision. Available from: http://www.goldcopd.org (accessed September 1, 2015).
- Celli BR, MacNee W. ATS/ERS Task Force. Standards for the diagnosis and treatment of patients with COPD: a summary of the ATS/ERS position paper. Eur Respir J 2004; 23:932–946.
- National Institute for Health and Clinical Excellence. Chronic Obstructive Pulmonary Disease: Management of Chronic Obstructive Pulmonary Disease in Adults in Primary and Secondary Care. London: National Clinical Guideline Centre, 2010. Available from: https://www.nice.org.uk/guidance/cg12 (accessed September 1, 2015).
- Higgins MW, Keller JB. Predictors of mortality in the adult population of Tecumseh. Arch Environ Health 1970; 21:418–424.
- Hole DJ, Watt GC, Davey-Smith G, Hart CL, Gillis CR, Hawthorne VM. Impaired lung function and mortality risk in men and women: findings from the Renfrew and Paisley prospective population study. BMJ 1996; 313:711–715.
- Jones PW. Health status measurement in chronic obstructive pulmonary disease. Thorax 2001; 56:880–887.
- Oga T, Nishimura K, Tsukino M, Sato S, Hajiro T, Mishima M. Longitudinal deteriorations in patient reported outcomes in patients with COPD. Respir Med 2007; 101:146–153.
- Westwood M, Bourbeau J, Jones PW, Cerulli A, Capkun-Niggli G, Worthy G. Relationship between FEV1 change and patient-reported outcomes in randomised trials of inhaled bronchodilators for stable COPD: a systematic review. Respir Res 2011; 12:40.
- Bestall JC, Paul EA, Garrod R, Garnham R, Jones PW, Wedzicha JA. Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax 1999; 54:581–586.
- Jones PW, Harding G, Berry P, Wiklund I, Chen WH, Kline Leidy N. Development and first validation of the COPD Assessment Test. Eur Respir J 2009; 34:648–654.
- Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD). 2011 revision. Available from: http://www.goldcopd.org (accessed September 1, 2015).
- Jones PW, Adamek L, Nadeau G, Banik N. Comparisons of health status scores with MRC grades in COPD: implications for the GOLD 2011 classification. Eur Respir J 2013; 42:647–654.
- Han MK, Muellerova H, Curran-Everett D, Dransfield MT, Washko GR, Regan EA, Bowler RP, Beaty TH, Hokanson JE, Lynch DA, Jones PW. Implications of the GOLD 2011 Disease Severity Classification in the COPD Gene Cohort. Lancet Respir Med 2013; 1:43–50.
- Kim S, Oh J, Kim Y, Ban HJ, Kwon YS, Oh IJ, Kim KS, Kim YC, Lim SC. Differences in classification of COPD group using COPD Assessment Test (CAT) or modified Medical Research Council (mMRC) dyspnea scores: a cross-sectional analyses. BMC Pulm Med 2013; 13:35.
- Price DB, Baker CL, Zou KH, Higgins VS, Bailey JT, Pike JS. Real-world characterization and differentiation of the Global Initiative for Chronic Obstructive Lung Disease strategy classification. Int J Chron Obstruct Pulmon Dis 2014; 9:551–561.
- Rieger-Reyes C, García-Tirado FJ, Rubio-Galán FJ, Marín-Trigo JM. Classification of chronic obstructive pulmonary disease severity according to the new Global Initiative for Chronic Obstructive Lung Disease 2011 guidelines: COPD Assessment Test versus modified Medical Research Council scale. Arch Bronconeumol 2014; 50:129–134.
- Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD). 2013 Update. Available from: http://www.goldcopd.org (accessed September 1, 2015).
- Miller MR, Hankinson J, Brusasco V, Burgos F, Casaburi R, Coates A, Crapo R, Enright P, Van Der Grinten CP, Gustafsson P, Jensen R. Standardisation of spirometry. Eur Respir J 2005; 26:319–338.
- Chhabra SK, Kumar R, Gupta U, Rahman M, Dash DJ. Prediction equations for spirometry in adults from Northern India. Indian J Chest Dis Allied Sci 2014; 56:221–229.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159–174.
- Rhee CK, Kim JW, Hwang YI, Lee JH, Jung KS, Lee MG, Yoo KH, Lee SH, Shin KC, Yoon HK. Discrepancies between modified Medical Research Council dyspnea score and COPD Assessment Test score in patients with COPD. Int J Chron Obstruct Pulmon Dis 2015; 10:1623–1631.
- Casanova C, Marin JM, Martinez-Gonzalez C, de Lucas-Ramos P, Mir-Viladrich I, Cosio B, Peces-Barba G, Solanes-García I, Agüero R, Feu-Collado N, Calle-Rubio M. Differential effect of modified Medical Research Council Dyspnea, COPD Assessment Test, and Clinical COPD questionnaire for symptoms evaluation within the new GOLD staging and mortality in COPD. Chest 2015; 148:159–168.