Risk Scores for Type 2 Diabetes Can Be Applied in Some Populations but Not All

  1. Charlotte Glümer, MD, PHD1,
  2. Dorte Vistisen, MSC, PHD1,
  3. Knut Borch-Johnsen, DMSC1,
  4. Stephen Colagiuri, MD2 and
  5. on behalf of the DETECT-2 Collaboration
  1. 1Steno Diabetes Center, Gentofte, Denmark
  2. 2Department of Endocrinology and Diabetes, Prince of Wales Hospital, Sydney, Australia
  1. Address correspondence and reprint requests to Charlotte Glümer, MD, PhD, Steno Diabetes Center, Niels Steensens Vej 2, 2820 Gentofte, Denmark. E-mail: chgl{at}steno.dk

Abstract

OBJECTIVE—Risk scores based on phenotypic characteristics to identify individuals at high risk of having undiagnosed diabetes have been developed in Caucasian populations. The impact of known risk factors on having undiagnosed type 2 diabetes differs between populations from different ethnic origin, and risk scores developed in Caucasians may not be applicable to other ethnic groups. This study evaluated the performance of one risk score in nine populations of diverse ethnic origin.

RESEARCH DESIGN AND METHODS—Data provided by centers from around the world to the DETECT-2 project were used. The database includes population-based surveys with information on at least 500 people without known diabetes having a 75-g oral glucose tolerance test. To date, 52 centers have contributed data on 190,000 individuals from 34 countries. In this analysis, nine cross-sectional studies were selected representing diverse ethnic and regional backgrounds. The risk score assessed uses information on age, sex, blood pressure treatment, and BMI.

RESULTS—This analysis included 29,758 individuals; 1,805 individuals had undiagnosed diabetes. The performance of the risk score varied widely, with sensitivity, specificity, and percentage needing further testing ranging between 12 and 57%, 72 and 93%, and 2 and 25%, respectively, with the worse performance in non-Caucasian populations. This variation in performance was related to differences in the association between prevalence of undiagnosed diabetes and components of the risk score.

CONCLUSIONS—A typical risk score developed in Caucasian populations cannot be applied to other populations of diverse ethnic origins.

Type 2 diabetes is a common and serious condition associated with reduced life expectancy and considerable morbidity. Recent estimates suggest that currently 195 million people throughout the world have diabetes, and this will increase to over 330 million by 2025 (1). Approximately 50% of people with diabetes are undiagnosed (2,3). Because type 2 diabetes may remain undetected for several years, at the time of clinical diagnosis, many people have one or more micro- or macrovascular complications (4).

Detecting people with undiagnosed type 2 diabetes is important for both public health policy and everyday clinical practice. Because of the rapidly increasing prevalence of type 2 diabetes (58), screening individuals at high risk of having undiagnosed diabetes is recommended in several countries (911). Several questionnaires have been developed to detect this high-risk group (1217). They all perform equally well, with a sensitivity of 70–75% and a specificity of 55–70%. Because the majority of risk scores have been developed and validated in Caucasians (12,18), their applicability to populations of different ethnic background and with different risk factor distribution is uncertain.

The DETECT-2 project is an international data pooling collaboration specifically addressing issues related to screening for type 2 diabetes, with an emphasis on the impact of ethnicity and population differences on screening protocols (19). The broad questions that DETECT-2 is investigating include evaluating selected strategies for screening for undiagnosed type 2 diabetes across a range of populations from diverse ethnic backgrounds, the development of a simple screening strategy for type 2 diabetes applicable to different populations throughout the world, and an assessment of the implications with regard to morbidity and mortality for individuals categorized on the basis of a screening program for diabetes.

The aim of this article is to compare and evaluate the performance of a typical risk score for undiagnosed type 2 diabetes developed in a Caucasian population, when applied in populations with diverse ethnic backgrounds.

RESEARCH DESIGN AND METHODS

A total of 52 centers from 34 countries worldwide have contributed data on >190,000 individuals to the DETECT-2 database. Minimum requirements for participation were population-based surveys or large cohorts of employees, including information on at least 500 people, with all people without previously known diabetes having a 75-g oral glucose tolerance test. Undiagnosed diabetes was classified according to the 1999 World Health Organization criteria (20). The dataset includes information on the blood specimen used for the glucose measurement (venous whole blood, venous plasma, or capillary whole blood) and the method of glucose assay. The World Health Organization equivalence table was used to convert all results into plasma glucose equivalents (20). For all studies used in this analysis, the fasting time was comparable between 10 and 12 h.

For this analysis, nine datasets from the DETECT-2 database were selected, which were representative of people from a diverse range of ethnic backgrounds (Northern and Southern Europe, U.S., Indian subcontinent, Asia, Australia, Pacific Islands, and Africa) (2,8,2127). For the Third National Health and Nutrition Examination Survey (NHANES III), only the subsample of individuals that underwent a 75-g oral glucose tolerance test was included (2). These samples are not necessarily representative of the whole population but are a convenience sample on which to test the risk score.

The risk score tested was the Rotterdam Predictive Model (RPM) (12). It was developed from the Rotterdam Study cohort (28) and was externally validated in the Hoorn study (29). It was chosen as the risk score for testing because it is simple, uses information collected during a routine consultation, and has been externally validated. The RPM includes information on age, weight, sex, and treatment with antihypertensive medications. The score was as follows:

  • Age: per 5-year increment from 55 years: 2 points

  • Male: 5 points

  • Use of antihypertensive medications: 4 points

  • BMI ≥30 kg/m2: 5 points

An individual with total score above 6 points was considered as high risk of having undiagnosed diabetes.

In each center, data collection was performed according to local ethical rules and according to the Helsinki Declaration.

Statistical analysis

All analyses were performed using SAS version 8.2 (SAS Institute, Cary, NC). The performance of the RPM was assessed when applied to each population. The particular performance characteristics examined included area under the receiver-operator characteristic (ROC) curve (AUC), sensitivity (proportion of people with undiagnosed diabetes having a positive screening test), specificity (proportion of people without undiagnosed diabetes having a negative screening test), positive predictive value (proportion of people tested positive having undiagnosed diabetes), and the proportion of the population identified as requiring further biochemical testing. Positive predictive value depends on the prevalence of diabetes and was calculated using the prevalence of newly diagnosed diabetes from each study. The ROC curve is sensitivity plotted against 1-specificity for each cutoff value (30). It provides a visual comparison of the test performance, and the AUC is a measure of diagnostic accuracy. The AUC represents the probability that within each study population, a randomly selected diseased individual has a higher value of the test than a randomly selected nondiseased individual. Approximate CIs for the performance outcomes were estimated by bootstrapping (1,000 bootstraps) (31). All analyses were applied to the 30- to 65-year age range.

RESULTS

The characteristics of the studies are shown in Table 1. The prevalence of undiagnosed diabetes ranged from 0.9% in Cameroon to 19.6% in the Western Pacific islands of Nauru and Tonga. The lowest mean BMI was observed in the Indian subcontinent (23.1 kg/m2) and the highest in the Western Pacific (34.9 kg/m2). The mean age ranged from 43.0 to 51.6 years.

Figure 1 and Table 2 show the performance characteristics of the RPM for each population compared with the results in the Dutch population (12). AUCs ranged from 0.53 to 0.70. The AUCs were similar in the Caucasian countries of Denmark, Spain, Australia, and the U.S., with a mean AUC of 0.70 (95% CI 0.68–0.72). This was not significantly different than the original result in the Dutch population (0.68 [0.64–0.72]). However, the RPM did not perform as well in discriminating between undiagnosed diabetes and nondiabetes in the non-Caucasian populations, with AUCs ranging from 0.53 in Africa to 0.62 in the Western Pacific population. The mean AUC for the non-Caucasian populations was 0.61 (0.59–0.62), which is significantly lower than for the Caucasian populations (P < 0.0001).

When the RPM-recommended cut point of >6 points was assessed, there was considerable variation in performance parameters (Table 2). Sensitivity ranged from 11.5 to 56.5%, specificity from 65.1 to 92.8%, positive predictive value from 1.7 to 25.4%, and the percentage of the population requiring further testing from 7.7 to 38.0%. Even the populations with satisfactory overall performance by the ROC curve showed considerable variation. Among the Caucasian populations, Denmark had the lowest sensitivity (41.9%) and the U.S. had the highest (56.5%) compared with 78% in the Dutch population. These differences were also reflected in the percentage of the population requiring further testing, which was also lowest for Denmark (17.1%) and highest in the U.S. (30.9%). Among the non-Caucasian populations, the Western Pacific had the highest sensitivity (51.4%) but also the lowest specificity (65.1%).

The factors contributing to the RPM score were examined to determine the reasons for the difference in performance. In all the Caucasian populations, the odds ratio of having undiagnosed diabetes increased by age, whereas in India and Africa, the odds ratios were higher among individuals aged 40–59 years compared with those aged ≥60 years (Table 3). The same tendency was observed in BMI. In India, there was a significantly increased risk of having diabetes if BMI was >22.5 kg/m2 compared with a BMI <22.5 kg/m2. In the Caucasian populations, BMI had very little impact when BMI was <25 kg/m2 (Table 3). The use of antihypertensive treatment was positively associated in all populations with the risk of having undiagnosed diabetes and with odds ratios of the same magnitude.

CONCLUSIONS

Assessment of risk of undiagnosed type 2 diabetes is commonly used to identify individuals who should be recommended for further biochemical testing. Several risk assessment tools have been developed for this purpose using a combination of demographic, clinical, and sometimes biochemical information (1217,32). These risk assessment tools have invariably been developed and tested in Caucasian populations.

This study has demonstrated that a risk assessment tool developed in a Caucasian population performs reasonably well in other Caucasian populations with similar distribution of risk factors, but not in other populations of diverse ethnic origin. The major reason for the lack of transferability of the risk score is differences of the impact of especially BMI and age on the prevalence of undiagnosed diabetes.

Even in the Caucasian populations, using the same risk score cut point gave substantial differences in sensitivity, specificity, positive predictive value, and percentage of the population requiring further testing. No attempt was made to modify the risk score to examine whether performance could be improved, since this was not the purpose of this study. However, because the ROC curves were relatively flat between a score of 5 and 6, it would be difficult to overcome the differences in performance parameters by adjusting the cut point alone. Because we used unweighted data from NHANES III, the predictive value and the percentage needing further testing are likely to be overestimated for the North American region. However, the overall performance measured as AUC, the sensitivity, and the specificity will not be affected. Decreasing the prevalence to 8% will decrease the positive predictive value from 18.7 to 15.1%, and the percentage that needed further testing will decrease from 30.9 to 30.4%. The prevalence of undiagnosed diabetes is low in Africa, which implies that the calculations are based on very few cases. This especially affects the sensitivity.

Few studies have assessed the performance of risk assessment tools developed in one country and then applied them to populations of different ethnic origins. Tabaei and Herman (32) developed a predictive equation in an Egyptian population and reported comparable performance when applied to a U.S. population. Their equation included a combination of demographic information and capillary blood glucose and had a sensitivity of ∼65% and specificity of 96% in both populations. The similarity in performance could be anticipated from the demographic similarities of the two populations, which had a similar age and mean BMI (29.8 kg/m2 in Egypt vs. 28.4 kg/m2 in the U.S.). Spijkerman et al. (33) assessed the performance of the Cambridge risk score in detecting undiagnosed hyperglycemia (fasting plasma glucose ≥7 mmol/l or HbA1c ≥6.5%) in ethnic minority groups from the Indian subcontinent and the Caribbean subjects living in the U.K. In the original cohort, a Cambridge risk score cut point of 0.199 gave a sensitivity of 77% and specificity of 72%, with 30% of the population requiring further testing. Even after adjusting the cut point to 0.127 for the Indian subcontinent and to 0.236 in Caribbean subjects, the overall performance AUC, sensitivity, and specificity decreased.

This study has shown that a simple risk assessment tool developed in a Caucasian population does not perform well in populations of different ethnic origins with different clinical characteristics, emphasizing the need to develop ethnic-specific risk scores for screening for undiagnosed type 2 diabetes. A major aim of the international collaboration, DETECT-2, is to develop screening strategies applicable across ethnic regions throughout the world (19) that take into account ethnicity and differences in distributions of important risk factors for undiagnosed diabetes.

APPENDIX

Investigators and study centers included in this analysis

Australia: Paul Zimmet, Jonathan Shaw, International Diabetes Institute (AusDiab). Cameroon: Léopold Fezeu, Jean-Claude Mbanya. Denmark: Torben Jørgensen, Charlotte Glümer, Knut Borch-Johnsen, Research Centre for Prevention and Health (Inter99). India: A. Ramachandran, Diabetes Research Centre, M.V. Hospital for Diabetes, Chennai, India. Korea: Nam H. Cho, Department of Preventive Medicine, Ajou University School of Medicine (The Korean Health Study); Kyu C. Kimm, Korean National Genome Institute–Korean National Institute of Health. Nauru: Paul Zimmet, Jonathan Shaw, International Diabetes Institute. Spain: Conxa Castell, Servei d’Educacio Sanit. I Prog. de Salut., Advisory Committee on Diabetes in Catalonia, Department of Health. Tonga: Taniela Palu, Stephen Colagiuri, National Diabetes Center (Tonga). U.S.: Yiling Cheng, Mike Engelgau, Centers for Disease Control (NHANES III).

DETECT-2 Steering Committee

Stephen Colagiuri, Knut Borch-Johnsen, George Alberti, Rhys Williams, Gojka Roglic, Beverley Balkau, A. Ramachandran, and Jaakko Tuomilehto.

DETECT-2 Secretariat

The secretariat comprise Knut Borch-Johnsen, Charlotte Glümer, and Dorte Vistisen, Steno Diabetes Center.

Figure 1—

ROC curves by study population.

Table 1—

Characteristics of the regional and country population data included in this analysis

Table 2—

Performance of the Rotterdam predictive model for each population

Table 3—

Crude odds ratios for age, antihypertensive treatment, and BMI for undiagnosed type 2 diabetes

Acknowledgments

The DETECT-2 project (evaluation of screening and early detection strategies for type 2 diabetes and impaired glucose tolerance) was undertaken on the initiative of the World Health Organization and the International Diabetes Federation.

We wish to acknowledge and thank all the centers that provided data for the DETECT-2 database.

Footnotes

  • C.G. and K.B.-J. hold stock in and have received research funding from Novo Nordisk.

    A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.

    • Accepted October 28, 2005.
    • Received May 24, 2005.

References

| Table of Contents