The Prediction of Type 1 Diabetes by Multiple Autoantibody Levels and Their Incorporation Into an Autoantibody Risk Score in Relatives of Type 1 Diabetic Patients
Abstract
OBJECTIVE We assessed whether a risk score that incorporates levels of multiple islet autoantibodies could enhance the prediction of type 1 diabetes (T1D).
RESEARCH DESIGN AND METHODS TrialNet Natural History Study participants (n = 784) were tested for three autoantibodies (GADA, IA-2A, and mIAA) at their initial screening. Samples from those positive for at least one autoantibody were subsequently tested for ICA and ZnT8A. An autoantibody risk score (ABRS) was developed from a proportional hazards model that combined autoantibody levels from each autoantibody along with their designations of positivity and negativity.
RESULTS The ABRS was strongly predictive of T1D (hazard ratio [with 95% CI] 2.72 [2.23–3.31], P < 0.001). Receiver operating characteristic curve areas (with 95% CI) for the ABRS revealed good predictability (0.84 [0.78–0.90] at 2 years, 0.81 [0.74–0.89] at 3 years, P < 0.001 for both). The composite of levels from the five autoantibodies was predictive of T1D before and after an adjustment for the positivity or negativity of autoantibodies (P < 0.001). The findings were almost identical when ICA was excluded from the risk score model. The combination of the ABRS and the previously validated Diabetes Prevention Trial–Type 1 Risk Score (DPTRS) predicted T1D more accurately (0.93 [0.88–0.98] at 2 years, 0.91 [0.83–0.99] at 3 years) than either the DPTRS or the ABRS alone (P ≤ 0.01 for all comparisons).
CONCLUSIONS These findings show the importance of considering autoantibody levels in assessing the risk of T1D. Moreover, levels of multiple autoantibodies can be incorporated into an ABRS that accurately predicts T1D.
Several autoantibodies have now been shown to be predictive of type 1 diabetes (T1D) (1–8). For the most part, prediction has been based on the positivity of those autoantibodies. Although the dichotomy of positivity and negativity has provided prediction accuracy, the consideration of autoantibody levels could further enhance prediction. Data from some studies already suggest this (3–7).
In addition to autoantibodies, other measures have been shown to be predictive of T1D (9–14). With the growing number of T1D predictors, it has become cumbersome and somewhat arbitrary to use prediction algorithms that rely on various combinations and cutoffs of those predictors. Thus, there is a rationale for developing risk scores based on multivariate models that can more efficiently optimize the accuracy of combined predictors. The Diabetes Prevention Trial–Type 1 Risk Score (DPTRS), which includes several metabolic measures along with age and BMI, is an example (15,16).
We assessed whether levels from multiple autoantibodies can be incorporated into an autoantibody risk score (ABRS) that accurately predicts T1D in participants of the TrialNet Natural History Study (TNNHS). In addition, we assessed whether the prediction of T1D can be further enhanced when autoantibody information is combined with information from the DPTRS.
RESEARCH DESIGN AND METHODS
The TNNHS cohort has been previously described (17). All participants in the analysis were relatives of T1D patients who were positive for at least one biochemical autoantibody (GADA, insulinoma-associated antigen-2 [IA-2A], and insulin [mIAA]) at the initial screening. The TNNHS was approved by an institutional review board, and written informed consent was obtained.
Participants were tested for GADA, IA-2A, and mIAA positivity at the initial screening. If any of those autoantibody tests were positive, participants were then tested for both islet cell autoantibodies (ICA) and zinc transporter-8 (ZnT8A). Participants positive for autoantibodies were subsequently followed with 2-h oral glucose tolerance tests (OGTTs) at 6-month intervals. After fasting samples were obtained, glucose was administered orally (1.75 g/kg, maximum 75 g of carbohydrate). Glucose measurements were then obtained at 30, 60, 90, and 120 min. An OGTT in the diabetic range (by American Diabetes Association criteria) was followed by a confirmatory OGTT, unless a diagnosis could be made by the clinical presentation. Diagnoses could also be made between visits according to clinical criteria.
Laboratory measures
ICA determinations were performed at the TrialNet Islet Cell Autoantibody Core Laboratory (Gainesville, FL). All the other assays were performed at the Barbara Davis Center (Denver, CO). The procedures for measuring ICA, GADA, mIAA, IA-2A, and ZnT8A have been previously described (6,8,18). Positive testing for the autoantibodies was defined as ≥10 JDFU for ICA, ≥0.033 for GADA, ≥0.010 for mIAA, ≥0.050 for IA-2A, and ≥0.021 for ZnT8A. The cutoffs for the biochemical autoantibodies were based on the 99th percentiles of normative data. Because the biochemical autoantibodies are expressed as indexes and ICA is expressed as titer, for simplicity, we use the term levels to indicate the autoantibody measurements.
The glucose oxidase method was used for plasma glucose measurements. C-peptide level was measured by the Tosoh assay for the TNNHS. In a prior analysis, 564 individuals had C-peptide measurements by both the Tosoh assay and the radioimmunoassay (RAI) used in the Diabetes Prevention Trial of Type 1 Diabetes (r = 0.961, Tosoh = 0.96 × RAI + 0.1).
Data analysis
Analyses were designed for two main purposes: developing an ABRS and assessing whether levels of multiple autoantibodies improve the prediction of T1D. The ABRS was based on a model that included positivity/negativity and level for each of the five autoantibodies. Another risk score was based on a model that included the ABRS and the DPTRS as variables. The risk score models and their calculations are shown in the Supplementary Data. The development and validation of the DPTRS has been described previously (15,16). The prediction variables for the DPTRS are the sum of glucose values at 30, 60, 90, and 120 min divided by 100; the sum of C-peptide values at 30, 60, 90, and 120 min divided by 10; the log fasting C-peptide level; age; and the log BMI. The methodology for the conversion of the risk scores to risk estimates was previously reported (15); it is also shown in the Supplementary Data. Differences were assessed with t and χ2 tests. A leave-one-out procedure was used for cross-validation (19). CIs were calculated by the bootstrapping procedure. Areas under receiver operating characteristic (ROC) curves, adjusted for length of follow-up (20), were used to assess the accuracy of the scores. Participants first tested for autoantibodies >1 year before the OGTT were excluded from the analyses. Four outliers with extreme values were also excluded from the analyses. The values of the outliers together with the ranges of values after their exclusion are shown in the Supplementary Data. SAS version 9.2 (SAS Institute, Inc.) was used. P < 0.05 (two-sided) was considered statistically significant.
RESULTS
There were 784 TNNHS participants analyzed (mean ± SD age 19.4 ± 13.8 years; 9% <5 years of age; 57% female) of whom 95 developed T1D during follow-up. Of the 784, 92% were first-degree relatives. Most of those studied were white (84%) and non-Hispanic (89%). The mean ± SD duration of follow-up was 1.7 ± 1.1 years for those diagnosed and 1.9 ± 1.1 years for those not diagnosed.
An ABRS was developed on the basis of determinations of positivity or negativity and the level of each of the five autoantibody measurements obtained from the TNNHS participants. The ABRS was highly predictive of T1D (hazard ratio 2.72 [95% CI 2.23–3.31], P < 0.001). When ICA was excluded from the risk score model, the hazard ratio was essentially identical (2.72 [2.23–3.32], P < 0.001). A composite of the levels used in the ABRS was in itself strongly predictive of T1D (P < 0.001) before and after an adjustment for the positivity or negativity of the autoantibodies.
Figure 1A shows the 3-year risk estimates according to ABRS categories. There was only a gradual increase in the 3-year risk estimates over the lower ABRS categories. However, the risk estimates then increased substantially from 0.10 for values in the 1.50 to <2.00 ABRS category to 0.24 for values in the 2.00 to <2.50 ABRS category. The risk estimate rose to 0.65 among those with ABRS values ≥3.00. The risk estimates were almost the same when ICA was excluded from the risk score model (Fig. 1B).
The curves represent 3-year risk estimates according to ABRS categories with (A) and without (B) the inclusion of ICA. Both curves show little overall increment for ABRS values <2.00. For values ≥2.00, the 3-year risk increases with increasing ABRS values in both curves. The number of participants receiving a diagnosis within 3 years of follow-up is shown above each point.
Table 1 shows the areas under the ROC curves for the prediction of T1D by the ABRS and by autoantibody number. The area under the ROC curve for the ABRS was significantly greater than that for autoantibody number at 2 years (P < 0.05) but not at 3 years. When ICA was excluded from the model for the risk score, the areas under the ROC curve were almost identical. There was also little change when ICA was not included in autoantibody number.
Areas (95% CI) under ROC curves* (adjusted for follow-up) for autoantibody number and autoantibody score with and without the inclusion of ICA (n = 784)
When the internal validation was performed to further assess the accuracy of the ABRS, the values were 0.80 at 2 years and 0.77 at 3 years (P < 0.001 for both). To further assess the specific effect of autoantibody levels on prediction accuracy, we developed a risk score based on positivity or negativity alone (without levels) of the five autoantibodies. In the absence of autoantibody levels, the areas under the ROC curve were lower when the validation was performed (0.75 at 2 years, 0.73 at 3 years).
We calculated the areas under the ROC curve specifically for the combination of ICA positivity and autoantibody level. The values were relatively low (0.64 [95% CI 0.58–0.70] at 2 years, 0.67 [0.60–0.73] at 3 years) compared with the ABRS values.
We calculated areas under the ROC curve for the prediction of T1D by the DPTRS. The number of participants was somewhat smaller (n = 704) for this analysis due to missing data (mostly BMI measurements). Areas under the ROC curve for the DPTRS were 0.89 (0.84–0.94) at 2 years and 0.87 (0.79–0.95) at 3 years (P < 0.001 for both).
Figure 2 is a scatterplot of the ABRS versus the DPTRS. The Pearson (parametric) and Spearman (nonparametric) correlation coefficients were the same (r = 0.44, P < 0.001 for both). The distribution of participants who ultimately received a diagnosis of T1D is shown within the scatterplot. The scatterplot was divided into quadrants on the basis of ABRS values <1.50 vs. ≥1.50 and DPTRS values <6.00 vs. ≥6.00 to show how the two scores could be used together to identify populations according to risk. It is evident that T1D was diagnosed much more frequently in those participants whose values were above both thresholds (65 of 244 [27%]) than in those whose values were below both thresholds (2 of 221 [1%]).
The scatterplot shows the association of the ABRS with the DPTRS. The closed circles represent participants diagnosed with T1D. The fractions indicate the number diagnosed over the total in each quadrant. There is a marked predominance of T1D diagnoses among those above both the ABRS and the DPTRS thresholds. The correlation coefficient is shown (Pearson and Spearman coefficients were the same value).
Both the ABRS and the DPTRS were highly predictive of T1D when they were included together in a proportional hazards model (P < 0.001 for both); however, the association was stronger for the DPTRS (χ2 = 66 and 22 for DPTRS and ABRS, respectively). When the ABRS and the DPTRS were included together in a combined risk score, there was improvement in prediction accuracy (0.93 [0.88–0.98] at 2 years, 0.91 [0.83–0.99] at 3 years) over the ABRS (difference from ABRS P < 0.001 and P < 0.01 at 2 and 3 years, respectively) and the DPTRS (difference from DPTRS P = 0.01 and P < 0.01 at 2 and 3 years, respectively) alone. The areas under the ROC curve were identical when ICA was excluded. Figure 3 shows the more substantial area under the ROC curve when the ABRS and DPTRS are combined.
ROC curves at 2 years of follow-up are shown for the ABRS and the combined ABRS and DPTRS. The area under the ROC curve increases from the ABRS to the combined ABRS and DPTRS (P < 0.001 for difference). (The irregularity of the curves reflects statistical estimation variability resulting from the adjustment for censoring.) AUC, area under the curve.
When the internal validation procedure was performed, the areas under the ROC curve of the combined ABRS and DPTRS were 0.89 at 2 years and 0.89 at 3 years (P < 0.001 for both). Of the 277 participants with a combined ABRS and DPTRS <6.00, only 4 (1%) developed T1D, whereas of the 102 participants with a combined risk score ≥8.00, 47 (46%) were diagnosed. The 3-year risk estimate for the latter group was 0.71.
CONCLUSIONS
The number of autoantibodies, their differing prediction accuracies (either singly or in combination), and the additional information from autoantibody levels make it cumbersome and inefficient to use algorithms to predict T1D. It would be necessary to choose from a multiplicity of combinations of positive autoantibodies and cutoffs of autoantibody levels. Such algorithms would still not use the full scope of information available for prediction. A risk score, such as the ABRS, provides a means of incorporating and maximizing available information into one measure.
There was a strong association of the progression to T1D with the composite of the levels from the five autoantibodies before and after an adjustment for their positivity or negativity. It has been previously shown that levels of autoantibodies can be predictive of the age of onset of T1D (4,6) and that levels can be predictive among those who tested positive for a particular autoantibody (3). The present analysis shows that autoantibody levels provide predictive information for the diagnosis of T1D beyond determinations of positivity. Moreover, to our knowledge, this is the first report of prediction based on a composite of autoantibody levels. Although IA-2A levels have been incorporated into a T1D risk score for siblings of patients (4), no previous studies have assessed and used as broad an array of autoantibody levels for a risk score. It is important to emphasize that the ABRS was developed in the context of a population of relatives of T1D patients who tested positive for autoantibodies. Thus, the findings might not be relevant to other populations. However, data suggest that the presentation of T1D in sporadic cases (nonrelatives) is similar to that of relatives (21–23).
ICA is a strong predictor of T1D, and it has been a basis for the selection of participants in major prevention trials that have been performed thus far (24–26). Yet, both the proportional hazards and the ROC data indicate that the inclusion of ICA in the ABRS adds little to the overall prediction beyond that provided by the four biochemical autoantibodies. This is demonstrated in Fig. 1A and B, which shows that the relationship of the 3-year cumulative incidence to the risk score changes little with the exclusion of ICA. However, it is still possible that ICA could be of value for prediction in certain individuals. (We have included the prediction models for the ABRS with and without ICA in the Supplementary Data.)
Findings from a recent report (8) suggest that risk misclassification could occur if autoantibody number alone is relied on for prediction without regard for the pattern of positivity and autoantibody levels. The report showed that among TrialNet participants with two or more autoantibodies, there was a marked risk difference according to the presence or absence of ZnT8A. Those with two or more autoantibodies and ZnT8A negativity were actually at relatively low risk for T1D (<10% at 3 years). Moreover, individuals with negative tests for ZnT8A represented a sizable proportion of those with two or more autoantibodies (118 of 237 [50%]). Reports describing T1D patients in the years after diagnosis suggest that ZnT8A autoantibodies are important to understanding pathogenesis (27–29). The particular significance of ZnT8A autoantibodies shows why specific types and levels of autoantibodies should be considered to avoid faulty determinations of eligibility for prevention trials.
The ABRS was highly predictive when the internal validation procedure was performed. Although other populations are not currently available for an external validation, the internal validation findings within TrialNet suggest that the ABRS, whether alone or in combination with the DPTRS, can help to accurately identify target populations for future TrialNet studies.
With the exception of ICA, end point titers were not measured. Because it is possible that end point titration could further improve prediction, it would be helpful for future studies to examine this.
The areas under the ABRS ROC curve were greater at 2 years than at 3 years. Although there is no definite explanation for this finding, more participants had follow-up at 2 years than at 3 years. Thus, there was more information at 2 years than at 3 years, resulting in greater precision of the 2-year estimate, which is evident in the narrower CIs at 2 years than at 3 years.
The ABRS would not be applicable if new autoantibodies are found to be predictive of T1D or if autoantibody measurement methodologies differ in other laboratories. However, the basic structure of the model for the ABRS (i.e., positivity/negativity, level, an interaction term for each autoantibody) can be used for developing a modification of the ABRS. The variables used for the ABRS were not selected on the basis of prediction performance per se. Rather, they were chosen a priori to be inclusive of the positivity/negativity and the levels of all the measured autoantibodies in the TNNHS.
The relative prediction accuracies of the ABRS and the DPTRS should be interpreted with caution. The DPTRS, which is largely based on metabolic predictors, could be more indicative of short-term risk, whereas the ABRS could be more indicative of long-term or even lifetime risk. Although confirmatory data are not available, this possibility is suggested by the earlier appearance of autoantibody abnormalities than of overt metabolic abnormalities in the natural history of T1D (30).
There are several considerations in deciding which of the risk scores would be best to use. The ABRS is limited to the specific autoantibody screening algorithm used in the TNNHS. In that algorithm, ICA and ZnT8A were only measured if another autoantibody was present. However, because ICA and ZnT8A are usually present in association with other autoantibodies (5,8), the ABRS is relevant to the majority of relatives who tested positive for autoantibodies. Another consideration is that the ABRS was validated internally with the leave-one-out procedure, whereas the DPTRS was validated in a separate cohort. However, only a blood sample is needed for the ABRS, whereas a 2-h OGTT is required for the DPTRS. Thus, the ABRS would probably be more advantageous for first-order screening.
Even though the ABRS and the DPTRS were correlated to some degree, the proportional hazards data suggest that it could be advantageous to use prediction information from both. The scatterplot in Fig. 2 shows a marked preponderance of T1D diagnoses among participants who exceeded the 1.5 ABRS and 6.00 DPTRS thresholds. It should be noted that the scatterplot does not take into account the length of follow-up. Additionally, the thresholds are arbitrary. Still, the scatterplot shows how information can be used from both risk scores. In interpreting the degree of correlation, the different times the components (autoantibody vs. metabolic) of the scores were obtained should be taken into account. Furthermore, the components of the scores could represent different stages in the progression to T1D.
The combined ABRS and DPTRS enables a more quantitative use of information from the two risk scores than a dependence on thresholds from each of their distributions. Although the ABRS and DPTRS combination has not been validated in a different population, the areas under the ROC curve were very high in the internal validation. Thus, the combination of ABRS and DPTRS appears to provide a highly accurate estimation of T1D risk that efficiently uses a composite of multiple sources of predictive information. The combined risk score also provides a more efficient means of prediction than the use of thresholds for each of the risk scores.
In conclusion, this study shows that a composite of levels from multiple autoantibodies can enhance the prediction of T1D. The levels from multiple autoantibodies can be incorporated into the ABRS, which is highly predictive of T1D. Moreover, the ABRS and the previously developed DPTRS can be used in tandem to further enhance the risk prediction of T1D through the use of risk thresholds either from each or from a combined risk score.
Acknowledgments
The sponsor of the trial was the Type 1 Diabetes TrialNet Study Group. The Type 1 Diabetes TrialNet Study Group is a clinical trials network funded by the National Institutes of Health (NIH) through the National Institute of Diabetes and Digestive and Kidney Diseases, the National Institute of Allergy and Infectious Diseases, and the Eunice Kennedy Shriver National Institute of Child Health and Human Development through cooperative agreements U01-DK-061010, U01-DK-061016, U01-DK-061034, U01-DK-061036, U01-DK-061040, U01-DK-061041, U01-DK-061042, U01-DK-061055, U01-DK-061058, U01-DK-084565, U01-DK-085453, U01-DK-085461, U01-DK-085463, U01-DK-085466, U01-DK-085499, U01-DK-085505, and U01-DK-085509 and contract HHSN267200800019C; the National Center for Research Resources through Clinical Translational Science Awards UL1-RR-024131, UL1-RR-024139, UL1-RR-024153, UL1-RR-024975, UL1-RR-024982, UL1-RR-025744, UL1-RR-025761, UL1-RR-025780, UL1-RR-029890, UL1-RR-031986, and P30-DK-017047 and General Clinical Research Center Award M01-RR-00400; the JDRF; and the American Diabetes Association (ADA).
No potential conflicts of interest relevant to this article were reported.
The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the NIH, JDRF, or ADA.
J.M.S. analyzed the data and wrote the manuscript. J.S.S., J.P.K., J.M., L.R., D.S., and G.E. conducted the study and reviewed the manuscript. J.P.P. conducted the study, assisted with writing the manuscript, and reviewed the manuscript. L.Y. performed the autoantibody measurements and reviewed the manuscript. C.A.B. contributed statistical support. D.C.B. contributed programming and statistical support. J.M.S. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Footnotes
This article contains Supplementary Data online at http://care.diabetesjournals.org/lookup/suppl/doi:10.2337/dc13-0425/-/DC1.
↵* A complete list of the Type 1 Diabetes TrialNet and the Diabetes Prevention Trial–Type 1 Study Groups can be found in the Supplementary Data.
- Received February 20, 2013.
- Accepted May 9, 2013.
- © 2013 by the American Diabetes Association.
Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. See http://creativecommons.org/licenses/by-nc-nd/3.0/ for details.
References
In this Issue
