Validity of Medication Adherence Self-Reports in Adults With Type 2 Diabetes

Participants were recruited from the Diabetes Center and primary care clinics at Massachusetts General Hospital or were self-referred via hospital e-mail mailing lists and radio advertisements for a larger intervention study on the efficacy of cognitive behavioral therapy for adherence and depression (18) in patients with type 2 diabetes. Entry criteria for the study included the ability to read and write in English, a diagnosis of type 2 diabetes, a prescription for medication to treat hyperglycemia (either insulin or oral medication) and taking oral medication for diabetes or a diabetes-related condition (i.e., hypercholesterolemia or hypertension), and age between 18 and 70 years. Analyses indicated no significant differences in the relationships of self-reported adherence to HbA_1c levels for patients not taking an oral antihyperglycemic medication (because they were taking insulin only; n = 25) versus those who were taking an oral antihyperglycemic medication, so both groups of participants were included.

To qualify for the intervention study and to receive an electronic bottle cap, participants were required to meet DSM-IV criteria for major depressive disorder (MDD) or dysthymia or to exhibit clinically significant depressive symptoms despite the prescription of an antidepressant. They were also required to have an HbA_1c ≥7.0%. Prescriptions for diabetes and depression medications were required to have been stable for 2 months. Those with severe mental illness requiring immediate treatment or precluding ability to participate were excluded. Participants provided informed consent, and the Partners HealthCare Institutional Review Board approved the study.

All participants were evaluated for depression severity, completed all self-report measures, and provided a blood sample to measure HbA_1c levels at the initial baseline evaluation. Self-report questions for medication adherence asked participants about their adherence to their overall diabetes-related oral medication regimen (i.e., prescribed for hyperglycemia, hypercholesterolemia, or hypertension). The subsample of participants who qualified for the intervention were given an electronic pill bottle cap at the baseline visit and were asked to continue using it until their next study visit. Thus, the electronically monitored adherence assessment followed the self-reports and captured only pre-intervention data.

The Montgomery Åsberg Depression Rating Scale (19), a structured clinical assessment of 10 commonly occurring symptoms of depression over the past week, was used to measure depression symptom severity. Scores range from 0 to 60: 0–6 indicates no depression; 7–19 indicates mild depression; 20–34 indicates moderate depression; and 35–60 indicates severe depression. Internal reliability was high (α = 0.85).

Questions developed by Lu et al. (12) to assess adherence to antiretroviral medications over the past month were adapted to assess adherence to oral diabetes-related medications. The instructions for all self-ratings were as follows: “The following questions ask about your diabetes medications. This includes oral medications you take for diabetes, high blood pressure, and high cholesterol.” Participants were asked to quantitatively evaluate their adherence using 11 response categories (0, 10, 20… 100%) with the question, “What percent of the time did you take all your diabetes medications as your doctor prescribed?” Participants were also asked to qualitatively rate their adherence (rating scale: very poor, poor, fair, good, very good, excellent) with, “On average, how would you rate your ability to take all your diabetes medications as your doctor prescribed?” As in Lu et al. (12), participants were asked to respond considering the past month. One-week time frames were added to facilitate comparison with other measures.

On the basis of the approach of the AIDS Clinical Trials Group (ACTG) Adherence Questionnaire (20), we asked participants to report the number of prescribed doses for each diabetes-related medication and record the number of missed doses, per medication, during the past 7 days. We refer to this measure as “missed doses.” A calendar was used to facilitate recall, and adherence was calculated as the percentage of prescribed doses taken for the past week. For participants taking multiple medications, a mean percentage of doses taken across medications was calculated. Participants also completed an item from the Summary of Diabetes Self Care Activities (SDSCA), which asked participants, “On how many of the last seven days did you take your recommended diabetes medication” (21). We refer to this measure as “adherent days.”

Medication Event Monitoring System (MEMS) bottle caps were used to track one medication per participant. For participants taking an antihyperglycemic medication, MEMS caps tracked the antihyperglycemic medication that the participant took the most frequently or found most difficult to remember. For the 25 participants taking insulin and no oral antihyperglycemic medication, hypertension- or hypercholesterolemia-related medications were monitored. MEMS adherence was calculated to provide the percentage of doses taken by dividing the number of times a bottle was opened by the number of openings prescribed during the time frame. Trained assessors followed the approach used by Safren et al. (22) to correct MEMS data only if participants could identify a day on the calendar when they took their medication in a way that it would not be recorded by MEMS (e.g., took out two pills in the morning and pocketed one for later that day). Participants were required to have at least 5 days of MEMS data for the current analyses.

Participants completed a structured interview of psychiatric diagnoses (23) and provided a blood sample for HbA_1c at the baseline visit, which was analyzed at the same hospital laboratory. They also completed study-generated questionnaires regarding their treatment regimen, comorbid illnesses, presence of diabetes complications, time since diagnosis, and demographic and socioeconomic factors.

All analyses were done in SPSS 18.0 software. All variables were examined for normality and were analyzed as continuous. The missed doses measure was not normally distributed; therefore, correlations and regression analyses for this variable are based on a base 10 logarithm of the original data. Bivariate correlations were used to determine correlations between HbA_1c, MEMs data, and self-report adherence data. We used Williams’ t test (Wt) (24) to compare the strength of correlations across adherence measures. Computations based on those of Lu et al. (12), transformed the 6-point Likert scales of the qualitative self-rating items to 100-point scales (0, 20, 40, 60, 80, and 100). We then calculated means and SDs for all indices of adherence on a 0–100 scale and computed discrepancies between self-reported adherence and MEMS adherence.

Multivariate linear regression analyses focused on HbA_1c (rather than MEMS) because it was available for the full sample. Sex, age, education, and the prescription of insulin were evaluated as potential covariates to be included in multivariate analyses if significantly correlated with HbA_1c or MEMS. Because depression was a focus of the study, we evaluated evidence for whether depression severity moderated the relationship between self-reported adherence and HbA_1c through interaction terms, with appropriate post hoc probing of significant moderation effects. This procedure involved the computation of two conditional variables for depression, one in which lower depression severity was coded as 1 SD below the centered mean and one in which higher depression severity was coded as 1 SD above. The slopes of these analyses were then used to plot regression lines for the relationship between self-reports and HbA_1c at high and low levels of depression severity (25). We did not examine MEMS in these moderation analyses because of the smaller subsample with available data. This subsample also had a restricted range of depression severity and glycemic control as a result of our design.

We limited our analyses to the 170 screened participants who provided a baseline HbA_1c and answered the self-reported adherence questions. Of these, 88 qualified for the intervention study and provided MEMS cap data before intervention. The average number of days of adherence collected by MEMS caps was 20.59 (SD, 11.81). Number of days of monitoring was not significantly associated with MEMS adherence (P > 0.10).

Of the 170 participants included in the first set of analyses, 56.5% were men, 81% were white, mean age was 56 years (SD, 10) and mean HbA_1c was 8.3% (SD, 1.7). Participants had an average of 14 years (SD, 3) of education, and 59% qualified for a diagnosis of MDD. Of the 88 people given an MEMS cap, 52.3% were men and their mean age was 57 years (SD, 8). These participants had an average of 14 years (SD, 3) of education, 84% were white, mean HbA_1c was 8.8% (SD, 1.6), and 74% met criteria for MDD. Table 1 presents descriptive statistics for those in the overall sample and for the subsample included in the MEMS analyses. Consistent with the expected effects of the intervention’s selection criteria, MEMS participants had significantly more severe Montgomery Åsberg Depression Rating Scale scores (t [166] = –3.77, P < 0.001) and were more likely to meet diagnostic criteria for MDD (χ² [1, n = 168] = 21.87, P < 0.001) than those who did not qualify for the intervention. They also had a significantly longer duration of diabetes (t [164] = −2.71, P = 0.008) and were more likely to be on insulin (χ² [1, n = 168] = 8.81, P = 0.003).

All adherence self-report items were significantly and inversely correlated with HbA_1c. MEMS adherence and HbA_1c were similarly significantly correlated (r = −0.25, P = 0.021; see first two data columns in Table 2). Comparing correlation coefficients revealed no significant differences in correlations between any of the self-report measures and HbA_1c levels (all P > 0.17).

All correlations between self-reported adherence and MEMS were positive and significant (see third and fourth data columns in Table 2). Comparisons by Wt test indicated that the quantitative 1-week adherence self-rating was more strongly correlated to MEMS adherence than the qualitative 1-week self-rating (Wt = −0.29, P = 0.005) and missed doses during the past week (Wt = −3.49, P < 0.001). In addition, the 1-month quantitative adherence self-rating was more strongly correlated to MEMS than the 1-month qualitative self-rating (Wt = −3.22, P = 0.002). There was no difference between the 1-week quantitative self-rating and the 1-month quantitative self-rating (Wt = −1.02, P = 0.31). The remaining comparisons were not significant (P > 0.06).

Reported adherence, expressed as a percentage, was highest for missed doses of medication and lowest for 1-month qualitative self-ratings (see fifth data column in Table 2). Discrepancies between self-reported adherence and subsequent MEMS adherence are presented in the last column of Table 2 and show that 1-month qualitative self-ratings and missed doses tended to have the greatest discrepancy with MEMS data, whereas quantitative self-ratings showed closest concordance. Mean differences were smallest, and near zero, for the 1-month percentage-based rating. Age, depression severity, and education were not significantly related to these discrepancies (data not shown).

Of potential covariates, only insulin use was significantly associated with HbA_1c (r = 0.32, P = 0.003) and was included in the multivariate models. Results from separate regression models for each measure of self-reported adherence are presented in Table 3. The inclusion of depression severity as a main effect did not meaningfully attenuate relationships between self-reports and HbA_1c. However, several significant moderation effects were found. Specifically, depression severity was a significant moderator of the relationship between 1-week qualitative adherence self-ratings (P = 0.025) and 1-week quantitative self-ratings (P = 0.046) and HbA_1c. A significant interaction (P < 0.001) was also found between depression and reports of missed doses. Post hoc probing of these significant interaction effects showed that the relationship between self-reported adherence and HbA_1c was negative and significant at low levels of depression severity but near zero and nonsignificant at high levels (Fig. 1).

The findings from this study support the validity of easily administered self-report measures to assess medication adherence in adults with type 2 diabetes. We compared self-reported and MEMS-measured adherence and found significant correlations for all self-report items, suggesting that self-reported adherence shares a significant amount (7–30%) of variance with objectively monitored adherence, our proximal indicator of validity. This was found despite the lack of overlap in time frames and even though MEMS only monitored one medication whereas the self-report items required participants to consider their diabetes-related oral medications in aggregate. These associations, especially those for the percentage-based global self-ratings, were considerably stronger than those found by a previous study of more than 500 patients with type 2 diabetes between the 4-item self-report Morisky Medication Adherence Scale or the adherent days item from the SDSCA and pharmacy refill records (4% shared variance each) (11). A previous study of more than 800 patients with heart failure or hypertension also found a weaker correlation between self-reported medication adherence based on a modified version of the Morisky scale, and MEMS adherence (9.6% shared variance) (26). Our results suggest that part of the variation in the amount of the shared variance between self-reports and MEMS adherence can be attributed to measurement characteristics (discussed in detail below).

The relatively more modest relationships between adherence and HbA_1c were expected, because glycemic control is affected by a variety of factors beyond medication adherence, including diet and exercise, degree of insulin deficiency, and adequacy of the prescribed treatment regimen. Although modest relationships between adherence self-reports and HbA_1c are often interpreted as reflecting the limited validity of these measures, this may not be an appropriate conclusion. For example, a recent meta-analysis of 21 studies in pediatric type 1 diabetes indicated that adherence and glycemic control share less than 8% of their variance. No differences in the strength of this association were found for studies that used self-reported adherence versus those that used objective meter downloads (27). Similarly, in a large sample of adults with type 2 diabetes, objective pharmacy refill data were significantly associated with current HbA_1c and change in HbA_1c over time; however, the amount of variance explained was only 4% and 1.7%, respectively (28). Thus, it is important to recognize that validating adherence self-reports based only on relationships with clinical outcomes such as HbA_1c may lead to less accurate conclusions because of the relatively modest impact of adherence on this outcome (2,29). The strength of the association between adherence and HbA_1c should also inform clinicians working with patients with suboptimal glycemic control who report being adherent to their treatment regimen.

Our findings provide novel information on the influence of characteristics of self-report measures on the validity of adherence estimates. Measures asking respondents to report missed doses tended to over-predict subsequently monitored MEMS adherence, whereas global quantitative self-ratings were more concordant with subsequent MEMS data. It may be surprising that less specific, single-item global measures resulted in more valid adherence estimates than more intensive reviews of missed doses. However, this is consistent with the literature on the validity of self-reported adherence, which suggests that patients can provide general estimates of adherence more accurately than specific missed doses (13,30,31). In contrast to the findings of Lu et al. (12), which showed that qualitative self-ratings were more accurate than other self-reports in predicting MEMS adherence, we found that percentage-based self-ratings were most strongly associated with MEMS adherence. Because these studies differed substantially in the characteristics of their samples, replication of these results is needed. However, both studies suggest that global ratings provide more valid estimates than self-reports that rely on a recall of missed doses.

Our findings also suggest an important role for depression severity in the measurement of self-reported medication adherence in individuals living with diabetes. Analyses revealed significant interaction effects between depression severity and several adherence self-reports, the 1-week qualitative and quantitative self-ratings, and the measure of missed doses during 1 week, in predicting HbA_1c. In each case, adherence reports were not associated with HbA_1c levels at higher levels of depression severity but were significantly related at lower levels. The moderation effect was not significant for the SDSCA item regarding adherent days or the 1-month adherence self-ratings. The moderating effect of depression is consistent with research demonstrating that mood can affect memory and recall abilities. Specifically, studies have consistently found that clinically depressed individuals demonstrate a recall bias that favors remembering negative stimuli rather than positive stimuli (32). Therefore, participants with greater depression severity possibly remembered themselves as less adherent than they actually were, whereas nondepressed participants more accurately remembered their adherence. We can only speculate about why the depression effect was not found for the days-adherent or 1-month self-rating items. It is plausible that the SDSCA item was less vulnerable to this effect due to its focus on days of adherence rather than nonadherence. It is also plausible that participants were more likely to attempt to remember missed doses during the past week when responding to the 1-week self-ratings but more likely to rely on a gestalt for their adherence when providing ratings for 1 month.

We are aware of only one other study that has examined the role of depression severity in the concordance between self-reports for adherence and objectively measured adherence (26). This study of patients with hypertension or heart failure found no significant differences in correlations between self-reported adherence and pharmacy refill adherence for depressed versus nondepressed patients. Although not significant (P = 0.07), this study found that the correlation between self-reported adherence and MEMS adherence was slightly higher for depressed individuals. However, this study used two different self-report measures to screen for depression, treated depression as a categorical variable, and did not conduct formal tests of moderation. Thus, results should be compared with caution. Given the increased prevalence of depression symptoms in patients with diabetes (33) and the consistent relationship between depression and diabetes treatment nonadherence (14), the role of depression in adherence self-reports deserves further investigation.

Our results should be considered in the context of our study design. First, our sample was recruited for a depression intervention study. Although this limits the generalizability of our findings, it allowed us to examine the effect of relatively severe symptoms of depression.

Second, we were limited in our ability to examine the concordance between self-reports and MEMS data because these measures did not cover the same time frame. On the other hand, this allowed us to evaluate predictive validity.

Third, MEMS only tracked one diabetes-related medication, whereas three self-reports (qualitative, quantitative, and missed doses) asked about all diabetes-related medications the participant was taking. In addition, the SDSCA adherent-days item asked about “diabetes medication,” though we expect that given the explicit instructions for the other items, participants likely answered thinking about the same class of medications used for the other self-report measures. No self-reports asked specifically about adherence to insulin. Furthermore, only a subset of the original sample, selected for depression severity and suboptimal glycemic control, was monitored with MEMS caps. Because this subsample had a restricted range of depression severity, we focused our depression moderation analyses on HbA_1c only.

Finally, we note that whenever adherence was measured, we were obtaining information on more than the ingestion of medication. Research shows that adherence to a placebo is consistently associated with mortality risk in various patient populations (34,35). The relationship between measures of medication adherence and clinical outcomes likely reflects other behaviors and respondent characteristics that directly influence health; this is often referred to as the “healthy adherer effect.” Given this context, we were less concerned with specifically measuring the link between antihyperglycemic medication and HbA_1c and instead focused on adherence to the broad class of medications necessary to manage the risk of diabetes complications. Future studies should build on this work by adapting these self-report measures to refer to specific medications and differentiate between insulin and oral medications.

The accurate measurement of medication adherence in diabetes is important for progress in the development of interventions to improve diabetes self-management and for appropriate treatment decisions in the delivery of care. Clear evidence demonstrates important negative health consequences of nonadherence in patients with diabetes (1). Objective measures of adherence are often impractical for various research designs and do not address the problem of how treatment providers should inquire about treatment adherence with their patients. Developing self-report measures of medication adherence with strong validity would therefore have important benefits for research and practice. Results of the current study contribute to our progress toward this goal and should encourage further investigation into the best practices for the assessment of diabetes treatment adherence.

This project was supported by National Institute of Mental Health (NIMH) Grant R01-MH-078571; Principal Investigator: S.A.S. Additional support came from the Investigator-Initiated Study Program of LifeScan, Inc., in the form of donated glucometers and glucose test strips, and the Harvard Catalyst/Harvard Clinical and Translational Science Center by National Institutes of Health (NIH) Grant 1 UL1-RR-025758-03, for a portion of nurse and dietician study visits. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH/NIMH, LifeScan, or the Harvard Catalyst. J.S.G. was partially supported by Grant DK-020541 from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). D.W. was supported by NIDDK Grant DK-080228-05.

No potential conflicts of interest relevant to this article were reported.

J.S.G. contributed to study design and data acquisition, analyzed and interpreted data, contributed to discussion, and wrote, reviewed, and edited the manuscript. H.E.S. analyzed and interpreted data and wrote the manuscript. D.J.W., C.P., L.M.D., E.C., and S.A.S. contributed to study design, data acquisition, and discussion and reviewed and edited the manuscript. J.S.G. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

The authors thank Dr. Lawrence Fisher, University of California, San Francisco, for helpful comments on an earlier draft.