© 2005 by the American Diabetes Association, Inc.
Validity and Reliability of an Instrument for Assessing Health-Related Quality of Life and Treatment PreferencesThe Insulin Delivery System Rating Questionnaire
1 Department of Sociology and Center for Social and Community Research, Loyola College, Baltimore, Maryland Address correspondence and reprint requests to Mark Peyrot, PhD, Center for Social and Community Research, Loyola College, 4501 North Charles St., Baltimore, MD 21210-2699. E-mail: mpeyrot{at}loyola.edu
OBJECTIVETo provide a preliminary assessment of the validity and reliability of a new measure of health-related quality of life (HRQOL) and treatment preference for insulin delivery systems. RESEARCH DESIGN AND METHODSStudy participants were 197 adults with type 1 or type 2 diabetes who completed the Insulin Delivery System Rating Questionnaire (IDSRQ), a self-administered questionnaire developed for this study. The IDSRQ assessed patient perceptions of treatment satisfaction, impact of treatment on daily activities, clinical efficacy, diabetes-related worries and social burdens, psychological well-being, and overall treatment system preference of patients using continuous subcutaneous insulin infusion (CSII) and multiple daily injections (MDI).
RESULTSThe IDSRQ subscales had acceptable reliability ( CONCLUSIONSPreliminary findings suggest that the IDSRQ is a valid and reliable measure of HRQOL and treatment preferences for insulin delivery systems. The subscales are comprehensive, sensitive to differences in user populations, and account for most of the preference for a particular insulin delivery system.
Abbreviations: CSII, continuous subcutaneous insulin infusion HRQOL, health-related quality of life IDSRQ, Insulin Delivery System Rating Questionnaire MDI, multiple daily injections
Over the past few decades, there have been a number of advances in insulin treatment. In addition to the development of new insulins (long-acting, rapid-acting, and various premixed formulations), there has been an influx of new insulin delivery devices (insulin pumps, pens, jet injectors, devices for the physically challenged, and inhalers). With all of the new insulin delivery systems, it is important to be able to assess and compare their advantages and disadvantages. Existing studies of insulin delivery systems have assessed a variety of factors, including 1) clinical outcomes such as glucose control, body weight, or insulin dose required (16); 2) psychosocial outcomes such as treatment satisfaction, treatment burden, diabetes-related worries and social burdens, and general quality of life or well-being (3,612); and 3) patient preferences and recommendations regarding the insulin delivery system (5,921). However, none of these studies has provided a comprehensive assessment of all patient-reported outcomes or has examined the associations between overall preference and the various perceived benefits of the various insulin delivery systems. The current study reports the development of a new measure, the Insulin Delivery System Rating Questionnaire (IDSRQ). The IDRSQ is designed to assess the impact of different insulin delivery systems as comprehensively as possible. Following the suggestions of others (22,23), it includes measures that are general (overall quality of life) and diabetes specific (perceived clinical efficacy, treatment satisfaction and burden, and diabetes-related worries and social burdens). In addition to assessing the psychometric properties and validity of this measure, this study also conducts specific analyses to determine which aspects of health-related quality of life (HRQOL) most strongly contribute to the overall preference patients have for different insulin delivery systems.
The IDSRQ was developed through a three-step procedure. First, the authors reviewed existing instruments and articles comparing different insulin delivery systems. Next, two focus groups were conducted with diabetes patients. Participants were asked about the advantages and disadvantages of the various insulin delivery systems they had used. Based on these sources and our own experiences with diabetes patients, we constructed a set of items and obtained feedback from several diabetes experts. The revised version of the instrument was used for the current pilot study. All participants were adults with insulin-treated diabetes. Participants were recruited from the practices of 12 physicians across the country. Each physician was asked to recruit up to 25 patients who were using an insulin pump (any make or model; continuous subcutaneous insulin infusion [CSII] group) and another 25 patients who were taking multiple (three or more) daily insulin injections using syringe or pen (multiple daily injections [MDI] group). These two insulin delivery systems were chosen because both represented relatively intensive forms of treatment, and previous research had indicated differences between the groups on many patient-reported outcomes. The study protocol called for physicians to recruit consecutive patients who were at least 18 years of age, had type 1 or type 2 diabetes for at least 1 year, were using an insulin pump (CSII) or taking MDI, and were able to read English well enough to fill out the IDSRQ questionnaire on their own. Patients were contacted by the physician or a staff member and asked to participate in the study. Those who expressed interest were given a packet containing an explanation of the study and a copy of the questionnaire, along with a stamped self-addressed envelope for returning the questionnaire directly to the researchers. Participants were asked to provide a name and address so another (identical) questionnaire could be sent to them 23 weeks after receiving the initial questionnaire to assess test-retest reliability of the IDSRQ. The follow-up questionnaire had an identification number so that it could be linked to the first questionnaire filled out by the subject. A return envelope was provided. This protocol was approved by the Human Subject Research Committee of Loyola College (Baltimore, MD).
Measures
Treatment satisfaction.
Treatment interference with daily activities.
Clinical efficacy.
Diabetes worries.
Diabetes social burdens.
Psychological well-being.
Overall treatment preference. The overall treatment preference measure was analyzed in two ways: with and without the item comparing current treatment strategy with prior treatment strategy. This item is most appropriate when the comparator is the same for all respondents or there are two groups in which the current and comparator systems are reversed (e.g., CSII versus MDI in one group and MDI versus CSII in the other group).
Statistical analysis Key determinants of overall preference were identified by hierarchical multiple regression analysis controlling for age, sex, and type of diabetes. Multiple regression analysis was used to determine which variables made an independent contribution to the prediction of overall preference and to eliminate the "halo effect," in which there is a tendency to rate something high or low in all respects. Incremental variance in overall preference explained by the IDRSQ subscales was assessed by comparing r2 before and after entry of these measures. Degree of mediation for overall preference was determined by comparing coefficients for the difference between CSII and MDI before and after entry of the IDRSQ measures. The criterion for statistical significance was set at P < 0.05, two-tailed for all analyses. All analyses were conducted using SPSS statistical software (version 11.5; SPSS, Chicago, IL).
Sample characteristics are shown in Table 1. Most participants (71.1%) used CSII at present. The sample was primarily patients with type 1 diabetes (72.1%), women (52.8%), and individuals of middle age (mean age 46.4 years); the mean duration of diabetes was 22.6 years. Most subjects (88.8%) monitored their blood glucose three or more times per day. Of those taking injections, the average number of daily injections was 4.2 (61.9% took four injections). Most injection users used a syringe exclusively (54.4%) and only 5.3% used a pen exclusively; the rest of the subjects used a combination. Of those using CSII, most subjects (77.9%) had used a syringe alone as their insulin delivery system just before switching to CSII.
Table 2 shows the descriptive statistics for the IDSRQ measures. Measures generally had acceptable psychometric properties. The reliability coefficients ranged from 0.67 to 0.92 (median 0.82). The reliability for the overall preference scale was increased to an acceptable level (>0.70) when the treatment comparison item was dropped; this is likely due to the fact that all subjects were not using the same treatment strategy for comparison. Item analysis indicated that could be increased slightly (<0.025) by dropping one item from four of the measures.
Floor effects (percent with minimum score) ranged from 0 to 10% (median 0%), and ceiling effects (percent with maximum score) ranged from 0 to 18% (median 4%). Only 1.65% of responses were missing; only the treatment comparison item had more than 10 missing responses (18 of 22 missing responses on this item were because the respondent did not have a prior treatment strategy against which to compare the current treatment strategy). Numbers of respondents with no missing data on a particular measure ranged from 173 (88%) to 195 (99%), with a median of 182 (92%). Of the 197 initial respondents, 114 (57.8%) completed the follow-up questionnaire and were used in the test-retest analysis. Test-retest reliability ranged from 0.67 to 0.94 (median 0.88). Four measures showed a statistically significant shift in the mean score over time, but all shifts were <0.25 SD units. Table 3 shows significant differences between treatment strategies on all measures except psychological well-being. Group differences on these measures ranged from 0.3 to 1.4 SD units (median 0.9). The differences were substantial for all items from the treatment preference measure. More injection users (58.2%) than CSII users (17.2%) expressed desire to switch to another insulin delivery system. More CSII users (93.6%) than injection users (56.4%) were very or completely satisfied overall with their insulin delivery system. More CSII users (83.6%) than injection users (24.6%) stated that they would definitely recommend their insulin delivery system to others. More CSII users (97.1%) than injection users (65.7%) reported that their current system was better than their prior treatment strategy.
Table 4 reports the results of multiple regression analysis. The first row of the table shows that the difference in overall preference between CSII and MDI was significant for the total sample after controlling for age, sex, and type of diabetes (and for both type 1 and type 2 participants after controlling for age and sex). In the full sample, factors assessed by IDSRQ subscales accounted for 27% of the variance in overall preference (32% for type 1 diabetes and 21% for type 2 diabetes), and mediated 53% of the difference in overall preference between CSII and MDI (60% for type 1 diabetes and 52% for type 2 diabetes). When each IDSRQ measure was entered into a model separately, each had a significant association with overall preference (results not shown). When all were entered into one model together, treatment satisfaction, clinical efficacy, and psychological well-being had significant independent associations with overall preference for all respondents and for those with type 1 diabetes. Only clinical efficacy was significantly associated with overall preference for those with type 2 diabetes, although the coefficients for treatment satisfaction, interference, and social burden would have been significant if the type 2 diabetes sample were as large as the type 1 diabetes sample. When a separate analysis was performed only for those who were CSII users, treatment satisfaction, clinical efficacy, and psychological well-being had independent associations with overall preference (results not shown).
The questionnaire used in this study differs in several important respects from the Diabetes Quality of Life (24) questionnaire, the most frequently used measure of HRQOL in people with diabetes. First, it assesses treatment satisfaction and impact of treatment specific to the insulin delivery system and is, therefore, likely to be more sensitive to specific but important differences in these factors among delivery systems (22,23,25). Second, the IDSRQ incorporates a broad range of measures, including perceived clinical efficacy and psychological well-being, neither of which the Diabetes Quality of Life assesses in detail. Third, the IDSRQ provides a multidimensional measure of overall preference, incorporating not only a retrospective comparison with the previously used delivery system but also a prospective measure of interest in changing systems and a willingness to recommend the current system to others (which may play an important role within the diabetes patient community). Although the IDSRQ was used in this study to assess MDI and CSII, it is designed to be used with any insulin delivery system (e.g., inhaled insulin) and to permit comparisons of various systems (e.g., pen versus syringe, one pump versus another pump, etc.). The IDSRQ demonstrated good validity by detecting significant differences between treatment groups on all but one domain. Although the observed differences between groups were statistically significant, that does not necessarily indicate that they are "important" or "meaningful." Unlike objective clinical measures such as HbA1c, there are no established clinical criteria for assessing the size of the differences on these measures. However, a recent review of self-report measures (26) identified 0.5 SD units as the criterion for a "minimally detectable difference," the smallest difference that a person would be able to detect. This difference, which is a "moderate" effect size (27), was observed for six of the seven IDSRQ measures. Therefore, the statistically significant differences between treatment groups identified by the IDSRQ reflect subjectively meaningful differences. That is, patients can recognize the differences involved, and those differences can influence decisions they make about their treatment. The IDSRQ had acceptable psychometric properties (internal consistency, floor/ceiling effects, and test-retest correlation). Although the test-retest analysis showed statistically significant shifts over time in responses for some measures, the shifts were relatively small (under 0.25 SD units, less than half the size of the minimally detectable difference). However, further research with larger samples is required to assess construct validity. Specifically, confirmatory factor analysis should be used to determine whether all items load significantly and primarily on hypothesized dimensions. The IDSRQ seems to be useful in identifying the factors that contribute to users preferences for different insulin delivery systems. Several earlier studies have attempted to identify the factors that influence patients preference for CSII over MDI, the two insulin delivery systems assessed in this study (5,1318). The authors of those studies offered conjecture about these factors, suggesting that any observed difference between CSII and MDI could be (one of) the key difference(s). The IDSRQ assessed these factors (and others) and found that there were differences between CSII and MDI on most. But until the current study, there had been no explicit attempt to determine which of the observed differences actually contribute to patient preference. Not all factors on which CSII and MDI differ significantly make independent contributions to the preferences for the different insulin delivery systems, as we have shown. Our analysis found that two IDSRQ domains (treatment satisfaction, clinical efficacy) made independent contributions to patients overall preference for their insulin delivery system as evidenced by a significant difference between groups, combined with a significant association with preference. Therefore, the results of previous studies have been confirmed using a more rigorous methodology. Future research examining determinants of treatment preference should use this methodology rather than offering conjecture about the possible contribution of these factors.
The major limitation of this study is the fact that the samples were not designed to provide a generalizable comparison between CSII and MDI. The samples are not representative of the populations using CSII and MDI or of all persons treating their diabetes with insulin. The physicians who recruited patients were chosen because they were favorable toward CSII and could, therefore, provide a substantial number of CSII patients to facilitate the comparison between treatment groups. Also, CSII was not necessarily offered to all their patients, so those who started on CSII may represent a sample that initially was more positively predisposed toward this insulin delivery system. However, the current study was designed for the purpose of instrument validation; we sought to create a group Although the IDSRQ was sensitive to differences between groups, this study was not designed to demonstrate sensitivity to within-subject change over time. Also, we do not know whether it will be sensitive enough to be able to detect differences between other insulin delivery systems. However, the preliminary results reported here suggest that it will be worthwhile to pursue these applications. We did not determine whether the differences between insulin delivery systems in perceived clinical efficacy were associated with differences in objective outcomes such as HbA1c levels or episodes of severe hypoglycemia. However, others have shown that satisfaction with care is associated with better glycemic control (28). Perceptions of ones insulin delivery system could well be related to ones willingness to initiate and maintain intensified treatment and, consequently, related to improved clinical outcomes. Therefore, perceived efficacy may be an important factor in its own right, independent of its relationship to objective treatment efficacy. In summary, this pilot study suggests that the IDSRQ is a valid and reliable instrument for assessing patient perceptions of their insulin delivery systems. Anyone interested in using this instrument may obtain a copy directly from the first author (there is no charge for its use). Given the importance of insulin in the management of diabetes and the increasing availability of alternative insulin delivery systems, we believe that further research should be conducted to ascertain the potential of this instrument.
This research was funded by an unrestricted educational grant from Medtronic MiniMed.
M.P. and R.R.R. are paid consultants of Medtronic MiniMed, Mannkind Biopharmaceuticals, and Novo Nordisk and have received research grants from Novo Nordisk. R.R.R. is a member of advisory boards for Novo Nordisk and Mannkind Biopharmaceuticals. A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances. Received for publication March 19, 2004. Accepted for publication September 22, 2004.
This article has been cited by other articles:
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||