Do patient characteristics matter?
OBJECTIVE—To examine state variability in diabetes care for Medicare beneficiaries and the impact of certain beneficiary characteristics on those variations.
RESEARCH DESIGN AND METHODS—Medicare beneficiaries with diabetes, aged 18–75 years, were identified from 1997 to 1999 claims data. Claims data were used to construct rates for three quality of care measures (HbA1c tests, eye examinations, and lipid profiles). Person-level variables (e.g., age, sex, race, and socioeconomic status) were used to adjust state rates using logistic regression.
RESULTS—A third of 2 million beneficiaries with diabetes aged 18–75 years did not have annual HbA1c tests, biennial eye examinations, or biennial lipid profiles. There was wide variability in the measures among states (e.g., receipt of HbA1c tests ranged from 52 to 83%). Adjustment using person-level variables reduced the variance in HbA1c tests, eye examinations, and lipid profiles by 30, 23, and 27%, respectively, but considerable variability remained. The impact of the adjustment variables was also inconsistent across measures.
CONCLUSIONS—Opportunities remain for improvement in diabetes care. Large variations in care among states were reduced significantly by adjustment for characteristics of state residents. However, much variability remained unexplained. Variability of measures within states and variable impact of the adjustment variables argues against systems effects operating with uniformity on the three measures. These findings suggest that a single approach to quality improvement is unlikely to be effective. Further understanding variability will be important to improving quality.
- CMS, Centers for Medicare and Medicaid Services
- DQIP, Diabetes Quality Improvement Project
- ESRD, end-stage renal disease
- QIO, quality improvement organization
- ZQ, zip quality
Diabetes is a major cause of morbidity and mortality for millions of Medicare recipients with diagnosed and undiagnosed diabetes (1–3). Over $27 billion was spent on diabetes care for individuals aged ≥65 years in the U.S. in 1997 (4). For these reasons, the Centers for Medicare and Medicaid Services (CMS), the nation’s largest purchaser of health care services, has chosen diabetes as a national priority for quality monitoring and improvement in the Medicare population (5,6). While providing national direction, CMS implements and evaluates its Medicare quality improvement program at the state level, using individual state quality improvement organizations (QIOs), formerly known as peer review organizations. This makes knowledge and understanding of state-specific differences in care important to the Medicare program and to CMS in particular.
Three diabetes quality of care measures (HbA1c tests, eye examinations, and lipid profiles) were selected as the focus of CMS quality improvement efforts in the late 1990s. Initial analysis of the 1997–1999 claims data, from which these measures were generated, demonstrated both a significant opportunity for improvement and wide variability among states (7). The median rate (and extremes) among the states for annual HbA1c testing was 71% (low 52%, high 85%). Similarly, it was 69% (low 56%, high 80%) for biennial eye exams and 57% (low 39%, high 73%) for biennial lipid profiles. Significant variation in diabetes care for Medicare beneficiaries living in three different states has been previously reported (8,9). However, few studies have explored variations in diabetes care among all states, and none have done so using such a large population of patients. We sought to determine how much of the state-to-state variation reported in the Medicare claims data remained after adjusting for differences in patient characteristics among states. Knowledge of the sources of variation is important for understanding use and for maximizing CMS and QIO efforts to improve quality.
RESEARCH DESIGN AND METHODS
Data source and data collection time frame
Medicare claims data for individual beneficiaries were obtained for all 50 states and the District of Columbia. The data were for use by QIOs who are under contract with CMS to conduct statewide diabetes quality improvement projects in all 50 states and in the District. These data were collected according to state QIO contract cycles, which occur in three groups, 90 days apart. Cycle 1 data were collected from 1 January 1997 to 31 December 1998 for eye examinations and lipid profile testing and from 1 January 1998 to 31 December 1998 for HbA1c testing. Cycle 2 and 3 data were collected 3 and 6 months, respectively, after the start of cycle 1 collection.
The three quality measures evaluated for this study were annual HbA1c testing, biennial eye examination by an eye care professional (i.e., an ophthalmologist or optometrist), and biennial lipid profile testing. These measures were closely based on the widely used measures developed by the Diabetes Quality Improvement Project (DQIP) (10). CMS chose to replicate the DQIP measures as closely as possible for its national diabetes quality improvement efforts (11), with the additional constraint that only claims-based measures be used. This constraint was imposed to minimize national program costs and prevent an additional administrative burden on health care practitioners participating in QIO improvement activities.
Medicare beneficiaries consist of three main subpopulations, based on their reason for entitlement to Medicare benefits. These are the elderly (aged ≥65 years), the disabled (aged <65 years), and individuals with end-stage renal disease (ESRD). These subpopulations are not mutually exclusive; for example, disabled and ESRD patients can successfully age into the Medicare elderly population. There are no contraindications to delivery of or reimbursement for any of the above measures based solely on an individual’s reason for entitlement.
Paralleling DQIP specifications, fee-for-service Medicare beneficiaries were included in the denominator if they had a diabetes diagnosis on either one acute hospital (inpatient or emergency room) encounter or two outpatient, home health, or emergency encounters; were between 18 and 75 years of age as of the end of the measurement period; had Part B coverage with no more than a 1-month break in enrollment during the 2-year measurement period; had no health maintenance organization coverage during the 2-year measurement period; were alive at the end of the measurement period; and were residents of their current state for the most recent 1-year period.
Several researchers have validated this algorithm for the identification of individuals with diabetes from Medicare claims data (9,12). Hebert et al. (13) reported positive predictive values over 90% using similar, though less rigorous, algorithms. Individuals were excluded from the denominator if they had a code for gestational diabetes or if they had missing data (e.g., age and sex) that prevented construction of any adjustment variable.
Numerators for the measures contained the number of individuals in the denominator population, defined above, who had a claim containing the specific Current Procedural Terminology codes for the test or procedure being measured at anytime during the specified data collection period (11). Complete numerator and denominator specifications are provided in the online appendix (http://diabetes.diabetesjournals.org).
To improve the comparability of rates among states, the three performance measures in this analysis were adjusted using individual descriptors such as age, sex, race, eligibility for both Medicare and Medicaid (dual eligibility), ESRD, zip quality (ZQ) rating (14), number of outpatient visits, comorbidities and illness severity (based on the Dartmouth-Manitoba modification of the Charlson comorbidity index ), and presence of diabetes complications as described in the Charlson criteria.
Age, sex, race, dual eligibility, and ESRD status were obtained from the CMS enrollment database. Dual eligibility was defined as Medicaid coverage for at least 1 month during the 2-year period of Medicare coverage. The ZQ score was used as a broader measure of socioeconomic status. Our ZQ was derived from the Claritas ZQ rating score (14) and was associated with the individual’s zip code as recorded in the CMS enrollment database. The Claritas ZQ score (mean = 50, SD = 10) is a zip code level aggregate measure composed of four components: household income, educational attainment, occupation, and home value. The number of outpatient visits for the 2-year measurement period included all visits to primary care or internal medicine specialists, as determined from the physician supplier, home health, and institutional outpatient databases. Several traditionally considered adjustment variables (e.g., mean area income and rural/urban residence as defined by the county-level U.S. Department of Agriculture rural-urban commuting area code ) were tested but were not used in the final adjustment due to either colinearity with retained variables or lack of explanatory power.
Statistical adjustment methods
We conducted exploratory analyses of the predictor variables and the measures for both the entire population and the three entitlement subpopulations. The large population sizes make even small differences (<0.1%) statistically significant, so Tilton’s overlap procedure (17,18) was used to calculate the percentage overlap in the distributions of the three entitlement subpopulations and to assure that the overlaps were sufficient to allow the three subpopulations, to be treated as one. Because the data covered slightly different periods of time, as required for use by the QIOs (periods staggered by 3 months), temporal differences were examined and found to be noncontributory to differences in rates of the three measures among states.
We used logistic regression to adjust the state-level rates (19–21). Each quality measure was first used as a dependent variable at the person-level, adjusting for the variables described above. The same set of adjustment variables was used in modeling all three measures. Using the output of the logistic regression, we predicted each beneficiary’s probability of receiving each of the three measures. The person-level predictions for each beneficiary were next aggregated by state to produce the predicted state rate for a given quality measure. State rates were then further adjusted by the degree to which the adjustment variables predicted each outcome measure, using a mean deviation procedure, according to the following formula (22): adjusted state rate = unadjusted rate − (predicted rate − mean predicted rate).
This adjustment formula modified state rates by the degree to which the explanatory variables validly predicted each outcome measure. The greater the predictive validity in the explanatory variables, the more the predicted state rates were different from their overall mean. If the explanatory variables were only randomly associated with the quality measure, the predicted state rates would all be the mean of the state rates and no adjustment would occur. After adjustment, Spearman’s rank correlations were used to further examine the adjustment’s impact.
Characteristics of the overall population and the three major entitlement subpopulations are presented in Table 1. Our final diabetic population consisted of 1,941,517 Medicare beneficiaries, aged 18–75 years, after 36,860 (1.9%) individuals were excluded due to missing data for any adjustment variable. Those who were eliminated did not substantively differ from the total population on the characteristics listed in Table 1. While some of the adjustment variable distributions had overlaps of <50% between subpopulations, all but one of the outcome measures showed distribution overlaps >85%, despite sometimes apparently large differences in rates.
The overall rates were 67.8% for annual HbA1c tests, 68.3% for biennial eye examinations, and 56.8% for biennial lipid profiles. Rates varied across the subpopulations, with the disabled subpopulation having a much lower rate of eye exam receipt (54.0%), and the ESRD subpopulation having much lower rates of receiving HbA1c tests (54.7%) and lipid profiles (33.1%).
Unadjusted rates for the three measures, stratified by the adjustment variables used, are presented in Table 2. In addition, the maximum and minimum unadjusted state-specific rates are shown. The outcome measures varied across all of the adjustment variables. Consistently lower rates of receipt for all three measures were seen among those <65 years of age, blacks, those from neighborhoods with ZQ scores below the median, and those with five or fewer outpatient visits. However, unadjusted differences across the states were wider than those across most of the adjustment variables: 37.7 percentage points for HbA1c, 21.9 for eye exams, and 32.1 for lipid profiles.
The impact of the adjustment variables was not uniform across the three measures (Table 3). Certain variables (when adjusted for all others) have an essentially uniform impact (e.g., outpatient visits: odds ratio [OR] 1.020–1.027, implying that each additional outpatient visit increases receipt of the services in question by 2.0–2.7%), while others (e.g., sex: OR 0.991–1.342) were variable. After adjustment, both beneficiaries with ESRD and those with dual eligibility were less likely to receive all three items. But the effect of dual eligibility was fairly consistent, while that of ESRD status varied by a factor of more than two.
Variability among and within states in adjusted rates
The wide variations among states in the measures were significantly reduced but not eliminated by adjustment of the state rates based on characteristics of state residents (Fig. 1). Geographic patterns also remained variable. For HbA1c testing and eye exams, adjusted rates tended to be higher in New England, the upper Midwest, and the Pacific Northwest and lower in the Ohio valley, lower Mississippi valley, and the Southwest. Adjusted lipid testing rates were higher in the mid-Atlantic states and some Southern costal states, but lower in the Rocky Mountain states and parts of the lower Mississippi valley.
Adjustment reduced the variance (i.e., σ2) in the state rates by 30% for annual HbA1c testing, 23% for biennial eye exam rates, and 27% for biennial lipid profile testing. The largest positive percentage point change from unadjusted to adjusted rates occurred in Mississippi, with gains of 5.4 percentage points for HbA1c testing, 5.8 for eye examinations, and 7.7 for lipid testing. This means that taking into account differences between the national population and that of Mississippi results in Mississippi showing higher rates for the three quality measures. The largest negative percentage point change from unadjusted to adjusted rates occurred in New Hampshire for HbA1c testing (−4.6 percentage points) and lipid testing (−5.8) and in Massachusetts for eye examinations (−4.1).
Spearman’s rank correlations for the unadjusted/adjusted pairs were 0.97, 0.91, and 0.83. Thus, the impact of the simultaneous combination of the adjustment variables did not greatly disrupt the overall relative rankings of the states. Rank-order changes were largest for lipid profiles, relatively small for eye examinations, and quite small for HbA1c tests, suggesting along with Table 3 that the adjustment variables function differently for the three outcome measures studied.
There was also considerable variability among the three measures of care within the states. Correlation coefficients between the pairs of adjusted measures were 0.64 for HbA1c testing and eye examinations, 0.48 for HbA1c and lipid profiles, and 0.38 for lipid profiles and eye examinations. Only North Dakota had rates for all three measures within the top quintile, and only Illinois had rates in the bottom quintile for all measures.
The 2002 American Diabetes Association guidelines for type 2 diabetes recommend, at minimum, semiannual HbA1c testing, annual eye exams, and beinnial lipid profiles for low-risk individuals (23). Nationally, a third or more of fee-for-service Medicare beneficiaries with diabetes did not receive each of these recommended services; in the worst performing state, closer to half or more did not receive these services. The present study both demonstrates opportunities for improvement in the care provided to Medicare beneficiaries with diabetes and indicates that differences in state Medicare population mix do not account for most of the variation in care between states.
There are several explanations for the apparent underutilization of these key elements of diabetes care. According to McNeil (24), provider “uncertainty with regard to decision making in individual cases and, more broadly, with regard to the establishment of guidelines or criteria for determining the appropriateness of care” plays a part in under use. In the case of individual decision making for members of the Medicare population, clinicians are dealing with an elderly or disabled population with competing comorbidities that may influence diabetes care decisions. In the case of uncertainty with regard to guidelines, many of the studies that have bearing on guidelines for diabetes care are recent, and in some cases conflicting (25–31). While glycemic control has long been held as the most important factor in preventing all diabetes complications (25,26), some recent studies suggest that blood pressure (29) and blood lipid control (30,32,33) are at least equally important in preventing cardiovascular complications. Finally, limitations of resources may preclude intensive efforts in all three clinical areas.
Variation among states in the care for diabetes is not unexpected, since similar variation has been documented previously (34–38). However, to our knowledge, this is the first study with this large a number of people (almost 2 million) to examine the impact of characteristics of individuals residing within states on the variability of care provided among states. Furthermore, although the impact of selected individual characteristics on care has been previously examined in several smaller scale studies (39–41), this study is the first large-scale examination of the impact of characteristics of residents with diabetes in a state on care provided within that state. For example, it has been reported that racial and ethnic minority groups in many areas do not use primary care diabetes services as frequently as the Caucasian population (42,43). However, the proportion of minority Medicare beneficiaries and other population characteristics explain less of the state-to-state variation than was expected.
While the variance explained by person-level characteristics (one-fourth to one-third of the variance among states) is considerable, a majority of the variation among states remains unexplained. It may be that factors other than the characteristics of state residents are the predominant determinants of variability. Differences among states could be influenced by regional or state characteristics, health care systems, and clinical practice patterns. Alternately, characteristics of state residents not measured in this study might be important. Other individual factors that might be important, but could not be included in the model, include lack of transportation, access barriers (44), and cultural factors.
It is also possible that some of the variables used are not accurate proxies for the characteristics they were intended to represent (e.g., ZQ score as a proxy for socioeconomic status), and some zip codes may not be accurately recorded in the CMS enrollment data. Otherwise, we feel our data are accurate in terms of individual characteristics and claims. Although racial misclassification occurs in the CMS enrollment data, these data have been shown to be reasonably accurate when limiting classifications to the white, black, and other categories used in this study (45).
There are several other limitations to the study. For example, use of administrative claims data limits the ability to characterize some components of the Charlson index (comorbidities and severity of illness) (15). Also, our claims data most accurately reflect lipid profiles and HbA1c testing, while the eye examination codes may result in overrepresentation of retinal exams. Some of the person-level adjustment variables considered were based on zip code or county-level data and thus may have introduced more homogeneity than would be the case had it been possible to truly construct these at the person level. For example, it is possible that using county-level urbanicity data (mapped to zip code), instead of person-level data, diluted the relationship of urban versus rural residence to receipt of care and caused this variable’s failure to appear in the final model. But no source of person-level (address-specific) rural versus urban status was available. Also, enrollment in Medicare may work to dilute the importance of urbanicity by providing uniform coverage and perhaps improved access to care.
We did not examine variation in the effect of individual characteristics in different states, nor did we examine systematic variation at the state level. The former analysis might reveal that certain states have better quality of care for blacks or ESRD patients, and the latter might reveal that states with more physicians per capita or higher Medicare reimbursement levels perform differently than those with fewer physicians or lower reimbursement.
Most recently, a better algorithm has been used for calculating lipid profile receipt based on Medicare claims data. While applying this algorithm would produce slightly higher point estimates for the lipid measure, it would not statistically alter the results we obtained.
The intriguing story to emerge from this large-scale study of the Medicare population with diabetes is one of both opportunities for improvement and widespread variability in care. For CMS and the QIOs, understanding the variability within and among states is essential to capitalizing on opportunities for quality improvement. Unfortunately, our findings argue against systems and systematic effects that uniformly influence processes of care that could easily be addressed by a homogeneous national program. Wide variability in the measures occurred both among states and among the three measures within states, and the impact of the adjustment variables on the three performance measures was also variable.
A homogeneous system of care would not result in many states performing in widely different quintiles for two or more of the three quality measures, and patient characteristics such as race or sex would not have a different influence on each of these measures if a systematic effect of the characteristic were occurring. A very important implication of this variability is that different approaches may be needed among and within the states to improve care for each of the quality measures examined here. Understanding the causes of variability is the key to developing and implementing targeted quality improvement programs. Further research is needed to explore the impact of physicians, nurses, and perhaps other providers, along with health care systems and state characteristics on the variability in care delivered to the Medicare population for this very personally devastating and resource-intensive disease.
All funding for this work was provided by the CMS.
The authors thank Sherrie Kaplan, PhD, and Pam Wolfe, MA, MS, for advice and comments on statistical design; Lori Lee, MS, for data analysis; and David Rodbard, MD, Sheila Roman, MD, MPH, and Alan Silver, MD, MPH, for manuscript review.
Address correspondence and reprint requests to James M. Turpin, Texas Medical Foundation, 901 Mopac Expressway South, Barton Oaks Plaza II, Suite 200, Austin, TX 78746. E-mail:.
Received for publication 30 January 2002 and accepted in revised form 11 September 2002.
Additional information for this article can be found in an online appendix at http://diabetes.diabetesjournals.org.
The content of this manuscript does not necessarily reflect the view of policies of the Department of Health and Human Services or the Centers for Medicare and Medicaid Services. The opinions expressed are those of the authors, who assume full responsibility for the accuracy and completeness of the ideas presented.
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.
- DIABETES CARE