Impact of Self-Reported Patient Characteristics Upon Assessment of Glycemic Control in the Veterans Health Administration

RESEARCH DESIGN AND METHODS

Data sources and cohort identification

The Large Veterans Health Survey (LVHS), a weighted national representative survey of about 1,400,000 VHA enrollees was conducted in the summer of 1999 (13) with a response rate of 63.1% (n = 887,775). In this survey 190,374 individuals reported being told by a doctor that they had diabetes. We merged utilization and laboratory data available in VHA administrative databases for this cohort as previously described (14). Briefly, inpatient and outpatient utilization data and ICD-9 codes were obtained from the National Patient Clinical Dataset (Austin, TX), and laboratory data were obtained from the VA Healthcare Analysis Information Group.

To approximate NCQA criteria for indemnity plan member inclusion in Healthcare Employee Data Information Set reporting (15), we included only veterans who had at least two diabetes-related (250.x diagnostic code) visits with a clinician in the VHA health care system in fiscal year 1998 and/or fiscal year 1999 and excluded individuals whom we identified as being deceased before 1 October 1999, using the VHA Beneficiary Identification and Records Locator System and the Medicare Denominator File (13). We identified 132,076 subjects who met these criteria.

To minimize any impact of difference in A1C methodologies, we eliminated medical centers that used A1C methodologies in fiscal year 2000 that were not certified by the National Glycohemoglobin Standardization Program as previously described (16). Although the Health Plan Employee Data and Information Set (HEDIS) measures for glycemic control count A1C not measured as being greater than the threshold value, we also excluded individuals at remaining medical centers who did not have laboratory tests performed in the VHA both to focus our study upon the influence of patient characteristics on glycemic control and because administrative data could not capture laboratory tests performed outside the VHA and recorded in progress notes. This resulted in exclusion of 54,460 subjects.

Of the remaining 77,616 individuals, we excluded 20,876 who did not answer questions on the LVHS that were included in our analysis (Table 1). Our final study population consisted of 56,740 VHA clinical users with diabetes at 105 medical centers. We compared baseline characteristics of veterans with diabetes in our final study cohort with those who were excluded as described above. The VA New Jersey Healthcare System Institutional Review Board approved this study.

Model development

Variables used in risk adjustment.

We selected LVHS variables that reflect characteristics largely outside the control of a health care plan yet are known to be associated with adherence to intensive treatment and improved outcomes (12). Demographic variables included age and sex. Health status variables included physical health and mental health component scores measured by the Veterans Short Form-36. Health behavior included smoking, alcohol use, and exercise level. Social support variables included marital status and living arrangements (living alone or not). Socioeconomic status variables included education level, employment, and extreme economic hardship. The latter was assessed by a food sufficiency question that asks “In the past 30 days have you been concerned about having enough food for you or your family?” with yes or no responses. Other variables included duration of diabetes and height and weight (enabling calculation of BMI).

Modeling.

We used general linear regression models to develop a risk adjustment model for individual A1C levels. We retained variables that were significant at the P < 0.05 level in the final model. Our comparison model included only age and sex (17) to evaluate the marginal impact of the variables of interest upon glycemic control and medical center level performance.

Medical center profiling.

We evaluated the impact of risk adjustment on medical center profiling, separately for each of two A1C thresholds (<7 and <8%), by comparing ranks from before and after risk adjustment. Whereas the <7% threshold has always been considered the target goal for glycemic control, the 8% level was consistent with 1999–2000 American Diabetes Association Clinical Practice Recommendations (18) for taking action. An individual subject met adherence criteria to an individual measure if the last A1C level achieved in fiscal year 2000 was below the threshold.

To determine the ranks for unadjusted A1C values among medical centers, we first identified individuals at each medical center whose A1C values were below the unadjusted A1C threshold (the “observed”). We then used the total number with diabetes at that medical center to generate a proportion and ranked medical centers on these proportions of observed patients (19). To determine the ranks using adjusted A1C values, we used two different models: Model 1 included only age and sex as independent variables; model 2 included all the variables described above. For each of the models, we performed the following steps. First, we calculated the percentage of individuals in the entire study population (105 medical centers) with A1C values below the threshold of observed A1C. Next, we identified individuals with adjusted A1C values at or below the corresponding population-level percentage for each threshold. Of these individuals, we then counted the number of individuals at each medical center (the “expected”). This information was used to generate observed-to-expected ratios for each medical center, and medical centers were then ordered on these ratios to obtain risk-adjusted ranks. We repeated this process for both 8 and 7% thresholds of A1C. For each medical center, six unique ranks were possible: by both 7 and 8% thresholds, ranking based on unadjusted ratios and ranking based on two adjusted models with age and sex in the first and all socioeconomic self-reported variables already described.

Because the use of best and worse decile rankings is one industry standard for identifying best and worse health plans (2), we ranked VHA medical centers into deciles and used league tables to determine the degree of ranking movement for medical centers in the best and worse two deciles for both the <7 and <8% threshold after risk adjustment. The degree of movement was evaluated by shifts in decile ranks. We reported the number of medical centers that changed decile ranks and the magnitude of the change.

RESULTS—

The individuals in the study sample with complete data were mostly male (mean 98%) and elderly (mean age 64.6 years). Individuals who were included in the analysis were slightly younger than those who were excluded (19.7 vs. 15.2% <55 years of age), more likely to be married (65.5 vs. 60.3%), and better educated (Table 1).

There were marked differences in the socioeconomic status and health-related characteristics of the patient populations among VHA medical centers (Table 2). For example, there were substantial variations in having completed only grade school education (mean 15.3% [range 2.3–32.7%]), being retired (38.3% [19.9−59.7%]) or married (65.2% [43.7–77.8%], reporting concern about food sufficiency (13.9% [7.2–24.6%]), or reporting no exercise (43.2% [31.1–53.6%]). There was also variation in self-reported health status as assessed by the mental health component score (43.8 [38.7–48.9]) and the physical health component score (31.4 [27.6–36.4]) of the Veterans Short Form-36. Mean medical center levels of A1C varied from 7.14 to 8.58%, and the proportion of individuals achieving A1C levels <7 and <8% varied from 23 to 56% and from 44 to 79%, respectively.

Variables and interaction terms that contributed significantly to variations in A1C and were retained in the final model included age, marital status, BMI, duration of diabetes, education level, employment status, concern for food sufficiency, smoking frequency, and exercise frequency (Table 2). The R² of this model was 7.8%. Being older, being female, having a higher education level, and exercising more frequently were associated with having lower A1C levels. Longer duration of diabetes and higher BMI were associated with higher A1C levels.

Medical center rankings from the <7 and <8% thresholds were highly correlated (Spearman rank coefficient unadjusted 0.89 [P < 0.001] and adjusted 0.85 [P < 0.001]). Medical center rankings using unadjusted and adjusted values were also highly correlated, although somewhat less so (Spearman rank coefficient 0.71 [P < 0.001] for <7% and 0.64 [P < 0.0001] for <8%).

There were changes in medical center ranks when the models using all available variables were compared with the age- and sex-adjustment models. For example, the top and bottom 20% (two deciles each) had 21 medical centers ranked. For the <7% threshold (Fig. 1A), 4 medical centers that initially ranked in the best decile in the model with age and sex moved down two deciles, no longer ranked in the best decile after adjustment using the full model. On the lower end, two medical centers ranked in the worst decile when using the age- and sex-adjusted model improved by one decile as a result of using the full model. For the 8% threshold (Fig. 1B), two medical centers moved from the best decile (shifting two deciles) and three medical centers moved from the worst decile (also shifting one decile) using the full model compared with using the age- and sex-adjusted model only.

For the <7% threshold measure, we compared the means of age, sex, and self-reported variables for those facilities (n = 4) changing two deciles with those for the other facilities (n = 7). However, despite considerable differences among facilities, we were unable to demonstrate differences in the group means of any individual variable (data not shown).

CONCLUSIONS—

Our study demonstrated that patient characteristics, which can influence glycemic control but are largely outside of the control of health providers or health plans, can change the identification of best- and worse-performing medical centers and thus could potentially change quality assessments by internal and external stakeholders. Self-reported socioeconomic status, health status, and health behaviors varied widely among VHA medical centers but explained only a small proportion of A1C variation. Nonetheless, when these characteristics were used to risk adjust A1C levels at <7 or <8%, about 29 and 24%, respectively, of medical centers initially identified as best and worse medical centers shifted out of these categories. Whereas <7% is considered a threshold for “excellent control” (10), an A1C threshold >8% has been proposed as a threshold by which to assess clinical inertia (20,21). Our findings demonstrate the importance of taking into account socioeconomic position factors when ranking systems of health care for public reporting on the basis of A1C levels lower than the current >9% threshold for poor glycemic control.

Our observations that patient-level factors such as fewer years of completed education, concerns over food sufficiency, less social support (unmarried or living alone), and unemployment were associated with worse glycemic control in the veteran population are consistent with the literature (12). Similarly, the observation that <10% of the variance in A1C levels could be explained by these variables is also consistent with prior studies demonstrating a poor explanatory value for individual-level variables on glycemic control (22–24). These findings extend recent findings (25) that poorer individuals enrolled in managed health care plans had slightly higher A1C values (8.1%) than those with higher incomes (7.8%). However, in contrast to our findings, there were no differences in glycemic control by education. It is important to note that none of these previous studies evaluated the impact of patient-level socioeconomic characteristics upon quality assessment of different health care plans.

It is unclear how these variables interact to impact glycemic control. For example, socioeconomic barriers could lead to belief systems or attitudes (26) that impact with provider-patient communication regarding glycemic goal setting (27). Alternatively, it may be more difficult to make healthy food choices or have an environment in which to exercise, thus impeding progress in achieving glycemic goals (12). Regardless of the mechanisms, our empirical findings indicate the difficulties in generalizing from landmark clinical trials with activated patients to real-world settings with more heterogeneous populations (28) for the purpose of public reporting and payment.

Our findings are most immediately relevant to the recent decision by NCQA to implement a public reporting measure using a <7% threshold. Although the NCQA groups health care plans into Commercial, Medicaid, and Medicare enrollment status, this level of aggregation is unlikely to be sufficient to control for differences in socioeconomic characteristics among plan enrollees. Whether or not health plans should be further stratified by rural or urban location or patient income and educational status needs to be considered. Geocoding patient addresses from enrollment data to the census block group level could be used as an alternative to individually collected data (29), although this approach is admittedly an approximation. Alternatively, there could be differential weighting of adherence to the <7 and <9% thresholds, as is done in the Bridges to Excellence Program at the physician practice level (30) to reflect the fact that “optimal” control is less under physician control than “poor ” control.

The strengths of our study were the ability to link survey and administrative data based on a large national sample of health care systems with at least 100 individuals with diabetes per medical center (31). We were also able to ascertain that each medical center in the study performed A1C testing with a methodology certified by the National Glycohemoglobin Standardization Program. We also recognize limitations of our study. The veteran population, predominantly male, tends to be older and have more comorbid conditions than the general population. On the other hand, because veterans who use the VHA tend to be of lower socioeconomic status than the general public (32), variations in socioeconomic status may be more marked among private sector health plans. Consequently, our findings need to be examined in other populations to ensure generalizability.

In summary, the socioeconomic status and health-related characteristics of VA medical center populations had a substantial impact on medical center rankings for quality of diabetes care. Our findings suggest that if an A1C <7% measure is to be used for public reporting of plans and therefore presumably for physician accountability within plans, then adjustment for patient socioeconomic positioning status may be necessary to assure that inferences regarding plan or physician group level performance in controlling glycemia are valid and do not have an unfair impact on payment that could lead to unintended consequences such as adverse selection at the physician level (5).