Translating Research to Practice
Lessons learned, areas for improvement, and future directions
Lessons learned, areas for improvement, and future directions
- DPP, Diabetes Prevention Program
- RE-AIM, Reach, Effectiveness, Adoption, Implementation, and Maintenance
Diabetes and its care have captured the attention of clinicians, managed care, regulatory agencies, and the media. Several successful trials over the past decade have brought the issue of translating evidence-based findings on diabetes care into practice to the forefront of health care discussions (1–3).
Despite these promising advances, it is well documented that there is a large gap between what is known about diabetes care and what is commonly practiced (4–6). Studies of the level of diabetes care provided in the real world, and especially in primary care practices where the vast majority of patients are seen, consistently show that performance levels fall short of what is recommended (4–7). Even relatively simple actions, such as ordering a blood sample for analysis or regularly checking HbA1c, are performed far less frequently than recommended (5,6). Adherence to behaviorally oriented aspects of good diabetes and preventive care are performed even less often (with the possible exception of smoking cessation advice) (4,7).
Diabetes Care has devoted a series of articles to the discussion of translation issues and different perspectives on this topic (8–10). This article contributes to the discussion by 1) discussing changes needed in the conduct of research studies if we are to reduce the gap between research and practice, and 2) identifying specific areas for future translation research. There have been two positive examples of the adoption of research-based innovations. First, there has been a paradigm shift in the approach to self-management education and behavior change, both within diabetes education and the broader behavioral science community. This shift has been from provider-centered “compliance” approaches to more patient-centered “empowerment” methods (11–13).
The second change that has become widely adopted, at least within leading medical centers and health care plans, is “systems change” approaches to improving the delivery of evidence-based medicine (14–16). Quality improvement approaches that have proven successful often have the following characteristics: they are population based (rather than addressing only those presenting for care), proactive (rather than waiting to treat problems after they occur), and patient-centered (rather than centered on what works best for the provider team) (17–18).
In particular, the Chronic Care Model of Wagner and colleagues (16,18) has been widely adopted by a variety of health care systems, including fee-for-service, the Veterans’ Administration, and managed care organizations. Especially impressive has been the adoption of this model for rapid-cycle quality improvement by over 500 community health centers nationwide. The six key components of this model describe general evidence-based principles or actions that characterize good chronic illness care. They are organizational support, information systems, practice design, decision support, self-management support, and community resources.
Most of these approaches have been multidisciplinary, but often the primary staff member responsible for delivering intervention and monitoring guidelines achievement has been a nurse care manager (16,19). The success of the two advances above have helped move diabetes care from a perspective that blamed poor outcomes on either the patient (much of the 1970s) or the primary care provider (much of the 1980s) to one that realizes that quality care delivery is a comanagement endeavor that needs to be supported by an appropriately designed system.
Proposed model and example
Considerable challenges remain to successfully translate diabetes care research into practice. Fundamental changes will need to be made to impact the population-based, public health consequences of diabetes (10). Our research group has developed an acronym, RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance), to help researchers, program developers, and evaluators to understand and address key translation issues (Table 1) (20,21) (www.re-aim.org).
“Reach” assesses the penetration of a program into its intended target audience. It is composed of the participation rate among eligible people and the representativeness of these participants. “Effectiveness” in the RE-AIM model includes change on the dependent variable(s) or intervention targets and also impact on quality of life, including any adverse consequences. “Adoption” is similar to reach but is assessed at the level of the settings (such as clinics or organizations). It consists of the participation rate among such potential settings and their representativeness. Central to both reach and adoption is the identification of a “denominator” of eligible people or settings for use in calculating participation rate. This can be challenging, but there are multiple approaches to determine or estimate such denominators (www.re-aim.org).
“Implementation” is sometimes referred to as “intervention fidelity.” It includes the extent to which different components of an intervention are delivered as intended and the level of intervention delivery across different staff. Finally, “maintenance” has indicants at both the individual and the setting or organizational level. At the individual level, it is the long-term effects of intervention on both targeted outcomes and quality of life. At the setting level, it refers to institutionalization or the extent to which a program is sustained (or modified or discontinued) over time.
I will use the RE-AIM framework to illustrate challenges inherent in translation. Assume that there is a new program, for example, a Diabetes Prevention Program (DPP)-like program (3), that does moderately well on all RE-AIM dimensions. Assume that the intervention is moderately effective and that 40% of participants achieve significant improvements.
Next assume that, of all the health care settings in the U.S., an unrealistically large 40% agree to adopt this exciting innovation; furthermore, 40% of all the various clinicians within these settings will attempt the innovation (Table 2). (Now, 4% of the population is potentially impacted assuming, unrealistically, that all people in the U.S. have health care.) Further, assume that 40% of all patients of these clinicians will take part in the relatively intensive program. Now, reality sets in, and due to many competing demands, the average clinician is able to implement only 40% of the rather complex program components. (Now, 0.4% of the population is impacted.) Finally, assume that an amazing 40% of the patients making successful initial changes are able to maintain these improvements over time. The result is that less than two-tenths of 1% of the target population will actually benefit in a meaningful way.
The point of this exercise is not to induce pessimism, but 1) to illustrate the need to attend to all RE-AIM dimensions, not just to effectiveness of change or effect size, when selecting interventions for translation, and 2) to illustrate that if improvements were made along two or more of these dimensions, the resultant public health benefits could be dramatically increased. To date, the vast majority of diabetes research has focused on efficacy, largely ignoring other RE-AIM dimensions.
What is needed?
Reach.
Greater attention needs to be focused on developing and documenting interventions that are broadly applicable. Programs are needed that increase participation rates, especially among minority patients and those at the highest risk of diabetes complications. Also, interventions should attract those who have comorbid chronic illnesses or early complications instead of screening them out of research studies. Reach, recruitment, and retention are areas in which in-depth follow-up and qualitative research with those who decline to participate or who drop out of programs is critical.
Effectiveness.
It is especially important to recognize the importance of the cardiovascular sequelae of diabetes, and that the majority of costs and deaths from diabetes are cardiovascular related (22). Effective interventions need to improve not just microvascular outcomes, but also macrovascular, behavioral, and economic outcomes. Smoking cessation interventions have been particularly neglected, and they have been documented to be more cost-effective than almost any other health-related intervention (23). Also important are evaluations of the impact of programs on health-related quality of life (24,25). We live in a world of limited resources; therefore, making substantial investments in one area necessarily involves the opportunity cost of not being able to do other things (26,27). Improvements in quality-adjusted life-years (28) are the ultimate bottom line in diabetes care and need to be assessed more consistently.
Adoption.
With few exceptions (9, 29), the RE-AIM area that has been most neglected in diabetes research is adoption, or the percent and representativeness of settings and clinicians who will participate in a given intervention (30,31). Research needs to move from tertiary care centers and residency training settings into real-world primary care and the community (9,29,32). It is important to develop programs that administrators and clinicians in settings such as community health centers, Veterans’ Administration clinics, Indian Health Service, and rural primary care practices are willing to adopt. These are not likely to be multidisciplinary multisession intensive education programs, but rather programs that rely on continuity of care and follow-up contact.
Implementation.
There has been a recent trend toward documenting intervention fidelity, or the extent to which intervention protocols are delivered as intended. Much can be learned from studies like the DPP that involve different intervention agents. More of such research is needed to address one of the controversies in diabetes education: what types of providers can successfully and most cost-effectively deliver what types of intervention components? In “real-world” effectiveness studies and when using staff such as lay educators, or when protocols are implemented by clinic staff with many other responsibilities, it is important to document the extent to which intervention components are delivered consistently. In such circumstances, there is a risk of a “type III error” (33), i.e., concluding that an intervention is not effective when in fact it was not delivered.
Maintenance.
More attention should be devoted to the longer-term effects of programs at both the individual or patient level and at the setting level. There are many social and environmental factors that impact behavior, especially during the maintenance phase. To produce lasting effects, programs will need to better understand and address these factors. Socioeconomic status, income inequity (34), and the concept of social capital (34–36), in particular, have received recent attention because of their strong relationship to health outcomes and implications for health disparities. Social capital has been defined as “the features of social organization, such as civic participation, norms of reciprocity, and trust in others that facilitate cooperation for mutual benefit” (35). Diabetes self-management support is not a one-time endeavor like an inoculation. Rather like primary care, it needs to be ongoing as the challenges and barriers faced by people living with diabetes change over time (18,37).
At the setting level, there has been almost no research on the sustainability of diabetes programs (or other types of health promotion programs) after research projects are completed (30,31). Investigation of the natural history of program continuation, decline, or modification is needed (38, 39). Also indicated are 1) interventions, including policy actions to enhance sustainability, and 2) identification of characteristics of organizations and partnerships that successfully institutionalize programs.
In summary, to accelerate the translation of research to practice, especially needed are methods that will 1) enhance and measure the reach of interventions, especially toward poor, underserved, and minority populations; 2) develop programs that can be widely adopted by diverse settings; 3) produce replicable effects and enhance quality of life, in addition to short-term behavioral or biological outcomes; 4) be consistently implemented by different staff members having moderate levels of training; and 5) produce maintenance at both individual and setting levels and at reasonable cost.
Changes by multiple parties are necessary
To produce significant improvement, substantial changes are needed in the practices of researchers, funding agencies, and review groups. Actions by any one segment of the research community will not be sufficient to accomplish or maintain meaningful change. The preceding section discussed recommended changes in research design and methods to enhance translation to practice. Such recommendations alone, however, will have little impact unless there are also fundamental changes in the way that studies are reported, that funding agencies prioritize and review proposals, and that review groups evaluate grant proposals and journal articles (40).
Research reporting.
There are four specific reporting changes that would substantially increase the value of information for structured, evidence-based reviews and meta-analyses. These practices include reporting on 1) the percentage and representativeness of potential participants who participate versus decline and the racial/ethnic and socioeconomic status characteristics of participants versus nonparticipants, 2) manuals and training materials to facilitate replication, 3) implementation and outcomes of interventions when conducted by a range of interventionists, and 4) recruitment of settings and interventionists, including exclusion rates, participation rates, and the characteristics and representativeness of the participants.
Funding agencies.
Multiple changes are needed by funding agencies to provide incentives for researchers to change the established practices in which they have been trained (41). First, research funding agencies should explicitly request studies that evaluate interventions in multiple and representative settings. Innovation should be encouraged in terms of methods to increase reach, adoption, implementation, maintenance, and sustainability of programs. In contrast, standardization should be encouraged in the way that exclusions, participation rate, and representativeness are reported both at participant and setting levels. Finally, funding initiatives should be developed that require a maintenance/sustainability phase and provide funding for longer than the typical 2- to 5-year maximum period.
Reviewers.
The present criteria employed by review groups and intervention research quality rating systems are predominantly focused on internal validity issues to the near total exclusion of issues related to external validity (42,43) (www.cochrane.org). To facilitate more investigator involvement in translation research, grant and journal article review groups need to be open to experimental designs, in addition to randomized controlled trials (which are not the most appropriate design for every question) (44); place equal weight on internal and external validity; relax the usual editorial criteria for badly needed reports of long-term follow-ups, program sustainability, and reports on especially challenged settings and populations (which are usually more difficult to study); and consider adding an additional review criteria of “potential for translation.”
Space limitations preclude a fuller discussion of the changes recommended for each of the parties above. A more complete set of recommendations and accompanying discussion is available elsewhere (41) (www.re-aim.org).
Special opportunities
Considering the RE-AIM criteria, several areas have great potential to capitalize on recent scientific advances and produce large returns for investment of translation research effort. First, more comprehensive evaluations of interventions that assess and address the social context in which a program is delivered and its participants live, work, and receive their health care are needed (45). Health disparities are well documented, as are some of the social determinants of health (46). Especially relevant would be interventions that could enhance social capital (35,36). Physical activity researchers, in studying exercise (one of the key determinants of diabetes and its control), have recently advanced our understanding of social context and social determinants (www.alpes.us) (46), and most of these findings should be applicable to diabetes.
Geneticists ascribe the greatest percent of variance determining diabetes and most other complex illnesses to gene-environment interactions. Yet the amount of research dollars devoted to characterizing the environmental and behavioral side of this equation pales in contrast to that spent on genetics. A modest proposal might be that for the next decade, one-tenth as many dollars should be spent on environmental factors and gene-environment interactions as on genetic and pharmacogenetic applications. In particular, there are many opportunities for therapeutic applications of risk perception (47,48) and shared decision making (49).
Primary prevention of diabetes is near the top of the list of future opportunities given the impressive results of the DPP (3). There are many interpretations of how to apply the knowledge learned from this study (3,10). Mine is that we need to investigate stepped-care programs and other less costly approaches for application of the behavioral science principles involved in the DPP intervention that may have higher reach and are likely to enhance sustainability. The Diabetes Control and Complications Trial (1) taught us that intensive management of glucose levels resulted in decreased microvascular complications, but it did not teach us how to implement these procedures on a broader basis. Similarly, the DPP taught us that behavioral science–based lifestyle change interventions can dramatically reduce the incidence of diabetes (as can metformin if adhered to consistently) but not how to translate these findings into practice.
We live in an information age and interactive computer technologies certainly offer promise if developed with appropriate attention to translation issues. Such approaches are very scalable and this technology has immense potential if it is used for informing and facilitating, rather than attempting to replace, human interactions. It is worth noting that to date, the interactive computer technology that has the best evidence base and meets the highest ratings on RE-AIM criteria for potential translation is somewhat counter-intuitive: automated telephone-based intervention (21,50,51). Future investigations of interactive technologies that are integrated with and can inform diabetes care and self-management could lead to significant advances.
The setting in which many (but not all) of the innovations above should be centered is the health care system and primary care, in particular. Progress has been made in the level of care provided, especially in systems such as the Indian Health Service, the Veterans’ Administration, the community health center system, and many managed care organizations (14,15, 52–54). However, much remains to be done. Research is needed to identify the characteristics of health systems that lead to broader, more rapid, and more sustained quality improvement. Second, more studies are needed that truly integrate different aspects of the Chronic Care Model. Improvements in care are especially likely to come from policy and social environmental interventions combined with Chronic Care Model interventions, particularly if self-management support and community resources are integrated into regular care (37,54).
Concluding thoughts
The recommendations above may seem overly ambitious. However, there is a very encouraging and illustrative real-life example. Millions of U.S. smokers have quit with the support of health care providers, backed by health care systems (and the pharmaceutical industry and supportive health policies), to implement evidence-based interventions. These practical interventions were developed by researchers who were supported by private, state, and federal funding. Most reviews have concluded that the intervention component most effective in producing lasting reductions in smoking prevalence, however, has been a supportive social environment including cigarette taxes and clean indoor air policies.
There are also lessons from how smoking cessation resources have been implemented. Many systems have addressed smoking cost-effectively by using a sequential stepped care model (Fig. 1) that begins with interventions, such as policies and media, that have low cost/intensity and that high reach, impacting almost the entire population (55). A second step, for those who do not succeed with the first-level intervention, is often to have them attend community or worksite programs that have a somewhat greater cost (and less overall reach). For those who are still not successful, provider-supported pharmacological intervention or referral to smoking cessation specialists may be required. Note that such a system does not begin with the most effective (and costly) interventions due to reasons of overall reach and cost. This type of thinking could also benefit diabetes care.
In conclusion, coordinated and substantial change will be required from all of the various stakeholders in diabetes care research to accelerate translation of research into practice. This not only includes the clinicians and researchers who develop and use interventions, but also patients (and patient-centered innovations) and health systems that provide resources and incentives to make these changes more population-wide. Change is also needed by policy makers and various government agencies to provide the necessary conditions (such as grant funding and incentives to do the right thing). With a shift in research priorities, such as that which occurred in smoking cessation research over the past 2 decades (55,56), there is reason for optimism. By broadening our research focus to address external validity issues as well as internal validity, much can be done to improve public health impact via improved reach, adoption, effectiveness, implementation, and maintenance (as in the example and Table 2) (41).
Multilevel pyramid model of stepped care interventions.
Recommendations for RE-AIMing translation research
Stages of translating an efficacious program into reality
Acknowledgments
The preparation of this article was facilitated by Agency for Healthcare Research and Quality Grant no. 1 RO1 HS10123 and National Institute of Diabetes, Digestive and Kidney Disease Grant no. 2 RO1 DK35524.
Drs. Marshall Chin and Paul Nutting are appreciated for their helpful comments on an earlier version of the manuscript.
An earlier version of this paper was presented at a meeting of the Diabetes Mellitus Interagency Coordinating Committee on “The Science of Translation Research: Outcomes and Opportunities,” Bethesda, MD, 17 September 2002.
Footnotes
-
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.
- Accepted April 18, 2003.
- Received November 25, 2002.
- DIABETES CARE
References
