|
 |
|
|||||||
|
|||||||||
© 2005 by the American Diabetes Association, Inc.
Letters: Comments and Responses |
Guidelines for Computer Modeling of Diabetes and Its Complications
Response to American Diabetes Association Consensus Panel
1 Economics Division, University of Liverpool Management School, Liverpool, U.K
2 School of Mathematics, Cardiff University, Cardiff, U.K
Address correspondence to Prof. A. Bagust, Economics Division, University of Liverpool Management School, Chatham Street, Liverpool, U.K. L69 7ZH . E-mail: a.bagust{at}liv.ac.uk
The ADA consensus panel guidelines for computer modeling (1) may create a misleading and unduly optimistic view of modeling for general readers. Repeated use of the terms "accurate" and "reliable" suggests that models are capable of generating precise predictions of long-term clinical outcomes and costs. Sadly, models do no more than encompass our current understanding (and ignorance) of a disease, its treatment , and its natural history. Despite powerful computers and sophisticated software, no model generates new data—it merely combines existing findings within a framework of human assumptions and generalizations. The modern era of diabetes modeling is only 7 years old (2) and, despite strenuous efforts in four Mount Hood challenges (3), shows no evidence of achieving convergent accuracy.
Readers should not be misled into concluding that "validation" confers "validity" on a model’s results. Internal validation is a very low standard, only requiring a model to reproduce the data originally used in its calibration. Even successful replication of an independent result (external validation) may only be a "lucky hit" and cannot guarantee that model predictions for longer timescales, different patient groups, or other clinical settings are any more accurate or reliable than long-range weather forecasts.
The discussion of uncertainty in the guidelines emphasizes parameter variability and thereby masks the main cause of imprecision. A model’s Achilles’ heel lies in the assumptions and design choices governing how numerical values are generated. Experience shows that small differences in functional forms, when extrapolated, often lead to widely differing predictions, yet these qualitative uncertainties are rarely discussed.
Despite these cautions, researchers in medicine, epidemiology, economics, and operations research have made important progress in developing various types of diabetes models for different purposes, including individual patient management, public health policy, service planning, and economic evaluation. However, diabetes presents the modeling community with its greatest test, encompassing probably the widest range of serious comorbid interacting conditions of any chronic disease. Many modeling problems (practical and methodological) remain unresolved for even the "simple" conditions, such as terminal cancer. Although the progress in diabetes modeling is encouraging, it should not be overstated, and those outside the modeling community should not be misled into believing that model predictions constitute accurate and reliable evidence similar to a clinical trial. A model is a tool to help decision makers explore various aspects of their dilemma, not an objective mechanism to relieve them of responsibility for weighing the evidence. Greater clarity in how models and results are reported is more helpful here than general prescriptive pronouncements.
Footnotes
A.B. has received research funding from GlaxoSmithKline. P.M. has received research funding from AstraZeneca.
References
- American Diabetes Association Consensus Panel: Guidelines for computer modeling of diabetes and its complications (Consensus Statement). Diabetes Care 27:2262–2265, 2004
[Free Full Text]
- Eastman RC, Javitt JC, Herman WH, Dasbach EJ, Zbrozek AS, Dong F, Manninen D, Garfield SA, Copley-Merriman C, Maier W, Eastman JF, Kotsanos J, Cowie CC, Harris M: Model of complications of NIDDM. I. Model construction and assumptions. Diabetes Care 20:725–734, 1997[Abstract]
- Brown JB, Palmer AJ, Bisgaard P, Chan W, Pedula K, Russell A: The Mt. Hood challenge: cross-testing two diabetes simulation models. Diabetes Res Clin Prac 50 (Suppl. 3):S57–S64, 2000[Medline]
CiteULike 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Diabetes | Diabetes Care | Clinical Diabetes | Diabetes Spectrum |