I dagarna har IHEInstitiutet för Hälsoekonomi Malmö även publicerat en ny valideringsartikel för the Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM), en mikrosimuleringsmodell för typ 2 diabetes.
Validation of the Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM)
Willis M, Johansen P, Nilsson A and Asseburg C
Type 2 diabetes mellitus (T2DM) is chronic and progressive disease and economic evaluation requires that the benefits and costs of treatment interventions be captured over the long run using economic simulation modeling techniques. From a modeling standpoint, T2DM ranks among the most challenging disease areas due to its impact on multiple inter-related organ systems and multiple treatment goals (including blood glucose, blood pressure, and blood lipids).
The usefulness of a model depends, naturally, on its ability to accurately predict health and economic outcomes of patients in real-life treatment settings and users of model results need to be assured of the soundness of model predictions. External validation is defined in applicable guidelines as replicating clinical trials or data registries with a model and comparing the model predictions with the actual observed results (ISPOR/SMDM). Models of T2DM have been at the forefront of model validation, in part because modeling the complex multi-organ pathophysiology usually requires advanced programming (often using compiled code) resulting in reduced transparency.
ECHO-T2DM (v. 2.3.0) is a stochastic, micro-simulation (i.e., patient-level) model, suitable for estimating long-term cost-effectiveness of T2DM interventions. Recent model upgrades created the need for a new model validation. We followed the principles espoused by ISPOR/SMDM. Specifically, study characteristics were entered into ECHO-T2DM to replicate 12 clinical studies (including 17 patient populations). Model predictions for 202 study endpoints were then compared to observed values using established statistical techniques. Sub-group analyses were conducted separately endpoints from studies used to construct the model and those independent of the model and by type of endpoint (microvascular, macrovascular, or mortality).
Using established statistical techniques, model predictions agreed generally well with observed clinical endpoints. Plotting predictions versus observed values, for example, found that most data points lie close to the 45-degree line, which corresponds to perfect match, and there was no clear pattern for under- vs. over-prediction. The R2 value was about 0.90 and an F test of perfect match could not be rejected, indicating good fit. Good fit was found for both dependent and independent endpoints, separately. Despite smaller sample sizes, fit was also reasonably good for individual types of endpoints as well, albeit with a tendency to over-predict mortality and rejection of the F test for macrovascular endpoints. ECHO-T2DM continues to match health outcomes in clinical trials in T2DM, with prediction accuracy similar to other leading models of T2DM.
Pharmacoeconomics. 2016 Nov 12. [Epub ahead of print]
Validation of the Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM).
Willis M1, Johansen P2, Nilsson A2, Asseburg C2.
The Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM) was developed to address study questions pertaining to the cost-effectiveness of treatment alternatives in the care of patients with type 2 diabetes mellitus (T2DM). Naturally, the usefulness of a model is determined by the accuracy of its predictions. A previous version of ECHO-T2DM was validated against actual trial outcomes and the model predictions were generally accurate. However, there have been recent upgrades to the model, which modify model predictions and necessitate an update of the validation exercises.
The objectives of this study were to extend the methods available for evaluating model validity, to conduct a formal model validation of ECHO-T2DM (version 2.3.0) in accordance with the principles espoused by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM), and secondarily to evaluate the relative accuracy of four sets of macrovascular risk equations included in ECHO-T2DM.
We followed the ISPOR/SMDM guidelines on model validation, evaluating face validity, verification, cross-validation, and external validation. Model verification involved 297 'stress tests', in which specific model inputs were modified systematically to ascertain correct model implementation. Cross-validation consisted of a comparison between ECHO-T2DM predictions and those of the seminal National Institutes of Health model. In external validation, study characteristics were entered into ECHO-T2DM to replicate the clinical results of 12 studies (including 17 patient populations), and model predictions were compared to observed values using established statistical techniques as well as measures of average prediction error, separately for the four sets of macrovascular risk equations supported in ECHO-T2DM. Sub-group analyses were conducted for dependent vs. independent outcomes and for microvascular vs. macrovascular vs. mortality endpoints.
All stress tests were passed. ECHO-T2DM replicated the National Institutes of Health cost-effectiveness application with numerically similar results. In external validation of ECHO-T2DM, model predictions agreed well with observed clinical outcomes. For all sets of macrovascular risk equations, the results were close to the intercept and slope coefficients corresponding to a perfect match, resulting in high R 2 and failure to reject concordance using an F test. The results were similar for sub-groups of dependent and independent validation, with some degree of under-prediction of macrovascular events.
ECHO-T2DM continues to match health outcomes in clinical trials in T2DM, with prediction accuracy similar to other leading models of T2DM.
Läs mer om hälsoekonomiska simuleringsmodeller (PDF)
Hälsoekonomiska simuleringsmodeller är vik ga verktyg för kostnads- e ek vitetsanalyser av nya läkemedel och medicinska teknologier. IHE har mångårig erfarenhet av att utveckla hälsoekonomiska modeller inom flera sjukdomsområden såsom diabetes, cancer och Parkinsons sjukdom.
Modellutformningen bestäms av modellanalysens natur och komplexa simuleringar blir allt vanligare, vilket i sin tur ställer ökade krav på mo- dellernas transparens och validitet. Nyligen publicerades därför en arti- kel som validerar the IHE Cohort Model of Type 2 Diabetes, en kohortmodell utvecklad av IHE för hälsoekonomiska analyser av typ 2 diabetes (sida 4-5).
Publicerad online: 12 november 2016
Publicerad online: 12 november 2016
www red DiabetologNytt