Ny studie kring modellering av typ-2 diabetes

Studien är en jämförelse av kohort- och mikrosimuleringsmodeller inom kostnadseffektivitets modellering av typ-2 diabetes. I artikeln som är publicerad i PharmacoEconomics presenteras en fallstudie med IHEs kohortmodell, the Diabetes Cohort Model (IHE-DCM), och mikro-simuleringsmodellen ECHO-T2DM.  

 

Läs mer och ladda ner artikeln här, free pdf

https://ihe.se/en/publicering/comparing-the-cohort-and-micro-simulation-modeling-approaches-in-cost-effectiveness-modeling-of-t2dm/

 

Comparing the Cohort and Micro-Simulation Modeling Approaches in Cost-Effectiveness Modeling of Type 2 Diabetes Mellitus: A Case Study of the IHE Diabetes Cohort Model and the Economics and Health Outcomes Model of T2DM

Willis M, Fridhammar A, Gundgaard J, Nilsson A, Johansen P

Economic modeling is widely used in estimating cost-effectiveness in type 2 diabetes mellitus. Because type 2 diabetes is complex and patients are heterogenous, the cohort modeling approach may generate biased estimates of cost-effectiveness. The IHE Diabetes Cohort Model (IHE-DCM) was constructed using the cohort approach as an alternative for stakeholders with limited resources, some of whom have voiced reasonable concerns about a lack of transparency with type 2 diabetes micro-simulation models and long run times.

The objective of this study was to inform decision makers by investigating the direction and magnitude of bias of IHE-DCM cost-effectiveness estimates that can be attributed to the cohort modeling approach.

Simulation scenarios inspired by the 9th Mount Hood Diabetes Challenge were simulated with IHE-DCM and with a micro-simulation model, the Economic and Health Outcomes Model of T2DM (ECHO-T2DM), and key metrics (absolute and incremental costs and quality-adjusted life-years, event rates, and cost-effectiveness) were compared for evidence of systematic differences. The models were harmonized to the extent possible to ensure that differences were driven primarily by the unit of observation and not by other model differences.

IHE-DCM run times were faster and IHE-DCM produced uniformly larger estimates of absolute life-years, quality-adjusted life-years, and costs than ECHO-T2DM but smaller between-arm (incremental) differences. Estimated incremental cost-effectiveness ratios and net monetary benefits varied similarly and predictably across the scenarios. On average, IHE-DCM estimates of incremental cost-effectiveness ratios and net monetary benefits were CAN$269 (3%) and CAN$2935 (10%) smaller, respectively, than ECHO-T2DM.

There was little evidence that estimated cost-effectiveness metrics, the outcomes that matter most to stakeholders, differed systematically.

Läs mer och ladda ner arrtikeln

https://link.springer.com/article/10.1007/s40273-020-00922-6

Comparing the Cohort and Micro-Simulation Modeling Approaches in Cost-Effectiveness Modeling of Type 2 Diabetes Mellitus: A Case Study of the IHE Diabetes Cohort Model and the Economics and Health Outcomes Model of T2DM

PharmacoEconomics (2020)

Abstract

Background

Economic modeling is widely used in estimating cost-effectiveness in type 2 diabetes mellitus. Because type 2 diabetes is complex and patients are heterogenous, the cohort modeling approach may generate biased estimates of costeffectiveness. The IHE Diabetes Cohort Model (IHE-DCM) was constructed using the cohort approach as an alternative for stakeholders with limited resources, some of whom have voiced reasonable concerns about a lack of transparency with type 2 diabetes micro-simulation models and long run times.

Objectives

The objective of this study was to inform decision makers by investigating the direction and magnitude of bias of IHE-DCM cost-effectiveness estimates that can be attributed to the cohort modeling approach.

Methods

Simulation scenarios inspired by the 9th Mount Hood Diabetes Challenge were simulated with IHE-DCM and with a micro-simulation model, the Economic and Health Outcomes Model of T2DM (ECHO-T2DM), and key metrics (absolute and incremental costs and quality-adjusted life-years, event rates, and cost-effectiveness) were compared for evidence of systematic differences. The models were harmonized to the extent possible to ensure that differences were driven primarily by the unit of observation and not by other model differences.

Results

IHE-DCM run times were faster and IHE-DCM produced uniformly larger estimates of absolute life-years, quality-adjusted life-years, and costs than ECHO-T2DM but smaller between-arm (incremental) differences. Estimated incremental cost-effectiveness ratios and net monetary benefits varied similarly and predictably across the scenarios. On average, IHE-DCM estimates of incremental cost-effectiveness ratios and net monetary benefits were CAN$269 (3%) and CAN$2935 (10%) smaller, respectively, than ECHO-T2DM.

Conclusions

There was little evidence that estimated cost-effectiveness metrics, the outcomes that matter most to stakeholders, differed systematically.

Key Points

 
 
 
 
Efficiently allocating scarce resources for chronic and progressive diseases such as type 2 diabetes mellitus (T2DM) is challenged by limited time and resources and an unusual degree of decision-making uncertainty (e.g., clinical and economic implications that extend far beyond trial durations, patient heterogeneity, evolving practice patterns, and practice patterns that differ between trials and ordinary use).
To extrapolate trial data to longer decision-making time horizons, economic modeling is routinely used. While economic models of T2DM would ideally be user friendly, transparent, fast, and accurate (i.e., good external validity), the complexity of T2DM generally requires comprehensive (including parallel sets of complications and sophisticated treatment-switching algorithms) to ensure good predictive accuracy. Established T2DM models are generally slow and relatively opaque, which imposes an additional demand on economic stakeholders for case-specific expertise to evaluate the suitability of manufacturer-submitted models and in some cases to run the models with tight deadlines.
To address a need that some economic stakeholders have for greater user friendliness and faster run times, the IHE Diabetes Cohort Model was constructed using the cohort rather than the micro-simulation approach. A well-known limitation of cohort modeling, however, is an inability to adequately model patient heterogeneity (at least not without a health state explosion) and a potential for biased cost-effectiveness estimates.
In exercises designed to evaluate the potential magnitude of bias of the IHE Diabetes Cohort Model, we compared results generated for a set of simulation scenarios with those of a micro-simulation model (Economic and Health Outcomes Model of T2DM), chosen because the structures are otherwise generally similar and because it was possible to harmonize the models even more to minimize between-model simulation differences. We found systematic differences in simulated costs and quality-adjusted life-years, but little evidence of systematic differences in the incremental costs and quality-adjusted life-years that underlie cost-effectiveness metrics or in incremental cost-effectiveness ratios and net monetary benefits themselves.
 
From the article

Discussion

Using well-established cross-validation tools [11] modified to allow structural standardization of the models, we examined whether IHE-DCM produces systematically biased estimates of cost-effectiveness related to the cohort approach. In a simple Reference Case performed to enable comparison with the results of 11 other models that participated in the 9th Mount Hood Diabetes Challenge, IHE-DCM produced consistently greater absolute survival, QALYs, and costs than ECHO-T2DM, which is consistent with the difference between modeling homogenous patients and heterogeneous patients when event risks are non-linear (specifically convex) in key parameters [16]. Between-model differences were generally small at the incremental level (i.e., different between the two comparator arms) used to construct cost-effectiveness metrics, however, and the ICER and NMB, which were also similar between models. As expected, IHE-DCM was considerably faster compared with ECHO-T2DM, with a run time of approximately 45 min compared with 30 h using ECHO-T2DM, an important aspect for many stakeholders under time constraints.

This same pattern was observed for the more realistic Expanded Reference Case and 18 scenario analyses, and both models responded to changes in model parameters similarly and predictably. This was supported statistically; incremental costs, incremental QALYs, and NMBs for each model fell uniformly within the 95% confidence interval generated by the other model. There was more uncertainty in the results of IHE-DCM, which was driven in large part by uncertainty in the parameter estimate for the hypoglycemia event rate (eliminating it roughly halved the confidence interval). The estimates of ECHO-T2DM falls within even half of the 95% confidence intervals generated by IHE-DCM. Estimates in the base case by IHE-DCM of the 40-year cumulative incidence of study outcomes, moreover, fell within the 95% confidence intervals generated by ECHO-T2DM. While the paired t tests did find statistically significant between-model differences in incremental costs, incremental QALYs, and the NMB for these 19 scenarios, the paired t test is grossly overpowered to reject the null hypothesis in this setting as the simulation scenarios (i.e., the sample draws) are not independent of each other. Interestingly, however, the paired t test failed to reject between-model differences for the ICER (p < 0.68) for the 18 scenarios for which both incremental costs and incremental QALYs were positive (producing a meaningful ICER). Further underscoring this absence of clear bias in cost-effectiveness estimates, there was no discernible pattern as to which model produced more favorable cost-effectiveness estimates, with each more favorable in roughly half of the scenarios.

The trade-off between cohort modeling and micro-simulation is sometimes (perhaps mistakenly) cast as a choice between time and transparency vs accuracy. Both models satisfy International Society for Pharmacoeconomics and Outcomes Research recommendations for model transparency, which accept complexity and call instead for a technical report that describes the structure, components, equations, and computer code that would enable experts to reproduce the model (full technical transparency) and non-technical documentation that, at a minimum, describes the type of model and intended applications, funding sources, model structure, inputs and outputs, data sources, model validation, and model limitations [11]. While transparency in a general sense is hard to quantify, and no fit-for-purpose diabetes models are likely to achieve “transparency” in a general sense, analysts (authors AN and AL) generally considered that IHE-DCM was easier to grasp and work with (and is constructed with approximately 50% fewer lines of code).

This analysis has several strengths, including the use of two models that were relatively similar and required limited standardization. Many of the remaining differences could be standardized to minimize the extent that differences would be driven by model differences other than units of observations. The scenarios were inspired by the Mount Hood Reference Case, which permits comparison (at least of the Reference Case results) with 11 health economic models of diabetes that participated in the 9th Mount Hood Diabetes Challenge Network. Finally, a wide range of scenarios was considered that explored different aspects of the model to enhance generalizability.

The models could not be entirely standardized, however, and remaining differences must be considered when interpreting the results of this analysis (i.e., between-model differences may reflect more than just the potential bias related to cohort vs micro-simulation modeling). In particular, the main structural difference is the modeling of CKD, for which there are different methods of simulating disease progression (transition probability vs biomarker driven) and which clearly impact the results. Indeed, for the cumulative incidence, the CKD outcomes (micro-and macroalbuminuria and end-stage renal disease) were clear outliers and the mean estimates for the IHE-DCM model were just within the 95% confidence interval of ECHO-T2DM. Foot ulcer is included only in ECHO-T2DM. To mitigate the impact on the analysis, costs and QALY weights were set to 0. The indirect impact on overall results was limited because foot ulcer affected only the risk of congestive heart failure (though patients simulated to develop congestive heart failure had in turn increased risks for ischemic heart disease, myocardial infarction, and mortality) and the simulated incidence of foot ulcer was low. Second, while the scenarios were constructed to mimic a cost-effectiveness analysis, the simulated scenarios are purely hypothetical.

While this study cannot provide a definitive (and universal) answer to concerns about possible bias, and it does not address the academic discussion of how much accuracy is reasonable to swap for increased transparency [30], this exercise provides a careful examination of how two otherwise similar models respond to the same set of stimuli (both absolutely and incrementally), which can be valuable for stakeholders charged with interpreting evidence produced by IHE-DCM.

Conclusions

The IHE-DCM was faster to load and to run than the micro-simulation model used in this study (ECHO-T2DM) and the modeling details are likely to be more easily understood by external reviewers, which can be an advantage for economic stakeholders with limited time and resources. Despite systematic differences in absolute predicted survival, QALYs, and costs, estimated cost-effectiveness metrics were similar suggesting that any bias related to the cohort approach is small in the outcomes that matter most. We believe that both models are suitable for use in cost-effectiveness evaluations for interventions in T2DM; the selection of one over the other should be made on the basis of stakeholder needs, resources, and preferences.

 

Nyhetsinfo

www red DiabetologNytt