- Papatheodoridis G.V.
- Lekakis V.
- Voulgaris T.
- Lampertico P.
- Berg T.
- Chan H.L.Y.
- et al.
- 1)We fully agree with authors that the strength of evidence is considered high when no significant statistical heterogeneity is found among pooled studies. However, it is well known that uncertainty remains in heterogeneity estimates, and this estimate can have large uncertainty which must be considered when interpreting the strength of evidence. The authors evaluated statistical heterogeneity by I2 and τ2 and they stated that there was no significant heterogeneity in almost all pooled risk estimates. All statistical tests for heterogeneity are weak, including I2.The clinical implications of this are considerable and must be examined on a case-by-case basis. Putting too much trust in homogeneity of effects may give a false sense of reassurance that one size fits all. Lack of evidence of heterogeneity is not evidence of homogeneity. In four pooled analyses (Figs 1C, 1D, 2D and 2E of the paper) I2 was 0%. Meta-analyses where I2 is estimated at 0% are affected by an important misconception. Many authors interpret this as an absence of heterogeneity, but the upper 95% CI limit may be substantiated. Confidence intervals of I2 should be calculated and considered when interpreting meta-analyses. Could you please provide the 95% CI of I2?
- 2)We fully agree with the authors that prediction intervals should be routinely reported to allow more informative inference in meta-analysis. However, the estimate of the prediction interval will be large if the estimates of the summary effect and between-study heterogeneity are imprecise. For example, if they are based on only a few small studies.In the very clinically relevant setting of immune checkpoint inhibitors in HBsAg+ patients without nucleoside analogue prophylaxis (Fig. 2B of the paper), prediction intervals range from 0 to 0.99. Is this quantitative estimate useful to support the recommendation?
- 3)HBVr is an uncommon event during treatment with some classes of immunosuppressive drugs, especially in patients receiving nucleoside analogue prophylaxis, as proven in a large amount of included studies, that reported 0 events. The authors reported that generalized linear mixed model was used, but it is not clear what method was used to handle rare events.Please verify if the application of proper methods for meta-analyses of rare events (for example, beta-binomial model, which seems to be more suitable for rare events than binomial normal models– the method that we assumed the authors applied) provides similar estimates.
- 4)The authors did not perform meta-regression analysis. We suggest exploring if study-level covariates (sample size, single-vs. multicentre, study design, and finally study quality) may be a potential source of heterogeneity. Moreover, there are many patient-level characteristics (i.e. ethnicity and the underlying disease needing immunosuppressive treatment) that can affect the risk of HBVr. Unfortunately, the sources of variability of the included studies were not reported with enough consistency, particularly regarding patient-level covariates. So, the results obtained by this meta-analysis of aggregate data may easily be affected by ecological bias. More accurate treatment comparisons could be achieved only by a meta-analysis of individual patient data.
- 5)The authors restricted the systematic review and meta-analysis to only studies where a single drug or drug class was used. However, in real-life clinical practice, most patients receive combination regimens including more drugs with different mechanisms of action associated with different risks of HBVr. This issue can limit the generalizability of these results to several clinical practice settings.
- 6)Although the authors stated that the meta-analysis was performed according to PRISMA-P, they did not report a formal quality assessment, nor did they assess the risk of bias of individual studies. Moreover, publication bias across studies should be assessed.
- 7)Non-randomized studies may experience many problems that could reduce their internal and external validity.When assessing retrospective studies, the most important bias is the likelihood of inappropriate selection of patients for treatment, which can lead to incorrect results and spurious associations. Although the authors were very fair in underlining the limitation related to the non-consecutive selection of patients studied, we suggest exploring if study design (prospective vs. retrospective) may be a significant source of heterogeneity using sensitivity analyses and meta-regressions.
Conflict of interest
- Multimedia component 1
- Hepatitis B virus reactivation associated with new classes of immunosuppressants and immunomodulators: a systematic review, meta-analysis, and expert opinion.J Hepatol. 2022 Jul 16; (S0168-8278(22)02935-X. https://doi.org/10.1016/j.jhep.2022.07.003. [Epub ahead of print])
- Uncertainty in heterogeneity estimates in meta-analyses.BMJ. 2007 Nov 3; 335: 914-916
- Plea for routinely presenting prediction intervals in meta-analysis.BMJ Open. 2016 Jul 12; 6e010247
- Statistical concerns for meta-analysis of rare events and small sample sizes.Lancet Infect Dis. 2022 Aug; 22: 1111
- Meta-analysis of proportions of rare events-A comparison of exact likelihood methods with robust variance estimation.Commun Stat Simul Comput. 2016; 45: 3036-3052
- Inborn/outborn status and neonatal survival: a meta-analysis of non-randomized studies.Stat Med. 1988 Dec; 7: 1207-1221