To the Editor:
Papatheodoridis et al. report a meta-analysis
[1]
of aggregate data from non-randomized (prospective or retrospective) studies evaluating the risk of HBV reactivation (HBVr) associated with new classes of immunomodulators and immunosuppressants. We fully agree with the study recommendations provided by the expert panel, but some concerns remain about the methodological approach used for the quantitative assessment of the risk of HBVr in this meta-analysis.- Papatheodoridis G.V.
- Lekakis V.
- Voulgaris T.
- Lampertico P.
- Berg T.
- Chan H.L.Y.
- et al.
Hepatitis B virus reactivation associated with new classes of immunosuppressants and immunomodulators: a systematic review, meta-analysis, and expert opinion.
J Hepatol. 2022 Jul 16; (S0168-8278(22)02935-X. https://doi.org/10.1016/j.jhep.2022.07.003. [Epub ahead of print])
- 1)We fully agree with authors that the strength of evidence is considered high when no significant statistical heterogeneity is found among pooled studies. However, it is well known that uncertainty remains in heterogeneity estimates, and this estimate can have large uncertainty which must be considered when interpreting the strength of evidence. The authors evaluated statistical heterogeneity by I2 and τ2 and they stated that there was no significant heterogeneity in almost all pooled risk estimates. All statistical tests for heterogeneity are weak, including I2.[2]The clinical implications of this are considerable and must be examined on a case-by-case basis. Putting too much trust in homogeneity of effects may give a false sense of reassurance that one size fits all. Lack of evidence of heterogeneity is not evidence of homogeneity. In four pooled analyses (Figs 1C, 1D, 2D and 2E of the paper) I2 was 0%. Meta-analyses where I2 is estimated at 0% are affected by an important misconception. Many authors interpret this as an absence of heterogeneity, but the upper 95% CI limit may be substantiated. Confidence intervals of I2 should be calculated and considered when interpreting meta-analyses. Could you please provide the 95% CI of I2?
- 2)We fully agree with the authors that prediction intervals should be routinely reported to allow more informative inference in meta-analysis. However, the estimate of the prediction interval will be large if the estimates of the summary effect and between-study heterogeneity are imprecise. For example, if they are based on only a few small studies.[3]In the very clinically relevant setting of immune checkpoint inhibitors in HBsAg+ patients without nucleoside analogue prophylaxis (Fig. 2B of the paper), prediction intervals range from 0 to 0.99. Is this quantitative estimate useful to support the recommendation?
- 3)HBVr is an uncommon event during treatment with some classes of immunosuppressive drugs, especially in patients receiving nucleoside analogue prophylaxis, as proven in a large amount of included studies, that reported 0 events. The authors reported that generalized linear mixed model was used, but it is not clear what method was used to handle rare events.[4]Please verify if the application of proper methods for meta-analyses of rare events (for example, beta-binomial model, which seems to be more suitable for rare events than binomial normal models[5]– the method that we assumed the authors applied) provides similar estimates.
- 4)The authors did not perform meta-regression analysis. We suggest exploring if study-level covariates (sample size, single-vs. multicentre, study design, and finally study quality) may be a potential source of heterogeneity. Moreover, there are many patient-level characteristics (i.e. ethnicity and the underlying disease needing immunosuppressive treatment) that can affect the risk of HBVr. Unfortunately, the sources of variability of the included studies were not reported with enough consistency, particularly regarding patient-level covariates. So, the results obtained by this meta-analysis of aggregate data may easily be affected by ecological bias. More accurate treatment comparisons could be achieved only by a meta-analysis of individual patient data.
- 5)The authors restricted the systematic review and meta-analysis to only studies where a single drug or drug class was used. However, in real-life clinical practice, most patients receive combination regimens including more drugs with different mechanisms of action associated with different risks of HBVr. This issue can limit the generalizability of these results to several clinical practice settings.
- 6)Although the authors stated that the meta-analysis was performed according to PRISMA-P, they did not report a formal quality assessment, nor did they assess the risk of bias of individual studies. Moreover, publication bias across studies should be assessed.
- 7)Non-randomized studies may experience many problems that could reduce their internal and external validity.[6]When assessing retrospective studies, the most important bias is the likelihood of inappropriate selection of patients for treatment, which can lead to incorrect results and spurious associations. Although the authors were very fair in underlining the limitation related to the non-consecutive selection of patients studied, we suggest exploring if study design (prospective vs. retrospective) may be a significant source of heterogeneity using sensitivity analyses and meta-regressions.
Financial support
The authors received no financial support to produce this manuscript.
Conflict of interest
Prof. Cammà is on the speaker’s bureau for Eisai, Ipsen, AstraZeneca, and Roche. Dr. Celsa is on the speakers’ bureau for Eisai, MSD, and Ipsen. Marco Enea has no conflict of interest.
Please refer to the accompanying ICMJE disclosure forms for further details.
Authors’ contributions
Calogero Cammà, Marco Enea and Ciro Celsa conceived, reviewed and approved the manuscript.
Supplementary data
The following are the supplementary data to this article:
- Multimedia component 1
References
- Hepatitis B virus reactivation associated with new classes of immunosuppressants and immunomodulators: a systematic review, meta-analysis, and expert opinion.J Hepatol. 2022 Jul 16; (S0168-8278(22)02935-X. https://doi.org/10.1016/j.jhep.2022.07.003. [Epub ahead of print])
- Uncertainty in heterogeneity estimates in meta-analyses.BMJ. 2007 Nov 3; 335: 914-916
- Plea for routinely presenting prediction intervals in meta-analysis.BMJ Open. 2016 Jul 12; 6e010247
- Statistical concerns for meta-analysis of rare events and small sample sizes.Lancet Infect Dis. 2022 Aug; 22: 1111
- Meta-analysis of proportions of rare events-A comparison of exact likelihood methods with robust variance estimation.Commun Stat Simul Comput. 2016; 45: 3036-3052
- Inborn/outborn status and neonatal survival: a meta-analysis of non-randomized studies.Stat Med. 1988 Dec; 7: 1207-1221
Article info
Publication history
Published online: September 29, 2022
Accepted:
September 14,
2022
Received:
September 5,
2022
Publication stage
In Press Journal Pre-ProofIdentification
Copyright
© 2022 European Association for the Study of the Liver. Published by Elsevier B.V. All rights reserved.
