Keywords
We read with great interest the article “Refining prediction of survival after TIPS with the novel Freiburg index of post-TIPS survival” by Bettinger et al.
[1]
We congratulate the authors on developing a novel model (the FIPS score) to predict 3- and 6-month survival after planned transjugular intrahepatic portosystemic shunt (TIPS) implantation using 4 easily obtainable prognostic factors: age, creatinine, bilirubin, and albumin.For a prediction model to be considered useful, it must be validated in an external, independent cohort.
[2]
In the Freiburg study, both the training set used to develop the model and the validation set were random samples from the same cohort with an equal allocation from each contributing study centre. As a result, the training and validation cohorts were identical except for sampling variation, meaning that there may still be substantial optimism bias in the estimates of model performance. This potential bias was not corrected for during model development by statistical shrinkage techniques, nor during assessment of its performance, e.g. by bootstrapping techniques.[4]
The lack of optimism correction might explain why the FIPS score outperformed the established scores in the Freiburg study. A related issue is the lack of external validation. The authors assessed the discriminatory performance of the FIPS score in an external, Chinese cohort of patients with preemptive TIPS implantation, but in that cohort the score had a poor discriminatory performance with c indices of 0.576 (95% CI 0.462–0.691) for 3-month survival and 0.574 (95% CI 0.462–0.639) for 6-month survival.Moreover, only 5 of 290 patients (1.7%) had a FIPS score ≥0.92, which was the limit set by the authors to indicate a high risk of mortality. Such a small proportion suggests that the FIPS score may have limited clinical utility.
We evaluated the FIPS score in a cohort of 104 patients who received TIPS implantation at our centre between 2013 and 2018. This cohort, which was similar to the Freiburg cohort with regards to indication and liver disease aetiology, is described in detail elsewhere.
[5]
Only 5 of our 104 patients (4.8%) had a FIPS score ≥0.92 before the implantation. None of these patients died less than 6 months after TIPS insertion. We computed the discriminatory performance of the FIPS score measured by Harrell's c index using Stata’s 'somersd' package. We similarly computed the discriminatory performance of the current model for end-stage liver disease (MELD)[6]
and Child-Pugh scores in our cohort. The FIPS score had the poorest discrimination with c indices of 0.59 (95% CI 0.44–0.74) for 3-month survival and 0.57 (95% CI 0.43–0.72) for 6-month survival. The MELD score performed slightly better, while the Child-Pugh score reached c indices of 0.75 (95% CI 0.56–0.94) for 3-month survival and 0.72 (95% CI 0.54–0.90) for 6-month survival (Table 1). Thus, our findings indicate that the Child-Pugh score is superior to the FIPS score for prediction of 3- and 6-month mortality after TIPS insertion.MELD policy changes 2016—UNOS. [cited 13/04/2021]; Available from: https://optn.transplant.hrsa.gov/news/meld-serum-sodium-policy-changes/.
Table 1The discriminatory performance of the FIPS, MELD, and Child-Pugh scores in our external TIPS cohort measured by Harrell's c index.
| FIPS°c index (95% CI) | MELD°c index (95% CI) | Child-Pugh score c index (95% CI) | |
|---|---|---|---|
| 3-months survival | 0.59 (0.44–0.74) | 0.63 (0.43–0.84) | 0.75 (0.56–0.94) |
| p values vs. FIPS | — | 0.74 | 0.20 |
| 6-months survival | 0.57 (0.43–0.72) | 0.60 (0.41–0.79) | 0.72 (0.54–0.90) |
| p values vs. FIPS | — | 0.82 | 0.21 |
FIPS, Freiburg index of post-TIPS; MELD, model for end-stage liver disease; TIPS, transjugular intrahepatic portosystemic shunt.
Our analyses do not invalidate the FIPS score, but they emphasize the importance of external validation of prediction models. Although our cohort is similar to the Freiburg cohort in the most relevant aspects (TIPS indication and liver disease aetiology), it differs in some ways, and these differences might explain the lower discriminatory performance of the FIPS score. In particular, creatinine was lower in our cohort and in the Chinese preemptive TIPS cohort than it was in the FIPS development cohort. However, the potential usefulness of the FIPS score depends on its ability to maintain discriminatory performance across different study populations. Of note, our cohort is small, and we await with great anticipation future validation studies assessing the utility of the novel FIPS score.
Financial support
The authors received no financial support to produce this manuscript.
Authors’ contributions
All authors contributed to the conception of the study and the final manuscript. FK carried out the formal analyses and wrote the original draft.
Conflict of interest
The authors declare no conflicts of interest that pertain to this work.
Please refer to the accompanying ICMJE disclosure forms for further details.
Supplementary data
The following is the supplementary data to this article:
- Multimedia component 1
References
- Refining prediction of survival after TIPS with the novel Freiburg index of post-TIPS survival.J Hepatol. 2021 Jun; 74: 1362-1372
- Risk prediction models: II. External validation, model updating, and impact assessment.Heart. 2012; 98: 691-698
- Overfitting and optimism in prediction models.2009: 83-100
- Towards better clinical prediction models: seven steps for development and an ABCD for validation.Eur Heart J. 2014; 35: 1925-1931
- Long-term effects and complications of the transjugular intrahepatic portosystemic shunt: a single-centre experience.Scand J Gastroenterol. 2019; 54: 899-904
MELD policy changes 2016—UNOS. [cited 13/04/2021]; Available from: https://optn.transplant.hrsa.gov/news/meld-serum-sodium-policy-changes/.
Article info
Publication history
Published online: April 26, 2021
Accepted:
April 21,
2021
Received:
April 15,
2021
Identification
Copyright
© 2021 European Association for the Study of the Liver. Published by Elsevier B.V. All rights reserved.
