Advertisement

Artificial intelligence predicts immune and inflammatory gene signatures directly from hepatocellular carcinoma histology

Published:February 07, 2022DOI:https://doi.org/10.1016/j.jhep.2022.01.018

      Highlights

      • AI-based pathology can predict the activation of immune gene signatures directly from hepatocellular carcinoma histology.
      • Our models generalize well in an independent series of samples with different gene expression profiling technology and staining protocols.
      • These approaches could represent a novel type of biomarker.

      Background & Aims

      Patients with hepatocellular carcinoma (HCC) displaying overexpression of immune gene signatures are likely to be more sensitive to immunotherapy, however, the use of such signatures in clinical settings remains challenging. We thus aimed, using artificial intelligence (AI) on whole-slide digital histological images, to develop models able to predict the activation of 6 immune gene signatures.

      Methods

      AI models were trained and validated in 2 different series of patients with HCC treated by surgical resection. Gene expression was investigated using RNA sequencing or NanoString technology. Three deep learning approaches were investigated: patch-based, classic MIL and CLAM. Pathological reviewing of the most predictive tissue areas was performed for all gene signatures.

      Results

      The CLAM model showed the best overall performance in the discovery series. Its best-fold areas under the receiver operating characteristic curves (AUCs) for the prediction of tumors with upregulation of the immune gene signatures ranged from 0.78 to 0.91. The different models generalized well in the validation dataset with AUCs ranging from 0.81 to 0.92. Pathological analysis of highly predictive tissue areas showed enrichment in lymphocytes, plasma cells, and neutrophils.

      Conclusion

      We have developed and validated AI-based pathology models able to predict the activation of several immune and inflammatory gene signatures. Our approach also provides insights into the morphological features that impact the model predictions. This proof-of-concept study shows that AI-based pathology could represent a novel type of biomarker that will ease the translation of our biological knowledge of HCC into clinical practice.

      Lay summary

      Immune and inflammatory gene signatures may be associated with increased sensitivity to immunotherapy in patients with advanced hepatocellular carcinoma. In the present study, the use of artificial intelligence-based pathology enabled us to predict the activation of these signatures directly from histology.

      Graphical abstract

      Keywords

      Linked Article

      • Translating artificial intelligence from code to bedside: The road towards AI-driven predictive biomarkers for immunotherapy of hepatocellular carcinoma
        Journal of HepatologyVol. 77Issue 1
        • Preview
          Hepatocellular carcinoma (HCC) is the second most common cause of cancer-related death worldwide with growing incidence rates particularly affecting North America and Western Europe.1,2 This increase in incidence rates is primarily the result of sequelae of hepatitis C viral infections (which are now curable), and the growing rates of non-alcoholic steatohepatitis (NASH) associated with diabetes and obesity.3 The last decade has seen rapid growth in the therapeutic options available for HCC. As such, increasingly data-driven approaches to surgical resection and orthotopic liver transplantation, as well as modern image-guided locoregional therapies, have dramatically improved outcomes in populations with early and intermediate stage disease.
        • Full-Text
        • PDF
      See Editorial, pages 6–8

      Introduction

      Hepatocellular carcinoma (HCC) remains one of the leading causes of cancer-related deaths worldwide and a global health challenge.
      • Llovet J.M.
      • Kelley R.K.
      • Villanueva A.
      • Singal A.G.
      • Pikarsky E.
      • Roayaie S.
      • et al.
      Hepatocellular carcinoma.
      The vast majority (80–90%) of cases develop in patients with chronic liver disease or cirrhosis and the main risk factors are viral hepatitis, alcohol intake or metabolic syndrome.
      • Llovet J.M.
      • Kelley R.K.
      • Villanueva A.
      • Singal A.G.
      • Pikarsky E.
      • Roayaie S.
      • et al.
      Hepatocellular carcinoma.
      Particular treatments, such as surgical resection, percutaneous ablation or liver transplantation, offer a chance of cure. However, more than two-thirds of patients present with advanced disease and are therefore not eligible for these strategies.
      • Llovet J.M.
      • Montal R.
      • Sia D.
      • Finn R.S.
      Molecular therapies and precision medicine for hepatocellular carcinoma.
      Nevertheless, the landscape of therapeutic options is rapidly evolving, and several drugs (including lenvatinib, cabozantinib, regaorafenib or ramucirumab) have recently been approved.
      • Kudo M.
      • Finn R.S.
      • Qin S.
      • Han K.H.
      • Ikeda K.
      • Piscaglia F.
      • et al.
      Lenvatinib versus sorafenib in first-line treatment of patients with unresectable hepatocellular carcinoma: a randomised phase 3 non-inferiority trial.
      • Abou-Alfa G.K.
      • Meyer T.
      • Cheng A.-L.
      • El-Khoueiry A.B.
      • Rimassa L.
      • Ryoo B.-Y.
      • et al.
      Cabozantinib in patients with advanced and progressing hepatocellular carcinoma.
      • Zhu A.X.
      • Kang Y.K.
      • Yen C.J.
      • Finn R.S.
      • Galle P.R.
      • Llovet J.M.
      • et al.
      Ramucirumab after sorafenib in patients with advanced hepatocellular carcinoma and increased α-fetoprotein concentrations (REACH-2): a randomised, double-blind, placebo-controlled, phase 3 trial.
      • Bruix J.
      • Qin S.
      • Merle P.
      • Granito A.
      • Huang Y.-H.
      • Bodoky G.
      • et al.
      Regorafenib for patients with hepatocellular carcinoma who progressed on sorafenib treatment (RESORCE): a randomised, double-blind, placebo-controlled, phase 3 trial.
      Immunotherapy also holds great promise to improve clinical outcomes. The combination of atezolizumab, an anti-PD-L1 (programmed death ligand 1) monoclonal antibody, and bevacizumab, an anti-angiogenic agent, was shown to be superior to sorafenib and is now the standard of care for patients with advanced disease.
      • Finn R.S.
      • Qin S.
      • Ikeda M.
      • Galle P.R.
      • Ducreux M.
      • Kim T.-Y.
      • et al.
      Atezolizumab plus bevacizumab in unresectable hepatocellular carcinoma.
      Other clinical trials testing immunomodulating agents, such as nivolumab, have turned out to be negative.
      • Sangro B.
      • Sarobe P.
      • Hervás-Stubbs S.
      • Melero I.
      Advances in immunotherapy for hepatocellular carcinoma.
      Durable responses are however observed with these drugs, and several clinical trials investigating their use in other clinical settings are underway.
      • Llovet J.M.
      • Kelley R.K.
      • Villanueva A.
      • Singal A.G.
      • Pikarsky E.
      • Roayaie S.
      • et al.
      Hepatocellular carcinoma.
      Fast and accurate identification of patients likely to respond is therefore a critical issue. In this line, by investigating HCC samples of advanced patients treated by nivolumab, Sangro et al. showed that several immune-related gene signatures were associated with survival.
      • Sangro B.
      • Melero I.
      • Wadhawan S.
      • Finn R.S.
      • Abou-Alfa G.K.
      • Cheng A.-L.
      • et al.
      Association of inflammatory biomarkers with clinical outcomes in nivolumab-treated patients with advanced hepatocellular carcinoma.
      These findings were also recently confirmed by Haber et al. who showed that upregulation of interferon gamma signaling and antigen presentation was able to predict the overall response rate of patients with HCC treated by nivolumab or pembrolizumab.
      • Haber P.K.
      • Torres-Martin M.
      • Dufour J.-F.
      • Verslype C.
      • Marquardt J.
      • Galle P.R.
      • et al.
      Molecular markers of response to anti-PD1 therapy in advanced hepatocellular carcinoma.
      These signatures noticeably include immune checkpoint inhibitors (LAG3, CD274-PDL1), cytolytic effectors (GZMA, PRF1), and molecules involved in antigen processing/presentation (HLA-DMA, HLA-DMB) and immune cell recruitment (CXCL9, CXCL10, IFNG).
      These results are consistent with data from other human solid cancers and suggest that such molecular biomarkers may help to i) identify the potential responders to immunomodulating approaches and ii) spare side effects for patients that are not likely to benefit from these strategies.
      • Havel J.J.
      • Chowell D.
      • Chan T.A.
      The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy.
      Their use in a clinical setting is however challenging, as they require access to molecular biology platforms for nucleic acid extraction and their processing/sequencing. They are also highly dependent on the quality of samples, and prone to standardization issues.
      On the other hand, histological slides are easily available from pathology departments. It is well-established that they contain an extensive amount of information to enable definitive diagnosis and clinical outcome prediction. The advent of digital pathology and artificial intelligence (AI) also enables pathologists to standardize their analysis and extract meaningful morphological features that are not easily accessible to the human eye.
      • Coudray N.
      • Ocampo P.S.
      • Sakellaropoulos T.
      • Narula N.
      • Snuderl M.
      • Fenyö D.
      • et al.
      Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.
      • Echle A.
      • Grabsch H.I.
      • Quirke P.
      • van den Brandt P.A.
      • West N.P.
      • Hutchins G.G.A.
      • et al.
      Clinical-grade detection of microsatellite instability in colorectal tumors by deep learning.
      • Kather J.N.
      • Calderaro J.
      Development of AI-based pathology biomarkers in gastrointestinal and liver cancer.
      • Calderaro J.
      • Kather J.N.
      Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers.
      We thus aimed, using AI-based pathology, to develop deep learning models able to predict the activation of 6 immune gene signatures associated with response to immunotherapy in patients with HCC.

      Materials and methods

      Patients and samples

      For our discovery series, we used The Cancer Genome Atlas Liver Hepatocellular Carcinoma (TCGA-LIHC) public dataset. It consisted of patients with primary HCC treated by surgical resection in more than 20 different clinical centers.
      Cancer Genome Atlas Research Network
      Comprehensive and integrative genomic characterization of hepatocellular carcinoma.
      The inclusion criteria were as follows: i) unequivocal morphological features of HCC (all slides were reviewed by a pathologist specializing in liver disease [JC], and cases with features suggestive of combined hepatocellular-cholangiocarcinoma were excluded), ii) ≥1 available digital histological slide(s) from formalin fixed-paraffin embedded material and iii) available gene expression profiling (obtained by RNA sequencing). Data and slides were accessed and downloaded in March 2020.
      The validation series consisted of primary HCC samples developed in patients treated by surgical resection in Henri Mondor University Hospital (Créteil, France). Inclusion criteria were: i) surgical resection performed between 2010-2019, ii) histological diagnosis of HCC confirmed by a liver pathologist (JC) and iii) available slides and tissue blocks for gene expression experiments. For 7 patients, pre-operative biopsies were available and included to test the models on this type of sample. The study was performed according to the declaration of Helsinki and was approved by an institutional review board (CPP Ile de France V). All necessary written informed consents were obtained from patients.
      The overall flowchart of the study is presented in Fig. 1.
      Figure thumbnail gr1
      Fig. 1Flowchart of the study.
      The expression of genes included in the 6 signatures was investigated in 336 HCC samples from the discovery series (TCGA-LIHC) using RNA sequencing data and unsupervised clustering. Cases were labeled as Cluster High or Median/Low and three different approaches were trained using the available 349 WSIs and the associated immune labels. The best-fold models were further validated in 139 tumors developed in patients treated in Henri Mondor University Hospital. For these samples, the gene expression was investigated using the NanoString PanCancer IO360™ panel. CLAM, clustering-constrained attention multiple-instance learning; HCC, hepatocellular carcinoma; MIL, multiple-instance learning; TCGA-LIHC, The Cancer Genome Atlas Liver Hepatocellular Carcinoma; WSIs, whole-slide digital histological images.

      RNA sequencing data processing and clustering analysis in the discovery series

      The fragments per kilobase million (FPKM) counts selected from the TCGA-LIHC dataset were processed for normalization due to non-linear bias induced by genomic screening. Log 2 transformation was then applied after adding 1 to the FPKM matrix. The Z-score approach was chosen for gene-wise standardization.
      We further aimed to investigate the 6 immune gene signatures that were previously shown to be associated, in patients with advanced HCC, with improved response and survival rates after treatment with the anti-PD-1 monoclonal antibody nivolumab: “6-Gene Interferon Gamma” (6G IFNg) (CXCL10, CXCL9, HLA-DRA, IDO1, IFNG, STAT1),
      • Ayers M.
      • Lunceford J.
      • Nebozhyn M.
      • Murphy E.
      • Loboda A.
      • Kaufman D.R.
      • et al.
      IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade.
      “Gajewski 13-Gene Inflammatory” (Gajewski 13G) (CCL2, CCL3, CCL4, CD8A, CXCL10, CXCL9, GZMK, HLA-DMA, HLA-DMB, HLA-DOA, HLA-DOB, ICOS, IRF1, CCL3 was not included in the Nanostring Panel so we also removed it from the discovery series),
      • Spranger S.
      • Bao R.
      • Gajewski T.F.
      Melanoma-intrinsic β-catenin signalling prevents anti-tumour immunity.
      “Inflammatory” (CD274/PD-L1, CD8A, LAG3, STAT1),
      • Sangro B.
      • Melero I.
      • Wadhawan S.
      • Finn R.S.
      • Abou-Alfa G.K.
      • Cheng A.-L.
      • et al.
      Association of inflammatory biomarkers with clinical outcomes in nivolumab-treated patients with advanced hepatocellular carcinoma.
      “Interferon Gamma Biology” (IFNg Biology) (CCL5, CD27, CXCL9, CXCR6, IDO1, STAT1),
      • Ayers M.
      • Lunceford J.
      • Nebozhyn M.
      • Murphy E.
      • Loboda A.
      • Kaufman D.R.
      • et al.
      IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade.
      “Ribas 10-Gene Interferon Gamma” (Ribas 10G) (CCR5, CXCL10, CXCL11, CXCL9, GZMA, HLA-DRA, IDO1, IFNG, PRF1, STAT1)
      • Ayers M.
      • Lunceford J.
      • Nebozhyn M.
      • Murphy E.
      • Loboda A.
      • Kaufman D.R.
      • et al.
      IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade.
      and “T-cell Exhaustion” (CD274/PD-L1, CD276, CD8A, LAG3, PDCD1LG2, TIGIT)
      • Ayers M.
      • Lunceford J.
      • Nebozhyn M.
      • Murphy E.
      • Loboda A.
      • Kaufman D.R.
      • et al.
      IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade.
      signatures. For each gene signature, we performed hierarchical clustering of samples using the Ward2 algorithm
      • Murtagh F.
      • Legendre P.
      Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?.
      (implemented as Ward.D2 in R stats package) and Euclidean distance.
      • Murtagh F.
      • Legendre P.
      Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?.
      ,
      • Deza M.M.
      • Deza E.
      Encyclopedia of distances.
      We flattened the dendrogram into 3 clusters, namely Cluster High, Cluster Median and Cluster Low, and then merged the latter 2 clusters as Cluster Median/Low. Heatmaps with clustered dendrograms were used for visual validation.

      mRNA extraction and quality control in the validation series

      For each HCC sample, 5 μm-thick sections were cut from formalin fixed-paraffin embedded blocks. Tumor tissue was then macro-dissected and total RNAs were further isolated using the Recover All™ Total Nucleic Acid Isolation Kit (Invitrogen, Thermo Fisher Scientific), according to manufacturer’s instructions. They were monitored to ensure their quality, purity and integrity using an Agilent 2100 Bioanalyzer device with the Pico assay (Agilent, Santa Clara, CA, USA). Samples with a DV200 (percentage of RNA fragments above 200 nucleotides) >30% were considered adequate for further analysis.

      Gene expression analysis in the validation series

      Gene expression was analyzed using the NanoString PanCancer Immuno-Oncology 360™ Panel (NanoString Technologies, Seattle, USA) that includes a set of more than 700 genes involved in the main biological pathways of human immunity. These experiments were performed by the Genomics platform of Institut Curie (Paris, France). Total RNAs were used as templates. A human Universal Reference RNA and a no-template control (water) and 10 cell lines were also hybridized in parallel with the HCC samples. NanoString positive and negative controls were also added to samples as spikes in controls. After an overnight hybridization at 65°C, samples were processed on the NanoString nCounter preparation station (NanoString Technologies) to immobilize biotinylated hybrids and remove probes in excess. The nCounter Digital Analyzer was used to scan the cartridges at maximum resolution (fields of view n = 555), count the fluorescent barcodes and quantify RNA molecules. Normalization was performed against the geometric mean of 20 housekeeping genes in combination with a positive control normalization which uses the geometric mean of 6 synthetic positive targets.
      As performed for the discovery series, gene expression values were log2-transformed and the Z-score approach was further applied for gene-wise standardization. For each gene signature, hierarchical clustering with the Ward2 algorithm and Euclidean distance was then performed. HCC samples were then labeled as Cluster High or Cluster Median/Low.

      Slide preprocessing and tessellation, staining conversion, color normalization and data augmentation

      Slides from the discovery series were stained with hematein-eosin and encoded in svs format while slides from the validation series were stained with hematein-eosin-saffron and encoded in ndpi format. Tissue regions were then exhaustively split into patches of 256×256 pixels (without overlapping) at 20x using the OpenSlide library in Python. Tumor regions were annotated by an expert pathologist (JC) in polygonal regions of interest using the open source QuPath software.
      • Bankhead P.
      • Loughrey M.B.
      • Fernández J.A.
      • Dombrowski Y.
      • McArt D.G.
      • Dunne P.D.
      • et al.
      QuPath: open source software for digital pathology image analysis.
      Patches were extracted from the intersection area of the detected tissue regions and the annotated tumor regions. Staining conversion, color normalization and data augmentation protocols were assessed (supplementary materials and methods).

      Deep learning models

      Baseline model: patch-based workflow with ShuffleNet

      A detailed description of all steps involved in the model’s development is provided in the supplementary materials and methods, and our code is publicly available (https://github.com/qinghezeng/Histo2GeneSignatures). For our baseline model, we re-implemented a patch-based strategy in Python (Fig. 2) (https://github.com/jnkather/DeepHistology/tree/v0.2). For the training and cross validation, 500 patches were randomly collected from each slide, with each patch inheriting the label of the whole-slide digital histological image (WSI). Cluster Median/Low training patches were randomly downsampled to match the same number of Cluster High training patches. We then trained a ShuffleNet, which was pre-trained on ImageNet. With Cluster High as the positive class, optimal patch-level thresholds were computed using the receiver-operating characteristic (ROC) curves on test patches. For inference, all patches extracted from a WSI were predicted by our trained ShuffleNet and classified as Cluster High or Low/Median using the optimal threshold determined on the discovery series. The WSI-level score was calculated by dividing the number of Cluster High patches by the total number of patches in that WSI.
      Figure thumbnail gr2
      Fig. 2Workflow for patch-based strategy.
      During the training, 500 patches were randomly sampled from each WSI. The Cluster Median/Low training set was then randomly downsampled to match the amount of Cluster High patches. The equalized training sets were used to finetune a ShuffleNet pre-trained on ImageNet, with an optimal patch-level threshold calculated from the ROC curve. For the inference, all the patches extracted from a test WSI were predicted by the trained ShuffleNet and further binarized using the previous threshold. The percentage of Cluster High patches was the probability of this WSI to be Cluster High, which was eventually thresholded at the WSI-level. ROC, receiver-operating characteristic; ROIs, regions of interest; WSI, whole-slide digital histological images. (This figure appears in color on the web.)

      WSI-based models: classic MIL and CLAM

      Apart from the patch-based approach, we also investigated multiple-instance learning (MIL) approaches (Fig. 3). MIL is a weakly supervised learning paradigm in which data is arranged in bags of instances. In the classic binary MIL assumption, a bag is labeled as i) negative if all the instances inside are negative or ii) positive if it contains ≥1 positive instance(s). Based on this assumption, if there is/are ≥1 Cluster High patch(es) in the WSI, the tumor WSI will be classified as Cluster High. Provided with the WSI-level label (and tumor regions of interest in the experiments with annotations), the MIL models have the ability to predict labels for unseen WSIs by taking account of the most predictive patches. We investigated 2 MIL approaches, namely classic MIL and clustering-constrained attention multiple-instance learning (CLAM).
      • Lu M.Y.
      • Williamson D.F.K.
      • Chen T.Y.
      • Chen R.J.
      • Barbieri M.
      • Mahmood F.
      Data-efficient and weakly supervised computational pathology on whole-slide images.
      Figure thumbnail gr3
      Fig. 3Workflow for MIL strategy.
      CLAM and classic MIL shared the same preprocessing steps of tessellation and feature extraction. A 1,024-dimensional feature embedding was extracted from each patch by a ResNet50 trained on ImageNet. The output of the deep learning network is two-class WSI-level probabilities and one of Cluster High was thereafter binarized into Cluster High or Cluster Median/Low. Ni: patch number in a WSI. FC: fully connected layer. : attention-based pooling, composed of element-wise multiplication and sum along the first (patch) axis. CLAM, clustering-constrained attention multiple-instance learning; MIL, multiple-instance learning; ROIs, regions of interest; WSI, whole-slide digital histological image. (This figure appears in color on the web.)
      For both strategies, the first stage involves feature extraction using a modified ResNet50 model pre-trained on ImageNet (Fig. 3). Each of the Ni patches from a given WSI was encoded as a 1,024-dimensional feature, allowing the possibility to load all the patches of the WSI into the GPU memory simultaneously. For the classic MIL approach, we used the implementation in the CLAM work (code available at https://github.com/mahmoodlab/CLAM). The first fully connected (FC) layer further reduced the features to 512 dimensions, and the second FC layer was used as a classifier to generate 2-class scores for each patch. A max-pooling function was then applied on the Cluster High class to select the top-1 patch and normalize its scores to WSI-level probabilities by softmax.
      CLAM is a recently reported more sophisticated MIL approach specifically designed for digital pathology (code available at https://github.com/mahmoodlab/CLAM).
      • Lu M.Y.
      • Williamson D.F.K.
      • Chen T.Y.
      • Chen R.J.
      • Barbieri M.
      • Mahmood F.
      Data-efficient and weakly supervised computational pathology on whole-slide images.
      Its attention mechanism helps the model to focus on representative patches automatically. The same ResNet50 network was used for the patch encoding, and, as performed for the classic MIL strategy, the feature vectors were reduced to 512 dimensions for each WSI. An FC layer with a softmax activation function was used as a classifier to generate 2-class WSI-level probabilities.
      For all models investigated, training was performed using a 10-fold Monte Carlo cross-validation strategy. For each fold, the discovery series was randomly partitioned into training (60% of cases)/validation (20%)/test (20%) sets. The whole validation series was used for external validation. Performance was further assessed using the area under the ROC curve (AUC). For each gene signature, we determined the optimal threshold(s) by selecting the cut-offs of the ROC curve with the highest Youden index. In the case of multiple optimal thresholds, the one closer to the median was selected.

      Attention map generation and pathological reviewing

      CLAM is able to produce interpretable heatmaps that allow users to visualize, within each WSI, the relative contribution of every tissue area to the model’s predictions.
      • Lu M.Y.
      • Williamson D.F.K.
      • Chen T.Y.
      • Chen R.J.
      • Barbieri M.
      • Mahmood F.
      Data-efficient and weakly supervised computational pathology on whole-slide images.
      These heatmaps thus allow pathologists to determine which histological and cytological features are associated with high predictive value.
      For each gene signature, an attention score was learned by CLAM for each patch and converted into percentiles. For each WSI, the percentiles were then normalized to [0, 1] with 1 being the most predictive and 0 being the most non-informative. The normalized scores were represented with a colormap (red for 1 and blue for 0), and reconstructed into a heatmap according to the spatial locations of the corresponding patches. We extracted, for each tumor correctly classified as Cluster High and each gene signature, the top 8 image patches classified as highly predictive and the top 8 patches considered non-informative by the model. They were further reviewed by 2 pathologists (JC and CTN). The following histological and cytological features were systematically recorded: tumor cells, blood, vessels, immune cells (lymphocytes, neutrophils, eosinophils, plasma cells), steatosis, clear cells, atypia, fibrosis, eosinophilic inclusions, cholestasis, hyperchromasia, sarcomatoid changes, multinucleated cells, necrosis, steatohepatitic pattern, vascular spaces, microtrabecular, macrotrabecular, compact and pseudoglandular growth patterns (Table S1).
      Differences between highly predictive and non-informative image patches were analyzed using Fisher's exact test. The degree of inter-observer agreement was assessed with Cohen Kappa statistics.

      Results

      Unsupervised hierarchical clustering of samples from the discovery series

      A total of 336 cases from the TCGA-LIHC dataset met our inclusion criteria (349 slides from 336 tumors). The main clinical, biological and pathological features of the patients and tumors are presented in Table S2. They were common for a series of patients with HCC treated by surgical resection. The median age at surgery was 61 years and a male predominance was observed (sex ratio 68%, 228/336). The main risk factors were alcohol consumption (35%, 111/318) and hepatitis B virus (HBV) (32%, 101/318). Microvascular invasion was identified in 29% of the tumors (83/285). As expected in a series of patients treated by surgery, a low rate of significantly fibrotic livers was observed (Ishak score 5-6, 37%, 74/198).
      We first performed unsupervised clustering analysis on RNA sequencing data for all 6 gene signatures. Three distinct sample clusters (High, Median and Low) were observed for each gene signature (Fig. 4). We further aimed to identify samples belonging to the cluster displaying the highest expression of the gene signatures investigated, as they are likely to constitute the subset of HCC that are the most sensitive to immunotherapy.
      • Sangro B.
      • Melero I.
      • Wadhawan S.
      • Finn R.S.
      • Abou-Alfa G.K.
      • Cheng A.-L.
      • et al.
      Association of inflammatory biomarkers with clinical outcomes in nivolumab-treated patients with advanced hepatocellular carcinoma.
      Tumors belonging to Cluster High for 6G IFNg, Gajewski 13G, Inflammatory, IFNg Biology, Ribas 10G, and T-cell Exhaustion signatures represented 13% (44/336), 14% (48/336), 12% (41/336), 11% (36/336), 12% (40/336) and 11% (36/336) of the cases, respectively (Table S3). A significant overlap in samples classified as Cluster High was observed (25 cases belonged to Cluster High for all signatures investigated). We investigated the associations between clusters and clinical and pathological features of the patients and tumors (Table S4). We observed significant associations (p <0.05) between Cluster High and: higher serum alpha-fetoprotein levels (all 6 gene signatures), hepatitis C virus (HCV) infection (all signatures except IFNg Biology), non-tumor liver fibrosis (Ishak score 5-6) (Ribas 10G) and G4 histological grade (Inflammatory).
      Figure thumbnail gr4
      Fig. 4Development of best-fold CLAM models in the discovery series (TCGA-LIHC).
      For the 6 each gene signatures, clustering heatmaps, ROC curves and confusion matrices on the test split using the best-fold model are provided (optimal threshold: 0.149, 0.288, 0.169, 0.277, 0.152, 0.187, respectively). AUC, area under the ROC curve; CLAM, clustering-constrained attention multiple-instance learning; FPKM, fragments per kilobase million; G, -gene; ROC, receiver-operating characteristic; TCGA-LIHC, The Cancer Genome Atlas Liver Hepatocellular Carcinoma. (This figure appears in color on the web.)

      Development of deep learning models for the prediction of HCC with activation of immune gene signatures

      The 349 WSIs from the discovery series were downloaded and we first extracted ∼1,000,000 and 4,982,872 patches (256x256 pixels) for the patch-based and classic MIL/CLAM models, respectively. The WSIs were processed as is, meaning that all patches from both the tumor and the adjacent non-tumor parenchyma were analyzed. These image patches were further fed, along with their corresponding immune cluster labels, into our 3 different models.
      The training was performed using a 10-fold Monte Carlo cross-validation strategy and, for each model and gene signature, each fold AUC was computed. We observed that the 3 different models showed a relatively weak overall performance (Table S5). The mean AUC ranged from 0.490 to 0.666, 0.516 to 0.577, and from 0.555 to 0.632 for the patch-based approach, MIL and CLAM, respectively (Table S5). The highest performance was achieved for the Gajewski 13G signature (mean AUC 0.577 and 0.632 for MIL and CLAM, respectively). We hypothesized that these suboptimal results were explained, at least in part, by the existence of irrelevant noise/patterns in the non-tumor tissue included in the WSIs.
      We thus modified our strategy, and an expert pathologist (JC) annotated the tumor areas for each of the 349 TCGA-LIHC WSIs available. The models were re-trained on patches extracted from the manually delineated tumor areas (total of ∼600,000 and 2,926,135 patches for the patch-based and classic MIL/CLAM, respectively). This approach yielded better performances with best-fold AUCs ranging from 0.661 to 0.783, 0.677 to 0.893 and 0.780 to 0.914 for the patch-based approach, the classic MIL and the CLAM models, respectively (Table 1). AUCs were higher for Gajewski 13G and IFNg Biology signatures. Among our 3 different approaches, the CLAM model showed an overall superior performance (Table 1). The results (receiver-operating characteristic curves and confusion matrix for the optimal threshold) for the best-fold models obtained with CLAM on the 6 gene signatures are displayed in Fig. 4.
      Table 1Performances of the models in the discovery series, using annotated WSIs (best-fold and mean AUCs).
      Gene signaturePatch-based approachClassic MILCLAM
      Best fold AUCMean ± SDBest fold AUCMean ± SDBest fold AUCMean ± SD
      6G Interferon Gamma0.6610.560 ± 0.0670.7580.630 ± 0.0780.7800.635 ± 0.097
      Gajewski 13G Inflammatory0.8090.688 ± 0.0620.8930.694 ± 0.1250.9140.728 ± 0.096
      Inflammatory0.7060.580 ± 0.0770.8060.641 ± 0.1230.7960.665 ± 0.081
      Interferon Gamma Biology0.7830.561 ± 0.1190.6770.610 ± 0.0510.8220.674 ± 0.102
      Ribas 10G Inflammatory0.7270.640 ± 0.0740.7260.618 ± 0.0650.8060.669 ± 0.067
      T-cell Exhaustion0.6610.543 ± 0.0730.7880.606 ± 0.0860.7880.577 ± 0.092
      AUC, area under the receiver operating characteristic curve; CLAM, clustering-constrained attention multiple-instance learning; G, -gene; MIL, multiple-instance learning; SD, standard deviation; WSIs, whole-slide digital histological images.

      External validation of the models

      Deep learning systems are prone to overfit the data they are trained on and thus external validation is critical. We thus aimed to validate our models in a completely independent series of samples with different i) gene expression profiling technology (NanoString Panel vs. RNA sequencing), ii) staining protocols (hematein-eosin-saffron vs. hematein-eosin), and iii) WSI image format (ndpi vs. svs). The fields of view were also slightly different (∼115x115 μm2, ∼0.45 μm/pixel vs. ∼128x128 μm2, ∼0.5 μm/pixel). A total of 139 patients treated by surgical resection in Henri Mondor University Hospital were included (Fig. 1). The most frequent risk factors for the liver disease were alcohol intake (31%, 43/138), HBV (24%, 34/138) and HCV (24%, 33/138) infection (Table S6). The disease stage, according to the BCLC (Barcelona Clinic of Liver Cancer) system, was 0/A in 78% of the patients and B/C in 22% of the patients. As observed in the discovery series, the frequency of cirrhosis was also low (35%, 34/97).
      After mRNA extraction, quality control and gene expression analysis using the NanoString PanCancer Immuno-Oncology 360™ Panel, we performed unsupervised hierarchical clustering on all samples. Tumors belonging to Cluster High for 6G IFNg, Gajewski 13G, Inflammatory, IFNg Biology, Ribas 10G, and T-cell Exhaustion signatures represented 14% (20/139), 30% (42/139), 9% (13/139), 16% (22/139), 12% (17/139) and 6% (8/139) of the cases, respectively (Table S7). As observed in the discovery series, an overlap among cases classified as Cluster High was identified (8 cases belonged to Cluster High for all signatures investigated). Cases classified as Cluster High were associated with the following clinical, biological and pathological features: poor differentiation (except Gajewski 13G), HCV infection (Gajewski 13G p = 0.03), higher age at surgery (6G IFNg p = 0.006, Ribas 10G p = 0.03), higher serum alpha-fetoprotein levels (Inflammatory p = 0.02, T-cell Exhaustion p = 0.002) and lower tumor size (Gajewski 13G p = 0.002) (Table S8).
      As performed for the discovery series, all tumor areas from the 139 WSIs were annotated by a pathologist (JC). A total of 1,555,984 patches were thus extracted and fed to the trained CLAM models developed in the discovery series. Samples were classified as Cluster High or Median/Low using the optimal thresholds identified on the test splits of the discovery series.
      We were able to validate their performance with AUCs of 0.817, 0.810, 0.850, 0.823, 0.810 and 0.921 for 6G IFNg, Gajewski 13G, Inflammatory, IFNg Biology, Ribas 10G and T-cell Exhaustion gene signatures, respectively (Table 2). The receiver-operating characteristic curves and confusion matrices are displayed in Fig. 5. We also investigated 3 different techniques that may increase the overall performance of our models: i) conversion of our hematein-eosin-saffron WSIs to hematein-eosin (staining used in the discovery series), ii) color normalization and iii) data augmentation techniques (during training) (supplementary materials and methods). We did not observe any significant improvement (Table S9-11).
      Table 2Performances (AUCs) of the best-fold models in the validation series, using annotated WSIs.
      Gene signatureCLAM AUC
      6G Interferon Gamma0.817
      Gajewski 13G Inflammatory0.810
      Inflammatory0.850
      Interferon Gamma Biology0.823
      Ribas 10G Inflammatory0.810
      T-cell Exhaustion0.921
      AUC, area under the receiver operating characteristic curve; CLAM, clustering-constrained attention multiple-instance learning; G, -gene; WSIs, whole-slide digital histological images.
      Figure thumbnail gr5
      Fig. 5Validation of best-fold CLAM models in the validation series (Mondor).
      For the 6 gene signatures, clustering heatmaps, ROC curves and confusion matrices are provided (best-fold models using the optimal thresholds determined on the discovery series). AUC, area under the ROC curve; CLAM, clustering-constrained attention multiple-instance learning; ROC, receiver-operating characteristic. (This figure appears in color on the web.)
      Finally, we were able to analyze pre-operative biopsies from 7 cases. WSIs were processed using our best-fold CLAM models and we observed that tumor immune clusters were accurately predicted in 38 out of the 42 cases (7 cases x 6 gene signatures) (Fig. S1).

      Generation of heatmaps and pathological review of areas with high predictive value

      In order to have a better understanding of the morphological and biological features involved in the classification process, we extracted, for each gene signature, the top 8 most predictive and the top 8 most non-informative patches from all tumors accurately classified as Cluster High (examples in Fig. 6). A total of 1,264 image patches were thus reviewed and 23 histological and cytological features were recorded for each patch. Inter-observer agreement between the 2 pathologists was substantial with Cohen Kappa values ranging from 0.69 to 0.88. We observed highly significant differences between highly predictive and non-informative areas for the following signatures:
      Figure thumbnail gr6
      Fig. 6Pathological reviewing of highly predictive patches.
      Microscopic examination of highly predictive patches showed enrichment of particular immune-related features. Examples for 6G Interferon Gamma and Interferon Gamma Biology signatures are provided in panels A and B, respectively. Patches associated with 6G Interferon Gamma included lymphocytes (yellow arrows) and plasma cells (white arrows). We also identified enrichment in neutrophils on patches associated with Interferon Gamma Biology (red arrows). G, -gene. (This figure appears in color on the web.)
      6G IFNg: presence of lymphocytes (p <0.0001), neutrophils (p <0.0001), plasma cells (p = 0.001) and lack of blood (p <0.0001), tumor cells (p <0.0001), steatosis (p = 0.03), fibrosis (p = 0.01), and macrotrabecular (p = 0.003) and compact (p = 0.03) tumor growth patterns.
      IFNg Biology: presence of lymphocytes (p <0.0001), neutrophils (p <0.0001) and plasma cells (p = 0.007) and lack of tumor cells (p <0.0001), steatosis (p <0.0001), fibrosis (p <0.0001) and compact growth pattern (p <0.0001).
      Ribas 10G: presence of lymphocytes (p <0.0001), neutrophils (p = 0.003) and plasma cells (p = 0.001) and lack of tumor cells (p <0.0001), steatosis (p = 0.006), compact growth pattern (p <0.0001) and atypia (p = 0.01).
      No significant associations were observed for the remaining 3 gene signatures: Gajewski 13G, Inflammatory and T-cell Exhaustion (Table S12).
      We finally compared the frequency of the 23 pathological features on the most predictive patches between the 6 signatures investigated and observed several significant differences (Table S13). Altogether, these findings show that the morphological characteristics captured by the models are, at least in part, different.

      Discussion

      Numerous gene signatures or transcriptomic subgroups have been proposed to better select patients that may benefit from particular targeted therapies.
      • Havel J.J.
      • Chowell D.
      • Chan T.A.
      The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy.
      However, the use of such biomarkers requires molecular biology and bioinformatics expertise, and very few are used in clinical practice.
      We show, in this proof-of-concept study, that deep learning applied to digital histological slides has the ability to predict several immune gene signatures related to response to immunotherapy.
      • Sangro B.
      • Melero I.
      • Wadhawan S.
      • Finn R.S.
      • Abou-Alfa G.K.
      • Cheng A.-L.
      • et al.
      Association of inflammatory biomarkers with clinical outcomes in nivolumab-treated patients with advanced hepatocellular carcinoma.
      Although the relevance of these signatures remains to be prospectively validated, our results demonstrate that AI-based pathology is a promising approach to extract clinically and biologically significant information from WSIs of HCC.
      • Calderaro J.
      • Kather J.N.
      Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers.
      ,
      • Saillard C.
      • Schmauch B.
      • Laifa O.
      • Moarii M.
      • Toldo S.
      • Zaslavskiy M.
      • et al.
      Predicting survival after hepatocellular carcinoma resection using deep-learning on histological slides.
      • Calderaro J.
      • Ziol M.
      • Paradis V.
      • Zucman-Rossi J.
      Molecular and histological correlations in liver cancer.
      • Ziol M.
      • Poté N.
      • Amaddeo G.
      • Laurent A.
      • Nault J.-C.
      • Oberti F.
      • et al.
      Macrotrabecular-massive hepatocellular carcinoma: a distinctive histological subtype with clinical relevance.
      • Calderaro J.
      • Couchy G.
      • Imbeaud S.
      • Amaddeo G.
      • Letouzé E.
      • Blanc J.-F.
      • et al.
      Histological subtypes of hepatocellular carcinoma are related to gene mutations and molecular tumour classification.
      • Calderaro J.
      • Rousseau B.
      • Amaddeo G.
      • Mercey M.
      • Charpy C.
      • Costentin C.
      • et al.
      Programmed death ligand 1 expression in hepatocellular carcinoma: relationship with clinical and pathological features.
      We investigated 3 different deep learning approaches and showed that the CLAM network yielded the best overall performance. In addition to the patch-based approach and the classic MIL, CLAM uses an attention-based learning process to automatically identify particular areas of high diagnostic value within a WSI.
      • Lu M.Y.
      • Williamson D.F.K.
      • Chen T.Y.
      • Chen R.J.
      • Barbieri M.
      • Mahmood F.
      Data-efficient and weakly supervised computational pathology on whole-slide images.
      ,
      • Lu M.Y.
      • Chen T.Y.
      • Williamson D.F.K.
      • Zhao M.
      • Shady M.
      • Lipkova J.
      • et al.
      AI-based pathology predicts origins for cancers of unknown primary.
      It involves an adaptive weighting for each patch, and the model can thus make more granular and flexible predictions. This feature allows one to automatically lower the impact of irrelevant tissue patches and may contribute, at least in part, to the higher performance of this method. Although this type of model is designed to reduce the need for manual annotations, we show that tumor area delineation by a pathologist results in a significant improvement for all gene signatures investigated. We may hypothesize that the negative impact of irrelevant noise/patterns from the adjacent non-tumor parenchyma was avoided by the use of expert-driven annotations. These findings underscore the importance of human-machine interactions for the development of highly efficient AI-based pathology.
      The processing of surgical and biopsy samples is complex with several critical steps (embedding, cutting, staining, and scanning) that may impact the overall quality of the WSIs. Models can easily overfit and thus learn noise or fluctuations that are specific to the training dataset, explaining why they most often do not generalize well on external datasets. Technical protocols for gene expression analyses also include multiple steps that are impacted by experimental conditions (RNA extraction, reverse transcription, and amplification, for example) and may introduce non-linear effects. The identification of robust tumor subgroups across different technological platforms thus remains challenging. These issues explain why, although numerous studies using deep learning on WSIs have recently been published, the majority of the proposed models were not validated on true independent series of samples processed by different histological and molecular protocols. We thus believe that one of the strengths of our work is the validation to which the various models investigated have been subjected. We were indeed able to validate our different models on a completely independent dataset that included i) patients treated in a different center, ii) slides stained with different protocols and encoded in a different format and iii) gene expression experiments performed using a different technology. The performance of our classifiers, as assessed by the AUC, was also in most cases >0.80, which is usually considered “excellent”.
      • Hosmer Jr., D.W.
      • Lemeshow S.
      • Sturdivant R.X.
      Assessing the fit of the model.
      By changing the classification thresholds, sensitivity and specificity of our AI-based pathology assays can also be modulated according to particular clinical settings/needs. Deep learning models using thresholds with high sensitivity can indeed be used to pre-screen patients and further trigger confirmatory gene expression profiling in case of a positive prediction. Even with suboptimal specificity, such assays may have the ability to speed up diagnostic workflows.
      The next step for the implementation of such models will be their broad validation on HCC biopsies (the only type of samples available for patients with advanced disease). Our preliminary results are encouraging, however, the investigation of larger series will be mandatory to determine if such samples are suitable for this type of AI-driven analysis. It may be challenging as they are rarely performed due to the existence of non-invasive diagnostic criteria. There is however a renewed interest in biopsy, in particular in the context of clinical trials, and several studies have shown that their analysis can provide meaningful molecular and prognostic information.
      • Nault J.
      • Martin Y.
      • Caruso S.
      • Hirsch T.Z.
      • Bayard Q.
      • Calderaro J.
      • et al.
      Clinical impact of genomic diversity from early to advanced hepatocellular carcinoma.
      We may thus be able to perform a large-scale validation of our models on this type of sample within the foreseeable future. Trials investigating neoadjuvant immunotherapy may also encourage us to reconsider the practice of biopsy even in the context of small, resectable tumors. Several drugs, including immunomodulatory molecules, are currently being tested in the adjuvant setting (after surgical resection) and our models may help to identify the patients most likely to benefit from these approaches.
      • Haber P.K.
      • Puigvehí M.
      • Castet F.
      • Lourdusamy V.
      • Montal R.
      • Tabrizian P.
      • et al.
      Evidence-based management of HCC: systematic review and meta-analysis of randomized controlled trials (2002-2020).
      • Hack S.P.
      • Spahn J.
      • Chen M.
      • Cheng A.-L.
      • Kaseb A.
      • Kudo M.
      • et al.
      IMbrave 050: a Phase III trial of atezolizumab plus bevacizumab in high-risk hepatocellular carcinoma after curative resection or ablation.
      • Pan Q.-Z.
      • Liu Q.
      • Zhou Y.-Q.
      • Zhao J.-J.
      • Wang Q.-J.
      • Li Y.-Q.
      • et al.
      CIK cell cytotoxicity is a predictive biomarker for CIK cell immunotherapy in postoperative patients with hepatocellular carcinoma.
      • Su Y.-Y.
      • Li C.-C.
      • Lin Y.-J.
      • Hsu C.
      Adjuvant versus neoadjuvant immunotherapy for hepatocellular carcinoma: clinical and immunologic perspectives.
      In order for AI-based algorithms to be used in daily practice, they must be prospectively validated in “real life” clinical workflows to show that they are able to provide useful information in a timely manner. Indeed, model development is the first step of an overall process that includes several technical (image acquisition/storage, reproducibility and robustness), regulatory (quality control framework, data privacy), and clinical (demonstration of improved clinical outcomes) barriers. Improvements in the error/misclassification rates are also critical for the adoption of models such as ours.
      Convolutional neural networks consist of multiple layers of complex mathematical computation, and deep learning models are thus often considered as black boxes. We took advantage of a particular feature of the CLAM method to generate attention maps and analyze the areas with high predictive value. It does not provide true explainability but highlights patches that may contain relevant information. For 3 signatures related to interferon gamma signaling, we were able to show that classification as Cluster High relied on patches containing lymphocytes, neutrophils and plasma cells. These observations are consistent with the function of the genes included, as molecules encoded by genes such as IFNG, CXCL10 or CXCL11 are known to promote the recruitment of various immune cell subsets. We did not identify any histological features associated with the 3 remaining signatures, suggesting that the models are also able to capture morphological characteristics that are not easily accessible to the human eye. We believe that the use of explainable models is key to gaining the required trust of pathologists and physicians, and thus easing their implementation in clinical practice. The analysis of tissue areas associated with particular predictions may also lead to new scientific discoveries.
      In conclusion, we have developed and validated deep learning-based models able to predict the activation of several immune gene signatures that may be associated with improved response to immunotherapy in patients with HCC. These approaches could represent a novel type of biomarker that will ease the translation of our biological knowledge of HCC into clinical practice.

      Abbreviations

      AI, artificial intelligence; CLAM, clustering-constrained attention multiple-instance learning; FC, fully connected; HCC, hepatocellular carcinoma; MIL, multiple-instance learning; TCGA-LIHC, The Cancer Genome Atlas Liver Hepatocellular Carcinoma; WSI(s), whole-slide digital histological image(s).

      Financial support

      Fondation Bristol Myers Squibb pour la Recherche en Immuno-Oncologie, Fondation de l'Avenir, CARPEM, and China Scholarship Council (CSC).

      Authors’ contributions

      Study conception and design: JC, NL, CK. Data collection: DS, AL, GA, CT, RB, HR, DG, AR, PM, CTN. Data analysis: QZ, CK, DG, AR, SC, NGL, JNK, CTN, JMP, MCM. Drafting the manuscript: QZ, CK, JC, NL, SC. Obtained funding: JC. Critical revision of the manuscript: All authors.

      Data availability statement

      All our code is available at the following website: https://github.com/qinghezeng/Histo2GeneSignatures. The TCGA-LIHC data including gene expression and WSIs are available from NIH GDC Data Portal (https://portal.gdc.cancer.gov/).

      Conflicts of interest

      JC consults for Crosscope, Keen Eye, and has received research funding from Fondation Bristol Myers Squibb pour la Recherche en Immuno-Oncologie.
      Please refer to the accompanying ICMJE disclosure forms for further details.

      Supplementary data

      The following are the supplementary data to this article:

      References

        • Llovet J.M.
        • Kelley R.K.
        • Villanueva A.
        • Singal A.G.
        • Pikarsky E.
        • Roayaie S.
        • et al.
        Hepatocellular carcinoma.
        Nat Rev Dis Prim. 2021; 7: 6https://doi.org/10.1038/s41572-020-00240-3
        • Llovet J.M.
        • Montal R.
        • Sia D.
        • Finn R.S.
        Molecular therapies and precision medicine for hepatocellular carcinoma.
        Nat Rev Clin Oncol. 2018; 15: 599-616https://doi.org/10.1038/s41571-018-0073-4
        • Kudo M.
        • Finn R.S.
        • Qin S.
        • Han K.H.
        • Ikeda K.
        • Piscaglia F.
        • et al.
        Lenvatinib versus sorafenib in first-line treatment of patients with unresectable hepatocellular carcinoma: a randomised phase 3 non-inferiority trial.
        Lancet. 2018; 391: 1163-1173https://doi.org/10.1016/S0140-6736(18)30207-1
        • Abou-Alfa G.K.
        • Meyer T.
        • Cheng A.-L.
        • El-Khoueiry A.B.
        • Rimassa L.
        • Ryoo B.-Y.
        • et al.
        Cabozantinib in patients with advanced and progressing hepatocellular carcinoma.
        N Engl J Med. 2018; 379: 54-63https://doi.org/10.1056/NEJMoa1717002
        • Zhu A.X.
        • Kang Y.K.
        • Yen C.J.
        • Finn R.S.
        • Galle P.R.
        • Llovet J.M.
        • et al.
        Ramucirumab after sorafenib in patients with advanced hepatocellular carcinoma and increased α-fetoprotein concentrations (REACH-2): a randomised, double-blind, placebo-controlled, phase 3 trial.
        Lancet Oncol. 2019; 20: 282-296https://doi.org/10.1016/S1470-2045(18)30937-9
        • Bruix J.
        • Qin S.
        • Merle P.
        • Granito A.
        • Huang Y.-H.
        • Bodoky G.
        • et al.
        Regorafenib for patients with hepatocellular carcinoma who progressed on sorafenib treatment (RESORCE): a randomised, double-blind, placebo-controlled, phase 3 trial.
        Lancet. 2017; 389: 56-66https://doi.org/10.1016/S0140-6736(16)32453-9
        • Finn R.S.
        • Qin S.
        • Ikeda M.
        • Galle P.R.
        • Ducreux M.
        • Kim T.-Y.
        • et al.
        Atezolizumab plus bevacizumab in unresectable hepatocellular carcinoma.
        N Engl J Med. 2020; 382: 1894-1905https://doi.org/10.1056/NEJMoa1915745
        • Sangro B.
        • Sarobe P.
        • Hervás-Stubbs S.
        • Melero I.
        Advances in immunotherapy for hepatocellular carcinoma.
        Nat Rev Gastroenterol Hepatol. 2021; : 1-19https://doi.org/10.1038/s41575-021-00438-0
        • Sangro B.
        • Melero I.
        • Wadhawan S.
        • Finn R.S.
        • Abou-Alfa G.K.
        • Cheng A.-L.
        • et al.
        Association of inflammatory biomarkers with clinical outcomes in nivolumab-treated patients with advanced hepatocellular carcinoma.
        J Hepatol. 2020; 73: 1460-1469https://doi.org/10.1016/j.jhep.2020.07.026
        • Haber P.K.
        • Torres-Martin M.
        • Dufour J.-F.
        • Verslype C.
        • Marquardt J.
        • Galle P.R.
        • et al.
        Molecular markers of response to anti-PD1 therapy in advanced hepatocellular carcinoma.
        J Clin Oncol. 2021; 39 (4100–4100)https://doi.org/10.1200/JCO.2021.39.15_suppl.4100
        • Havel J.J.
        • Chowell D.
        • Chan T.A.
        The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy.
        Nat Rev Cancer. 2019; 19: 133-150https://doi.org/10.1038/s41568-019-0116-x
        • Coudray N.
        • Ocampo P.S.
        • Sakellaropoulos T.
        • Narula N.
        • Snuderl M.
        • Fenyö D.
        • et al.
        Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.
        Nat Med. 2018; 24: 1559-1567https://doi.org/10.1038/s41591-018-0177-5
        • Echle A.
        • Grabsch H.I.
        • Quirke P.
        • van den Brandt P.A.
        • West N.P.
        • Hutchins G.G.A.
        • et al.
        Clinical-grade detection of microsatellite instability in colorectal tumors by deep learning.
        Gastroenterology. 2020; 159: 1406-1416.e11https://doi.org/10.1053/j.gastro.2020.06.021
        • Kather J.N.
        • Calderaro J.
        Development of AI-based pathology biomarkers in gastrointestinal and liver cancer.
        Nat Rev Gastroenterol Hepatol. 2020; 17: 591-592https://doi.org/10.1038/s41575-020-0343-3
        • Calderaro J.
        • Kather J.N.
        Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers.
        Gut. 2021; 70: 1183-1193https://doi.org/10.1136/gutjnl-2020-322880
        • Cancer Genome Atlas Research Network
        Comprehensive and integrative genomic characterization of hepatocellular carcinoma.
        Cell. 2017; 169: 1327-1341.e23https://doi.org/10.1016/j.cell.2017.05.046
        • Ayers M.
        • Lunceford J.
        • Nebozhyn M.
        • Murphy E.
        • Loboda A.
        • Kaufman D.R.
        • et al.
        IFN-γ–related mRNA profile predicts clinical response to PD-1 blockade.
        J Clin Invest. 2017; 127: 2930-2940https://doi.org/10.1172/JCI91190
        • Spranger S.
        • Bao R.
        • Gajewski T.F.
        Melanoma-intrinsic β-catenin signalling prevents anti-tumour immunity.
        Nature. 2015; 523: 231-235https://doi.org/10.1038/nature14404
        • Murtagh F.
        • Legendre P.
        Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion?.
        J Classif. 2014; 31: 274-295https://doi.org/10.1007/s00357-014-9161-z
        • Deza M.M.
        • Deza E.
        Encyclopedia of distances.
        Springer, Berlin, Heidelberg2009
        • Bankhead P.
        • Loughrey M.B.
        • Fernández J.A.
        • Dombrowski Y.
        • McArt D.G.
        • Dunne P.D.
        • et al.
        QuPath: open source software for digital pathology image analysis.
        Sci Rep. 2017; 7: 16878https://doi.org/10.1038/s41598-017-17204-5
        • Lu M.Y.
        • Williamson D.F.K.
        • Chen T.Y.
        • Chen R.J.
        • Barbieri M.
        • Mahmood F.
        Data-efficient and weakly supervised computational pathology on whole-slide images.
        Nat Biomed Eng. 2021; 5: 555-570https://doi.org/10.1038/s41551-020-00682-w
        • Lu M.Y.
        • Chen T.Y.
        • Williamson D.F.K.
        • Zhao M.
        • Shady M.
        • Lipkova J.
        • et al.
        AI-based pathology predicts origins for cancers of unknown primary.
        Nature. 2021; 594: 106-110https://doi.org/10.1038/s41586-021-03512-4
        • Saillard C.
        • Schmauch B.
        • Laifa O.
        • Moarii M.
        • Toldo S.
        • Zaslavskiy M.
        • et al.
        Predicting survival after hepatocellular carcinoma resection using deep-learning on histological slides.
        Hepatology. 2020; (hep.31207)https://doi.org/10.1002/hep.31207
        • Calderaro J.
        • Ziol M.
        • Paradis V.
        • Zucman-Rossi J.
        Molecular and histological correlations in liver cancer.
        J Hepatol. 2019; 71: 616-630https://doi.org/10.1016/j.jhep.2019.06.001
        • Ziol M.
        • Poté N.
        • Amaddeo G.
        • Laurent A.
        • Nault J.-C.
        • Oberti F.
        • et al.
        Macrotrabecular-massive hepatocellular carcinoma: a distinctive histological subtype with clinical relevance.
        Hepatology. 2018; 68: 103-112https://doi.org/10.1002/hep.29762
        • Calderaro J.
        • Couchy G.
        • Imbeaud S.
        • Amaddeo G.
        • Letouzé E.
        • Blanc J.-F.
        • et al.
        Histological subtypes of hepatocellular carcinoma are related to gene mutations and molecular tumour classification.
        J Hepatol. 2017; 67: 727-738https://doi.org/10.1016/j.jhep.2017.05.014
        • Calderaro J.
        • Rousseau B.
        • Amaddeo G.
        • Mercey M.
        • Charpy C.
        • Costentin C.
        • et al.
        Programmed death ligand 1 expression in hepatocellular carcinoma: relationship with clinical and pathological features.
        Hepatology. 2016; 64: 2038-2046https://doi.org/10.1002/hep.28710
        • Hosmer Jr., D.W.
        • Lemeshow S.
        • Sturdivant R.X.
        Assessing the fit of the model.
        2nd ed. Appl Logist Regression. vol. 398. John Wiley & Sons, New York2013: 160-164https://doi.org/10.2307/2532419
        • Nault J.
        • Martin Y.
        • Caruso S.
        • Hirsch T.Z.
        • Bayard Q.
        • Calderaro J.
        • et al.
        Clinical impact of genomic diversity from early to advanced hepatocellular carcinoma.
        Hepatology. 2020; 71: 164-182https://doi.org/10.1002/hep.30811
        • Haber P.K.
        • Puigvehí M.
        • Castet F.
        • Lourdusamy V.
        • Montal R.
        • Tabrizian P.
        • et al.
        Evidence-based management of HCC: systematic review and meta-analysis of randomized controlled trials (2002-2020).
        Gastroenterology. 2021; https://doi.org/10.1053/j.gastro.2021.06.008
        • Hack S.P.
        • Spahn J.
        • Chen M.
        • Cheng A.-L.
        • Kaseb A.
        • Kudo M.
        • et al.
        IMbrave 050: a Phase III trial of atezolizumab plus bevacizumab in high-risk hepatocellular carcinoma after curative resection or ablation.
        Futur Oncol. 2020; 16: 975-989https://doi.org/10.2217/fon-2020-0162
        • Pan Q.-Z.
        • Liu Q.
        • Zhou Y.-Q.
        • Zhao J.-J.
        • Wang Q.-J.
        • Li Y.-Q.
        • et al.
        CIK cell cytotoxicity is a predictive biomarker for CIK cell immunotherapy in postoperative patients with hepatocellular carcinoma.
        Cancer Immunol Immunother. 2020; 69: 825-834https://doi.org/10.1007/s00262-020-02486-y
        • Su Y.-Y.
        • Li C.-C.
        • Lin Y.-J.
        • Hsu C.
        Adjuvant versus neoadjuvant immunotherapy for hepatocellular carcinoma: clinical and immunologic perspectives.
        Semin Liver Dis. 2021; https://doi.org/10.1055/s-0041-1730949