Deep learning-based lymphocyte infiltration detection on pathological images

Han ZHUANG; Weigang HU; Zhen ZHANG; Jiazhou WANG

doi:10.19401/j.cnki.1007-3639.2024.04.008

您当前的位置：

首页 >

文章列表页 >

Deep learning-based lymphocyte infiltration detection on pathological images

Article | 更新时间：2025-12-31

- Deep learning-based lymphocyte infiltration detection on pathological images
- China Oncology Vol. 34, Issue 4, Pages: 409-417(2024)
- 作者机构：
  
  复旦大学附属肿瘤医院放射治疗中心，复旦大学上海医学院肿瘤学系，上海市放射肿瘤学重点实验室，上海市放射治疗临床医学研究中心，上海 200023
- 作者简介：
- 基金信息：
- DOI：10.19401/j.cnki.1007-3639.2024.04.008
  CLC： R730.2
- Received：05 September 2023，
  
  Revised：2024-04-09，
  
  Published：30 April 2024
- 稿件说明：
移动端阅览
Han ZHUANG, Weigang HU, Zhen ZHANG, et al. Deep learning-based lymphocyte infiltration detection on pathological images[J]. China Oncology, 2024, 34(4): 409-417.
DOI：

Han ZHUANG, Weigang HU, Zhen ZHANG, et al. Deep learning-based lymphocyte infiltration detection on pathological images[J]. China Oncology, 2024, 34(4): 409-417. DOI： 10.19401/j.cnki.1007-3639.2024.04.008.

摘要

背景与目的：

深度学习方法可用来在病理切片上开展淋巴细胞的自动分割和检测。本研究探讨使用变分自编码模型预训练的方法对病理学图片进行淋巴细胞浸润检测的性能，以及去除肿瘤坏死区域对模型性能的影响。

方法：

使用变分自编码模型（variational auto-encoder，VAE）先在来自肿瘤基因组图谱（the Cancer Genome Atlas

TCGA）数据库中的无标注的TCGA-COAD和TCGA-READ 病理切片图像上进行预训练，获得一个自编码预训练模型，再在已标注淋巴细胞的公开数据集上进行淋巴细胞浸润检测的模型训练。为避免与坏死的区域相混淆，还训练了分割肿瘤坏死区域的Unet模型，以去除肿瘤坏死区域对淋巴细胞识别的影响。分析经过变分自编码模型预训练的淋巴细胞浸润检测模型在训练集和测试集上的受试者工作特征曲线（receiver operating characteri

stic curve，ROC）的曲线下面积（area under curve，AUC）及其95%置信区间（confidence interval，CI）。

结果：

检测模型在训练集上AUC为0.979（95% CI：0.978 ~ 0.980），准确率为92.5%（95% CI：92.3% ~ 92.6%），Kappa值为0.849，灵敏度为0.908，特异度为0.941，精确率为0.939，召回率为0.908，F1为0.923。验证集的AUC为0.968（95% CI：0.964 ~ 0.972），准确率为91.3%（95% CI：90.6% ~ 92.0%），Kappa值为0.826，灵敏度为0.898，特异度为0.928，精确率为0.925，召回率为0.898，F1为0.912。Resnet18模型在已标注的数据集上直接训练的结果为：验证集的准确率为83.2%（95% CI：82.2% ~ 84.1%），Kappa值为0.664，灵敏度为0.823，特异度为0.840，精确率为0.838，召回率为0.823，F1为0.830。分割肿瘤坏死区域的Unet模型最终在训练集上的DICE值为0.78，在验证集上为0.76。去除坏死区域后本文提出的变分自编码模型预训练的淋巴细胞浸润检测模型的预测性能获得了一定的提升，在验证集上的AUC从0.968（95% CI：0.964 ~ 0.972）提升至0.971（95% CI：0.968 ~ 0.975）。准确率为92.4%（95% CI：91.7% ~ 93.0%），Kappa值为0.849，灵敏度为0.928，特异度为0.921，精确率为0.921，召回率为0.928，F1为0.925。

结论：

采用变分自编码模型预训练的方法，对淋巴细胞浸润的病理学图片进行浸润检测，可获得比直接训练更优的模型表现；并且通过Unet去除肿瘤坏死区域的影响能够进一步提高模型的性能。

Abstract

Background and purpose:

Deep learning methods can be used for automatic segmentation and detection of lymphocytes on pathological images. This study aimed to assess the performance of using variational autoencoding pre-training method for lymphocyte infiltration detection on pathological images

as well as the impact of removing tumor necrosis regions on model performance.

Methods:

Using variational autoencoding (VAE) pre-training method

pre-training was performed on a large number of unlabeled pathological images from the Cancer Genome Atlas (TCGA) database (TCGA-COAD and TCGA-READ) to obtain an auto-encoding pre-training model

and then a classifier model of lymphocyte infiltration was trained on the public data samples. To avoid confusion with necrotic regions

a Unet segmentation model for tumor necrotic regions was trained to remove the influence of tumor necrotic regions on lymphocyte identification.

Results:

The lymphocyte infiltration detection model pre-trained with the VAE model had an area under curve (AUC) of 0.979 (95% CI: 0.978-0.980)

an accuracy of

92.5% (95% CI: 92.3%-92.6%)

a kappa value of 0.849

sensitivity of 0.908

specificity of 0.941

precision of 0.939

recall of 0.908

and F1 of 0.923 under the receiver operating characteristic (ROC) curve on the training set. The AUC for the validation set was 0.968 (95% CI: 0.964-0.972)

the accuracy was 91.3% (95% CI: 90.6%-92.0%)

kappa value was 0.826

sensitivity was 0.898

specificity was 0.928

precision was 0.925

recall was 0.898

and F1 was 0.912. The results of Resnet18 model on the labeled dataset were as follows: accuracy of the validation set was 83.2 % (95% CI: 82.2%-84.1%)

kappa value was 0.664

sensitivity was 0.823

specificity was 0.840

precision was 0.838

recall was 0.823 and F1 was 0.830. The Unet model that segmented the necrotic regions of the tumors had a final DICE of 0.78 for the training set

and 0.76 for the validation. After removing the necrotic region

the predictive performance of the pre-trained lymphocyte infiltration detection model using the VAE proposed in this article was improved to some extent

with the AUC on the validation set increasing from 0.968 (95% CI: 0.964-0.972) to 0.971 (95% CI: 0.968-0.975). The accuracy was 92.4% (95% CI: 91.7%-93.0%)

kappa value was 0.849

sensitivity was 0.928

specificity was 0.921

precision was 0.921

recall was 0.928

and F1 was 0.925.

Conclusion:

Using the variational autoencoding model pre-training method to classify the pathological pictures of lymphocyte infiltration can obtain better model performance compared with direct training

and removing the influence of tumor necrosis areas through Unet can further improve the performance of the model.

关键词

Keywords

references

FRIDMAN W H , PAGÈS F , SAUTÈS-FRIDMAN C , et al . The immune contexture in human tumours: impact on clinical outcome [J ] . Nat Rev Cancer , 2012 , 12 ( 4 ): 298 - 306 . DOI: 10.1038/nrc3245 http://doi.org/10.1038/nrc3245

SALGADO R , DENKERT C , DEMARIA S , et al . The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: recommendations by an International TILs Working Group 2014 [J ] . Ann Oncol , 2015 , 26 ( 2 ): 259 - 271 . DOI: 10.1093/annonc/mdu450 http://doi.org/10.1093/annonc/mdu450

ORHAN A , VOGELSANG R P , ANDERSEN M B , et al . The prognostic value of tumour-infiltrating lymphocytes in pancreatic cancer: a systematic review and meta-analysis [J ] . Eur J Cancer , 2020 , 132 : 71 - 84 . DOI: S0959-8049(20)30154-4 http://doi.org/S0959-8049(20)30154-4

FU Q F , CHEN N , GE C L , et al . Prognostic value of tumor-infiltrating lymphocytes in melanoma: a systematic review and meta-analysis [J ] . Oncoimmunology , 2019 , 8 ( 7 ): 1593806 .

ALLARD M A , BACHET J B , BEAUCHET A , et al . Linear quantification of lymphoid infiltration of the tumor margin: a reproducible method, developed with colorectal cancer tissues, for assessing a highly variable prognostic factor [J ] . Diagn Pathol , 2012 , 7 : 156 .

ROBINS H S , ERICSON N G , GUENTHOER J , et al . Digital genomic quantification of tumor-infiltrating lymphocytes [J ] . Sci Transl Med , 2013 , 5 ( 214 ): 214ra169.

ERIKSEN A C , ANDERSEN J B , KRISTENSSON M , et al . Computer-assisted stereology and automated image analysis for quantification of tumor infiltrating lymphocytes in colon cancer [J ] . Diagn Pathol , 2017 , 12 ( 1 ): 65 . DOI: 10.1186/s13000-017-0653-0 http://doi.org/10.1186/s13000-017-0653-0

YIN Y D , ROTHENBERG E . Probing the spatial organization of molecular complexes using triple-pair-correlation [J ] . Sci Rep , 2016 , 6 : 30819 . DOI: 10.1038/srep30819 http://doi.org/10.1038/srep30819

CORREDOR G , WANG X X , LU C , et al . A watershed and feature-based approach for automated detection of lymphocytes on lung cancer images [C ] // SPIE Medical Imaging. Proc SPIE 10581, Medical Imaging 2018:Digital Pathology, Houston, Texas, USA. 2018 , 10581 : 213 - 218 .

BASAVANHALLY A N , GANESAN S , AGNER S , et al . Computerized image-based detection and grading of lymphocy tic infiltration in HER2 + breast cancer histopathology [J ] . IEEE Trans Biomed Eng , 2010 , 57 ( 3 ): 642 - 653 .

KUO Y L , KO C C , LEE M J . Lymphatic infiltration detection in breast cancer H-E image prior to lymphadenectomy [J ] . Biomed Eng Appl Basis Commun , 2014 , 26 ( 4 ): 1440007 .

WANG S D , WANG T , YANG L , et al . ConvPath: a software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network [J ] . EBioMedicine , 2019 , 50 : 103 - 110 . DOI: S2352-3964(19)30703-0 http://doi.org/S2352-3964(19)30703-0

WOUTERS M C , KOMDEUR F L , WORKEL H H , et al . Treatment regimen, surgical outcome, and T-cell differentiation influence prognostic benefit of tumor-infiltrating lymphocytes in high-grade serous ovarian cancer [J ] . Clin Cancer Res , 2016 , 22 ( 3 ): 714 - 724 . DOI: 10.1158/1078-0432.CCR-15-1617 http://doi.org/10.1158/1078-0432.CCR-15-1617

Views

1043

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Artificial intelligence in gastric cancer diagnosis, treatment and prognostic prediction: current application and future perspective

Current status and challenges of artificial intelligence-enabled prediction of synergistic cancer drug combinations

Research progress of artificial intelligence based on deep learning in digital pathology

Research progress and prospects of MRI in predicting lymph node status in breast cancer

MRI-based radiomics and deep learning model construction: non-invasive differentiation of molecular subtypes in primary intracranial diffuse large B-cell lymphoma

Related Author

Ning LU

Ziye WAN

Dongge PENG

Yi XIONG

Shenggeng LIN

Hong TAN

章　真

杨　鑫

Related Institution

Department of Oncology, Xinjiang Military Region General Hospital of the Chinese People’s Liberation Army, Urumqi 830000, Xinjiang Uygur Autonomous Region

Graduate School of Xinjiang Medical university, Urumqi 830000, Xinjiang Uygur Autonomous Region

Artificial Intelligence Biomedical Center, Zhangjiang Institute for Advanced Study, Shanghai Jiao Tong University

School of Life Sciences and Biotechnology, Shanghai Jiao Tong University

复旦大学附属肿瘤医院放射治疗中心，复旦大学上海医学院肿瘤学系

AI问答

⁰