Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer

Mengqi JIANG; Yuchen HAN; Xiaolong FU

doi:10.19401/j.cnki.1007-3639.2024.03.009

您当前的位置：

首页 >

文章列表页 >

Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer

Review | 更新时间：2025-12-31

- Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer
- China Oncology Vol. 34, Issue 3, Pages: 306-315(2024)
- 作者机构：
  
  1. 上海市胸科医院/上海交通大学医学院附属胸科医院放疗科，上海 200030
  2. 上海市胸科医院/上海交通大学医学院附属胸科医院病理科，上海 200030
- 作者简介：
- 基金信息：
- DOI：10.19401/j.cnki.1007-3639.2024.03.009
  CLC： R734.2
- Received：05 January 2024，
  
  Revised：2024-02-05，
  
  Published：30 March 2024
- 稿件说明：
移动端阅览
Mengqi JIANG, Yuchen HAN, Xiaolong FU. Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer[J]. China Oncology, 2024, 34(3): 306-315.
DOI：

Mengqi JIANG, Yuchen HAN, Xiaolong FU. Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer[J]. China Oncology, 2024, 34(3): 306-315. DOI： 10.19401/j.cnki.1007-3639.2024.03.009.

摘要

病理学是疾病诊断的金标准。利用全切片扫描技术将病理切片转化为数字图像后，人工智能特别是深度学习模型在病理学图像分析领域展现出了巨大潜力。人工智能在肺癌全切片扫描中的应用涉及组织病理学分型、肿瘤微环境分析、疗效及生存预测等多个方面，有望辅助临床进行精准治疗决策。然而标注数据不足、切片质量差异等因素也限制了病理学图像分析的发展。本文总结了肺癌领域利用人工智能手段进行病理学图像分析的应用进展，并对未来发展方向进行展望。

Abstract

Pathology is the gold standard for diagnosis of neoplastic diseases. Whole slide imaging turns traditional slides into digital images

and artificial intelligence has shown great potential in pathological image analysis

especially deep learning models. The application of artificial intelligence in whole slide imaging of lung cancer involves many aspects such as histopathological classification

tumor microenvironment analysis

efficacy and survival prediction

etc.

which is expected to assist clinical decision-making of accurate treatment. Limitations in this field include the lack of precisely annotated data and slide quality varying among institutions. Here we summarized recent research in lung cancer pathology image analysis leveraging artificial intelligence and proposed several future directions.

关键词

Keywords

references

JAHN S W , PLASS M , MOINFAR F . Digital pathology: Advantages, limitations and emerging perspectives [J ] . J Clin Med , 2020 , 9 ( 11 ): 3697.

MUKHOPADHYAY S , FELDMAN M D , ABELS E , et al . Whole slide imaging versus microscopy for primary diagnosis in surgical pathology: a multicenter blinded randomized noninferiority study of 1992 cases (pivotal study) [J ] . Am J Surg Pathol , 2018 , 42 ( 1 ): 39 - 52 . DOI: 10.1097/PAS.0000000000000948 http://doi.org/10.1097/PAS.0000000000000948

DA SILVA L M , PEREIRA E M , SALLES P G , et al . Independent real-world application of a clinical-grade automated prostate cancer detection system [J ] . J Pathol , 2021 , 254 ( 2 ): 147 - 158 . DOI: 10.1002/path.v254.2 http://doi.org/10.1002/path.v254.2 https://pathsocjournals.onlinelibrary.wiley.com/toc/10969896/254/2 https://pathsocjournals.onlinelibrary.wiley.com/toc/10969896/254/2

COUDRAY N , OCAMPO P S , SAKELLAROPOULOS T , et al . Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning [J ] . Nat Med , 2018 , 24 ( 10 ): 1559 - 1567 . DOI: 10.1038/s41591-018-0177-5 http://doi.org/10.1038/s41591-018-0177-5

ZHAO L , XU X W , HOU R P , et al . Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning [J ] . Phys Med Biol , 2021 , 66 ( 23 ).

KHADER F , KATHER J N , HAN T Y , et al . Cascaded cross-attention networks for data-efficient whole-slide image classification using transformers [M ] //CAO X, XU X, REKIK I, et al. International Worksho p on Machine Learning in Medical Imaging. Cham : Springer , 2024 : 417 - 426 .

YANG H , CHEN L L , CHENG Z Q , et al . Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study [J ] . BMC Med , 2021 , 19 ( 1 ): 80.

TERAMOTO A , TSUKAMOTO T , KIRIYAMA Y , et al . Automated classification of lung cancer types from cytological images using deep convolutional neural networks [J ] . Biomed Res Int , 2017 , 2017 : 4067832 .

TSUKAMOTO T , TERAMOTO A , YAMADA A , et al . Comparison of fine-tuned deep convolutional neural networks for the automated classification of lung cancer cytology images with integration of additional classifiers [J ] . Asian Pac J Cancer Prev , 2022 , 23 ( 4 ): 1315 - 1324 . DOI: 10.31557/APJCP.2022.23.4.1315 http://doi.org/10.31557/APJCP.2022.23.4.1315 http://journal.waocp.org/article_90065.html http://journal.waocp.org/article_90065.html

GUAN Q , WAN X C , LU H T , et al . Deep convolutional neural network Inception-v3 model for differential diagnosing of lymph node in cytological images: a pilot study [J ] . Ann Transl Med , 2019 , 7 ( 14 ): 307.

YANG J W , SONG D H , AN H J , et al . Classification of subtypes including LCNEC in lung cancer biopsy slides using convolutional neural network from scratch [J ] . Sci Rep , 2022 , 12 ( 1 ): 1830.

ILIÉ M , BENZAQUEN J , TOURNIAIRE P , et al . Deep learning facilitates distinguishing histologic subtypes of pulmonary neuroendocrine tumors on digital whole-slide images [J ] . Cancers , 2022 , 14 ( 7 ): 1740.

GONZALEZ D , DIETZ R L , PANTANOWITZ L . Feasibility of a deep learning algorithm to distinguish large cell neuroendocrine from small cell lung carcinoma in cytology specimens [J ] . Cytopathology , 2020 , 31 ( 5 ): 426 - 431 . DOI: 10.1111/cyt.12829 http://doi.org/10.1111/cyt.12829

TRAVIS W D , BRAMBILLA E , NOGUCHI M , et al . International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society international multidisciplinary classification of lung adenocarcinoma [J ] . J Thorac Oncol , 2011 , 6 ( 2 ): 244 - 285 . DOI: 10.1097/JTO.0b013e318206a221 http://doi.org/10.1097/JTO.0b013e318206a221

MOREIRA A L , OCAMPO P S S , XIA Y H , et al . A grading system for invasive pulmonary adenocarcinoma: a proposal from the international association for the study of lung cancer pathology committee [J ] . J Thorac Oncol , 2020 , 15 ( 10 ): 1599 - 1610 . DOI: S1556-0864(20)30468-8 http://doi.org/S1556-0864(20)30468-8

WEI J W , TAFE L J , LINNIK Y A , et al . Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks [J ] . Sci Rep , 2019 , 9 ( 1 ): 3358.

ZHAO Y L , HE S , ZHAO D , et al . Deep learning-based diagnosis of histopathological patterns for invasive non-mucinous lung adenocarcinoma using semantic segmentation [J ] . BMJ Open , 2023 , 13 ( 7 ): e069181.

ALSUBAIE N , RAZA S E A , SNEAD D , et al . Growth pattern fingerprinting for automatic analysis of lung adenocarcinoma overall survival [J ] . IEEE Access , 2023 , 11 : 23335 - 23346 . DOI: 10.1109/ACCESS.2023.3251220 http://doi.org/10.1109/ACCESS.2023.3251220 https://ieeexplore.ieee.org/document/10056841/ https://ieeexplore.ieee.org/document/10056841/

SADHWANI A , CHANG H W , BEHROOZ A , et al . Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images [J ] . Sci Rep , 2021 , 11 ( 1 ): 16605.

KARASAKI T , MOORE D A , VEERIAH S , et al . Evolutionary characterization of lung adenocarcinoma morphology in TRACERx [J ] . Nat Med , 2023 , 29 ( 4 ): 833 - 845 . DOI: 10.1038/s41591-023-02230-w http://doi.org/10.1038/s41591-023-02230-w

YU K H , BERRY G J , RUBIN D L , et al . Association of omics features with histopathology patterns in lung adenocarcinoma [J ] . Cell Syst , 2017 , 5 ( 6 ): 620 - 627 .e3.

MAYER C , OFEK E , FRIDRICH D E , et al . Direct identification of ALK and ROS 1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms [J ] . Mod Pathol , 2022 , 35 ( 12 ): 1882 - 1887 . DOI: 10.1038/s41379-022-01141-4 http://doi.org/10.1038/s41379-022-01141-4 https://linkinghub.elsevier.com/retrieve/pii/S0893395222055004 https://linkinghub.elsevier.com/retrieve/pii/S0893395222055004

TOMITA N , TAFE L J , SURIAWINATA A A , et al . Predicting oncogene mutations of lung cancer using deep learning and histopathologic features on whole-slide images [J ] . Transl Oncol , 2022 , 24 : 101494 . DOI: 10.1016/j.tranon.2022.101494 http://doi.org/10.1016/j.tranon.2022.101494 https://linkinghub.elsevier.com/retrieve/pii/S193652332200153X https://linkinghub.elsevier.com/retrieve/pii/S193652332200153X

TERADA Y , TAKAHASHI T , HAYAKAWA T , et al . Artificial intelligence-powered prediction of ALK gene rearrangement in patients with non-small-cell lung cancer [J ] . JCO Clin Cancer Inform , 2022 , 6 : e220 0070.

CHOI Y L , PARK S , JUNG H A , et al . Deep learning-based ensemble model using hematoxylin and eosin (H & E) whole slide images (WSIs) for the prediction of MET mutations in non-small cell lung cancer (NSCLC) [J ] . J Clin Oncol , 2023 , 41 ( 16 _suppl): e13578.

KONISHI T , GRYNKIEWICZ M , SAITO K , et al . Deep learning-based approach to predict multiple genetic mutations in colorectal and lung cancer tissues using hematoxylin and eosin-stained whole-slide images [J ] . J Clin Oncol , 2023 , 41 ( 16 _suppl): 1549.

SHA L D , OSINSKI B L , HO I Y , et al . Multi-field-of-view deep learning model predicts nonsmall cell lung cancer programmed death-ligand 1 status from whole-slide hematoxylin and eosin images [J ] . J Pathol Inform , 2019 , 10 : 24 . DOI: 10.4103/jpi.jpi_24_19 http://doi.org/10.4103/jpi.jpi_24_19

ZHENG Y N , PIZURICA M , CARRILLO-PEREZ F , et al . Digital profiling of cancer transcriptomes from histology images with grouped vision attention [J ] . bioRxiv , 2024 : 2023 .09.28.560068.

YU K H , WANG F R , BERRY G J , et al . Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks [J ] . J Am Med Inform Assoc , 2020 , 27 ( 5 ): 757 - 769 . DOI: 10.1093/jamia/ocz230 http://doi.org/10.1093/jamia/ocz230 https://academic.oup.com/jamia/article/27/5/757/5828209 https://academic.oup.com/jamia/article/27/5/757/5828209

SALTZ J , GUPTA R , HOU L , et al . Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images [J ] . Cell Rep , 2018 , 23 ( 1 ): 181 - 193 .e7. DOI: S2211-1247(18)30447-9 http://doi.org/S2211-1247(18)30447-9

GRAHAM S , VU Q D , RAZA S E A , et al . Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images [J ] . Med Image Anal , 2019 , 58 : 101563 . DOI: 10.1016/j.media.2019.101563 http://doi.org/10.1016/j.media.2019.101563 https://linkinghub.elsevier.com/retrieve/pii/S1361841519301045 https://linkinghub.elsevier.com/retrieve/pii/S1361841519301045

HOU L , GUPTA R , VAN ARNAM J S , et al . Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types [J ] . Sci Data , 2020 , 7 ( 1 ): 185.

CHEN P J , ROJAS F R , HU X , et al . Pathomic features reveal immune and molecular evolution from lung preneoplasia to invasive adenocarcinoma [J ] . Mod Pathol , 2023 , 36 ( 12 ): 100326.

WANG S D , WANG T , YANG L , et al . ConvPath: a software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network [J ] . EBioMedicine , 2019 , 50 : 103 - 110 . DOI: S2352-3964(19)30703-0 http://doi.org/S2352-3964(19)30703-0

WANG S D , RONG R C , YANG D M , et al . Computational staining of pathology images to study the tumor microenvironment in lung cancer [J ] . Cancer Res , 2020 , 80 ( 10 ): 2056 - 2066 . DOI: 10.1158/0008-5472.CAN-19-1629 http://doi.org/10.1158/0008-5472.CAN-19-1629

DIAO J A , WANG J K , CHUI W F , et al . Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes [J ] . Nat Commun , 2021 , 12 ( 1 ): 1613.

WANG H Y , XING F Y , SU H , et al . Novel image markers for non-small cell lung cancer classification and survival prediction [J ] . BMC Bioinformatics , 2014 , 15 ( 1 ): 310.

YU K H , ZHANG C , BERRY G J , et al . Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features [J ] . Nat Commun , 2016 , 7 : 12474 . DOI: 10.1038/ncomms12474 http://doi.org/10.1038/ncomms12474

LUO X , ZANG X , YANG L , et al . Comprehensive computational pathological image analysis predicts lung cancer prognosis [J ] . J Thorac Oncol , 2017 , 12 ( 3 ): 501 - 509 . DOI: S1556-0864(16)31236-9 http://doi.org/S1556-0864(16)31236-9

LUO X , YIN S , YANG L , et al . Development and validation of a pathology image analysis-based predictive model for lung adenocarcinoma prognosis-a multi-cohort study [J ] . Sci Rep , 2019 , 9 ( 1 ): 6886.

ALSUBAIE N M , SNEAD D , RAJPOOT N M . Tumour nuclear morphometrics predict survival in lung adenocarcinoma [J ] . IEEE Access , 2021 , 9 : 12322 - 12331 . DOI: 10.1109/Access.6287639 http://doi.org/10.1109/Access.6287639 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6287639 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6287639

SALI R , JIANG Y M , ATTARANZADEH A , et al . Morphological diversity of cancer cells predicts prognosis across tumor types [J ] . J Natl Cancer Inst , 2023 : djad243 .

DIAO S H , CHEN P J , SHOWKATIAN E , et al . Automated cellular-level dual global fusion of whole-slide imaging for lung adenocarcinoma prognosis [J ] . Cancers , 2023 , 15 ( 19 ): 4824.

LEVY-JURGENSON A , TEKPLI X , KRISTENSEN V N , et al . Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer [J ] . Sci Rep , 2020 , 10 ( 1 ): 18802.

PAN X P , LIN H , HAN C , et al . Computerized tumor-infiltrating lymphocytes density score predicts survival of patients with resectable lung adenocarcinoma [J ] . iScience , 2022 , 25 ( 12 ): 105605.

SHVETSOV N , GRØNNESBY M , PEDERSEN E , et al . A pragmatic machine learning approach to quantify tumor-infiltrating lymphocytes in whole slide images [J ] . Cancers , 2022 , 14 ( 12 ): 2974.

WANG X X , JANOWCZYK A , ZHOU Y , et al . Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images [J ] . Sci Rep , 2017 , 7 ( 1 ): 13543.

CORREDOR G , WANG X X , ZHOU Y , et al . Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non-small cell lung cancer [J ] . Clin Cancer Res , 2019 , 25 ( 5 ): 1526 - 1534 . DOI: 10.1158/1078-0432.CCR-18-2013 http://doi.org/10.1158/1078-0432.CCR-18-2013

TERADA K , YOSHIZAWA A , LIU X Q , et al . Deep learning for predicting effect of neoadjuvant therapies in non-small cell lung carcinomas with histologic images [J ] . Mod Pathol , 2023 , 36 ( 11 ): 100302.

PAN Y T , SHENG W , SHI L T , et al . Whole slide imaging-based deep learning to predict the treatment response of patients with non-small cell lung cancer [J ] . Quant Imaging Med Surg , 2023 , 13 ( 6 ): 3547 - 3555 . DOI: 10.21037/qims http://doi.org/10.21037/qims http://qims.amegroups.com http://qims.amegroups.com

HU J , CUI C L , YANG W X , et al . Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images [J ] . Transl Oncol , 2021 , 14 ( 1 ): 100921.

PARK S , OCK C Y , KIM H , et al . Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes as complementary biomarker for immune checkpoint inhibition in non-small-cell lung cancer [J ] . J Clin Oncol , 2022 , 40 ( 17 ): 1916 - 1928 . DOI: 10.1200/JCO.21.02010 http://doi.org/10.1200/JCO.21.02010 https://ascopubs.org/doi/10.1200/JCO.21.02010 https://ascopubs.org/doi/10.1200/JCO.21.02010

EVANS H , SNEAD D . Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them ?[J ] . Histopathology , 2024 , 84 ( 2 ): 279 - 287 . DOI: 10.1111/his.v84.2 http://doi.org/10.1111/his.v84.2 https://onlinelibrary.wiley.com/toc/13652559/84/2 https://onlinelibrary.wiley.com/toc/13652559/84/2

CAMPANELLA G , HANNA M G , GENESLAW L , et al . Clinical-grade computational pathology using weakly supervised deep learning on whole slide images [J ] . Nat Med , 2019 , 25 ( 8 ): 1301 - 1309 . DOI: 10.1038/s41591-019-0508-1 http://doi.org/10.1038/s41591-019-0508-1

XU X , HOU R , ZHAO W , et al . A weak supervision-based framework for automatic lung cancer classification on whole slide image [J ] . Annu Int Conf IEEE Eng Med Biol Soc , 2020 , 2020 : 1372 - 1375 . DOI: 10.1109/EMBC44109.2020.9176620 http://doi.org/10.1109/EMBC44109.2020.9176620

WINKLER J K , FINK C , TOBERER F , et al . Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition [J ] . JAMA Dermatol , 2019 , 155 ( 10 ): 1135 - 1141 . DOI: 10.1001/jamadermatol.2019.1735 http://doi.org/10.1001/jamadermatol.2019.1735 https://jamanetwork.com/journals/jamadermatology/fullarticle/2740808 https://jamanetwork.com/journals/jamadermatology/fullarticle/2740808

MARON R C , HEKLER A , KRIEGHOFF-HENNING E , et al . Reducing the impact of confounding factors on skin cancer classification via image segmentation: technical model study [J ] . J Med Internet Res , 2021 , 23 ( 3 ): e21695.

PHAM H H N , FUTAKUCHI M , BYCHKOV A , et al . Detection of lung cancer lymph node metastases from whole-slide histopathologic images using a two-step deep learning approach [J ] . Am J Pathol , 2019 , 189 ( 12 ): 2428 - 2439 . DOI: S0002-9440(19)30718-7 http://doi.org/S0002-9440(19)30718-7

HAO J , KOSARAJU S C , TSAKU N Z , et al . PAGE-net: Interpretable and integrative deep learning for survival analysis using histopathological images and genomic data [J ] . Pac Symp Biocomput , 2020 , 25 : 355 - 366 .

ALVAREZ-JIMENEZ C , SANDINO A A , PRASANNA P , et al . Identifying cross-scale associations between radiomic and pathomic signatures of non-small cell lung cancer subtypes: Preliminary results [J ] . Cancers , 2020 , 12 ( 12 ): 3663.

WU S X , HONG G B , XU A B , et al . Artificial intelligence-based model for lymph node metastases detection on whole slide images in bladder cancer: a retrospective, multicentre, diagnostic study [J ] . Lancet Oncol , 2023 , 24 ( 4 ): 360 - 370 . DOI: 10.1016/S1470-2045(23)00061-X http://doi.org/10.1016/S1470-2045(23)00061-X

ALATAKI A , ZABAGLO L , TOVEY H , et al . A simple digital image analysis system for automated Ki-67 assessment in primary breast cancer [J ] . Histopathology , 2021 , 79 ( 2 ): 200 - 209 . DOI: 10.1111/his.v79.2 http://doi.org/10.1111/his.v79.2 https://onlinelibrary.wiley.com/toc/13652559/79/2 https://onlinelibrary.wiley.com/toc/13652559/79/2

Views

2373

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research progress and prospects of MRI in predicting lymph node status in breast cancer

Artificial intelligence-based assessment of surgical skills in laparoscopic right hemicolectomy for colon cancer

Artificial intelligence in gastric cancer diagnosis, treatment and prognostic prediction: current application and future perspective

Current status and challenges of artificial intelligence-enabled prediction of synergistic cancer drug combinations

Deep learning-based lymphocyte infiltration detection on pathological images

Related Author

Zihan ZHAI

Sheng CHEN

Pengyu WEI

Jiale GAO

Maotong JI

Yuanyuan TAN

Hongwei YAO

Dongge PENG

Related Institution

Department of Breast Surgery, Fudan University Shanghai Cancer Center, Department of Oncology, Shanghai Medical College, Fudan University

Department of General Surgery, Beijing Friendship Hospital, National Clinical Research Center for Digestive Diseases, State Key Lab of Digestive Health, Clinical Practice and Research Center for Colorectal Neoplasm, Capital Medical University, Laboratory for Clinical Medicine, Capital Medical University

School of Basic Medical Sciences, Capital Medical University

Graduate School of Xinjiang Medical university, Urumqi 830000, Xinjiang Uygur Autonomous Region

Department of Oncology, Xinjiang Military Region General Hospital of the Chinese People’s Liberation Army, Urumqi 830000, Xinjiang Uygur Autonomous Region

AI问答

⁰