Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer

doi:10.19401/j.cnki.1007-3639.2024.03.009

Abstract

Abstract:

Pathology is the gold standard for diagnosis of neoplastic diseases. Whole slide imaging turns traditional slides into digital images, and artificial intelligence has shown great potential in pathological image analysis, especially deep learning models. The application of artificial intelligence in whole slide imaging of lung cancer involves many aspects such as histopathological classification, tumor microenvironment analysis, efficacy and survival prediction, etc., which is expected to assist clinical decision-making of accurate treatment. Limitations in this field include the lack of precisely annotated data and slide quality varying among institutions. Here we summarized recent research in lung cancer pathology image analysis leveraging artificial intelligence and proposed several future directions.

Key words: Whole slide imaging, Lung cancer, Artificial intelligence, Convolutional neural network

CLC Number:

R734.2

JIANG Mengqi, HAN Yuchen, FU Xiaolong. Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer[J]. China Oncology, 2024, 34(3): 306-315.

References 62

[1]	JAHN S W, PLASS M, MOINFAR F. Digital pathology: Advantages, limitations and emerging perspectives[J]. J Clin Med, 2020, 9(11): 3697.
[2]	MUKHOPADHYAY S, FELDMAN M D, ABELS E, et al. Whole slide imaging versus microscopy for primary diagnosis in surgical pathology: a multicenter blinded randomized noninferiority study of 1992 cases (pivotal study)[J]. Am J Surg Pathol, 2018, 42(1): 39-52. doi: 10.1097/PAS.0000000000000948 pmid: 28961557
[3]	DA SILVA L M, PEREIRA E M, SALLES P G, et al. Independent real-world application of a clinical-grade automated prostate cancer detection system[J]. J Pathol, 2021, 254(2): 147-158. doi: 10.1002/path.v254.2
[4]	COUDRAY N, OCAMPO P S, SAKELLAROPOULOS T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning[J]. Nat Med, 2018, 24(10): 1559-1567. doi: 10.1038/s41591-018-0177-5 pmid: 30224757
[5]	ZHAO L, XU X W, HOU R P, et al. Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning[J]. Phys Med Biol, 2021, 66(23).
[6]	KHADER F, KATHER J N, HAN T Y, et al. Cascaded cross-attention networks for data-efficient whole-slide image classification using transformers[M]//CAO X, XU X, REKIK I, et al. International Workshop on Machine Learning in Medical Imaging. Cham: Springer, 2024: 417-426.
[7]	YANG H, CHEN L L, CHENG Z Q, et al. Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study[J]. BMC Med, 2021, 19(1): 80.
[8]	TERAMOTO A, TSUKAMOTO T, KIRIYAMA Y, et al. Automated classification of lung cancer types from cytological images using deep convolutional neural networks[J]. Biomed Res Int, 2017, 2017: 4067832.
[9]	TSUKAMOTO T, TERAMOTO A, YAMADA A, et al. Comparison of fine-tuned deep convolutional neural networks for the automated classification of lung cancer cytology images with integration of additional classifiers[J]. Asian Pac J Cancer Prev, 2022, 23(4): 1315-1324. doi: 10.31557/APJCP.2022.23.4.1315
[10]	GUAN Q, WAN X C, LU H T, et al. Deep convolutional neural network Inception-v3 model for differential diagnosing of lymph node in cytological images: a pilot study[J]. Ann Transl Med, 2019, 7(14): 307.
[11]	YANG J W, SONG D H, AN H J, et al. Classification of subtypes including LCNEC in lung cancer biopsy slides using convolutional neural network from scratch[J]. Sci Rep, 2022, 12(1): 1830.
[12]	ILIÉ M, BENZAQUEN J, TOURNIAIRE P, et al. Deep learning facilitates distinguishing histologic subtypes of pulmonary neuroendocrine tumors on digital whole-slide images[J]. Cancers, 2022, 14(7): 1740.
[13]	GONZALEZ D, DIETZ R L, PANTANOWITZ L. Feasibility of a deep learning algorithm to distinguish large cell neuroendocrine from small cell lung carcinoma in cytology specimens[J]. Cytopathology, 2020, 31(5): 426-431. doi: 10.1111/cyt.12829 pmid: 32246504
[14]	TRAVIS W D, BRAMBILLA E, NOGUCHI M, et al. International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society international multidisciplinary classification of lung adenocarcinoma[J]. J Thorac Oncol, 2011, 6(2): 244-285. doi: 10.1097/JTO.0b013e318206a221 pmid: 21252716
[15]	MOREIRA A L, OCAMPO P S S, XIA Y H, et al. A grading system for invasive pulmonary adenocarcinoma: a proposal from the international association for the study of lung cancer pathology committee[J]. J Thorac Oncol, 2020, 15(10): 1599-1610. doi: S1556-0864(20)30468-8 pmid: 32562873
[16]	WEI J W, TAFE L J, LINNIK Y A, et al. Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks[J]. Sci Rep, 2019, 9(1): 3358.
[17]	ZHAO Y L, HE S, ZHAO D, et al. Deep learning-based diagnosis of histopathological patterns for invasive non-mucinous lung adenocarcinoma using semantic segmentation[J]. BMJ Open, 2023, 13(7): e069181.
[18]	ALSUBAIE N, RAZA S E A, SNEAD D, et al. Growth pattern fingerprinting for automatic analysis of lung adenocarcinoma overall survival[J]. IEEE Access, 2023, 11: 23335-23346. doi: 10.1109/ACCESS.2023.3251220
[19]	SADHWANI A, CHANG H W, BEHROOZ A, et al. Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images[J]. Sci Rep, 2021, 11(1): 16605.
[20]	KARASAKI T, MOORE D A, VEERIAH S, et al. Evolutionary characterization of lung adenocarcinoma morphology in TRACERx[J]. Nat Med, 2023, 29(4): 833-845. doi: 10.1038/s41591-023-02230-w pmid: 37045996
[21]	YU K H, BERRY G J, RUBIN D L, et al. Association of omics features with histopathology patterns in lung adenocarcinoma[J]. Cell Syst, 2017, 5(6): 620-627.e3.
[22]	MAYER C, OFEK E, FRIDRICH D E, et al. Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms[J]. Mod Pathol, 2022, 35(12): 1882-1887. doi: 10.1038/s41379-022-01141-4
[23]	TOMITA N, TAFE L J, SURIAWINATA A A, et al. Predicting oncogene mutations of lung cancer using deep learning and histopathologic features on whole-slide images[J]. Transl Oncol, 2022, 24: 101494. doi: 10.1016/j.tranon.2022.101494
[24]	TERADA Y, TAKAHASHI T, HAYAKAWA T, et al. Artificial intelligence-powered prediction of ALK gene rearrangement in patients with non-small-cell lung cancer[J]. JCO Clin Cancer Inform, 2022, 6: e2200070.
[25]	CHOI Y L, PARK S, JUNG H A, et al. Deep learning-based ensemble model using hematoxylin and eosin (H&E) whole slide images (WSIs) for the prediction of MET mutations in non-small cell lung cancer (NSCLC)[J]. J Clin Oncol, 2023, 41(16_suppl): e13578.
[26]	KONISHI T, GRYNKIEWICZ M, SAITO K, et al. Deep learning-based approach to predict multiple genetic mutations in colorectal and lung cancer tissues using hematoxylin and eosin-stained whole-slide images[J]. J Clin Oncol, 2023, 41(16_suppl): 1549.
[27]	SHA L D, OSINSKI B L, HO I Y, et al. Multi-field-of-view deep learning model predicts nonsmall cell lung cancer programmed death-ligand 1 status from whole-slide hematoxylin and eosin images[J]. J Pathol Inform, 2019, 10: 24. doi: 10.4103/jpi.jpi_24_19 pmid: 31523482
[28]	ZHENG Y N, PIZURICA M, CARRILLO-PEREZ F, et al. Digital profiling of cancer transcriptomes from histology images with grouped vision attention[J]. bioRxiv, 2024: 2023.09.28.560068.
[29]	YU K H, WANG F R, BERRY G J, et al. Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks[J]. J Am Med Inform Assoc, 2020, 27(5): 757-769. doi: 10.1093/jamia/ocz230
[30]	SALTZ J, GUPTA R, HOU L, et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images[J]. Cell Rep, 2018, 23(1): 181-193.e7. doi: S2211-1247(18)30447-9 pmid: 29617659
[31]	GRAHAM S, VU Q D, RAZA S E A, et al. Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images[J]. Med Image Anal, 2019, 58: 101563. doi: 10.1016/j.media.2019.101563
[32]	HOU L, GUPTA R, VAN ARNAM J S, et al. Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types[J]. Sci Data, 2020, 7(1): 185.
[33]	CHEN P J, ROJAS F R, HU X, et al. Pathomic features reveal immune and molecular evolution from lung preneoplasia to invasive adenocarcinoma[J]. Mod Pathol, 2023, 36(12): 100326.
[34]	WANG S D, WANG T, YANG L, et al. ConvPath: a software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network[J]. EBioMedicine, 2019, 50: 103-110. doi: S2352-3964(19)30703-0 pmid: 31767541
[35]	WANG S D, RONG R C, YANG D M, et al. Computational staining of pathology images to study the tumor microenvironment in lung cancer[J]. Cancer Res, 2020, 80(10): 2056-2066. doi: 10.1158/0008-5472.CAN-19-1629 pmid: 31915129
[36]	DIAO J A, WANG J K, CHUI W F, et al. Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes[J]. Nat Commun, 2021, 12(1): 1613.
[37]	WANG H Y, XING F Y, SU H, et al. Novel image markers for non-small cell lung cancer classification and survival prediction[J]. BMC Bioinformatics, 2014, 15(1): 310.
[38]	YU K H, ZHANG C, BERRY G J, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features[J]. Nat Commun, 2016, 7: 12474. doi: 10.1038/ncomms12474
[39]	LUO X, ZANG X, YANG L, et al. Comprehensive computational pathological image analysis predicts lung cancer prognosis[J]. J Thorac Oncol, 2017, 12(3): 501-509. doi: S1556-0864(16)31236-9 pmid: 27826035
[40]	LUO X, YIN S, YANG L, et al. Development and validation of a pathology image analysis-based predictive model for lung adenocarcinoma prognosis-a multi-cohort study[J]. Sci Rep, 2019, 9(1): 6886.
[41]	ALSUBAIE N M, SNEAD D, RAJPOOT N M. Tumour nuclear morphometrics predict survival in lung adenocarcinoma[J]. IEEE Access, 2021, 9: 12322-12331. doi: 10.1109/Access.6287639
[42]	SALI R, JIANG Y M, ATTARANZADEH A, et al. Morphological diversity of cancer cells predicts prognosis across tumor types[J]. J Natl Cancer Inst, 2023: djad243.
[43]	DIAO S H, CHEN P J, SHOWKATIAN E, et al. Automated cellular-level dual global fusion of whole-slide imaging for lung adenocarcinoma prognosis[J]. Cancers, 2023, 15(19): 4824.
[44]	LEVY-JURGENSON A, TEKPLI X, KRISTENSEN V N, et al. Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer[J]. Sci Rep, 2020, 10(1): 18802.
[45]	PAN X P, LIN H, HAN C, et al. Computerized tumor-infiltrating lymphocytes density score predicts survival of patients with resectable lung adenocarcinoma[J]. iScience, 2022, 25(12): 105605.
[46]	SHVETSOV N, GRØNNESBY M, PEDERSEN E, et al. A pragmatic machine learning approach to quantify tumor-infiltrating lymphocytes in whole slide images[J]. Cancers, 2022, 14(12): 2974.
[47]	WANG X X, JANOWCZYK A, ZHOU Y, et al. Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images[J]. Sci Rep, 2017, 7(1): 13543.
[48]	CORREDOR G, WANG X X, ZHOU Y, et al. Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non-small cell lung cancer[J]. Clin Cancer Res, 2019, 25(5): 1526-1534. doi: 10.1158/1078-0432.CCR-18-2013 pmid: 30201760
[49]	TERADA K, YOSHIZAWA A, LIU X Q, et al. Deep learning for predicting effect of neoadjuvant therapies in non-small cell lung carcinomas with histologic images[J]. Mod Pathol, 2023, 36(11): 100302.
[50]	PAN Y T, SHENG W, SHI L T, et al. Whole slide imaging-based deep learning to predict the treatment response of patients with non-small cell lung cancer[J]. Quant Imaging Med Surg, 2023, 13(6): 3547-3555. doi: 10.21037/qims
[51]	HU J, CUI C L, YANG W X, et al. Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images[J]. Transl Oncol, 2021, 14(1): 100921.
[52]	PARK S, OCK C Y, KIM H, et al. Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes as complementary biomarker for immune checkpoint inhibition in non-small-cell lung cancer[J]. J Clin Oncol, 2022, 40(17): 1916-1928. doi: 10.1200/JCO.21.02010
[53]	EVANS H, SNEAD D. Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them?[J]. Histopathology, 2024, 84(2): 279-287. doi: 10.1111/his.v84.2
[54]	CAMPANELLA G, HANNA M G, GENESLAW L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images[J]. Nat Med, 2019, 25(8): 1301-1309. doi: 10.1038/s41591-019-0508-1 pmid: 31308507
[55]	XU X, HOU R, ZHAO W, et al. A weak supervision-based framework for automatic lung cancer classification on whole slide image[J]. Annu Int Conf IEEE Eng Med Biol Soc, 2020, 2020: 1372-1375. doi: 10.1109/EMBC44109.2020.9176620 pmid: 33018244
[56]	WINKLER J K, FINK C, TOBERER F, et al. Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition[J]. JAMA Dermatol, 2019, 155(10): 1135-1141. doi: 10.1001/jamadermatol.2019.1735
[57]	MARON R C, HEKLER A, KRIEGHOFF-HENNING E, et al. Reducing the impact of confounding factors on skin cancer classification via image segmentation: technical model study[J]. J Med Internet Res, 2021, 23(3): e21695.
[58]	PHAM H H N, FUTAKUCHI M, BYCHKOV A, et al. Detection of lung cancer lymph node metastases from whole-slide histopathologic images using a two-step deep learning approach[J]. Am J Pathol, 2019, 189(12): 2428-2439. doi: S0002-9440(19)30718-7 pmid: 31541645
[59]	HAO J, KOSARAJU S C, TSAKU N Z, et al. PAGE-net: Interpretable and integrative deep learning for survival analysis using histopathological images and genomic data[J]. Pac Symp Biocomput, 2020, 25: 355-366. pmid: 31797610
[60]	ALVAREZ-JIMENEZ C, SANDINO A A, PRASANNA P, et al. Identifying cross-scale associations between radiomic and pathomic signatures of non-small cell lung cancer subtypes: Preliminary results[J]. Cancers, 2020, 12(12): 3663.
[61]	WU S X, HONG G B, XU A B, et al. Artificial intelligence-based model for lymph node metastases detection on whole slide images in bladder cancer: a retrospective, multicentre, diagnostic study[J]. Lancet Oncol, 2023, 24(4): 360-370. doi: 10.1016/S1470-2045(23)00061-X pmid: 36893772
[62]	ALATAKI A, ZABAGLO L, TOVEY H, et al. A simple digital image analysis system for automated Ki-67 assessment in primary breast cancer[J]. Histopathology, 2021, 79(2): 200-209. doi: 10.1111/his.v79.2