Early Detection and Classification of Lung Cancer using Segment Anything Model 2 and Dense Net

Hafza Eman; Syed M. Adnan; Wakeel Ahmad; Ishtiaque Mahmood

Authors

Hafza Eman University of Engineering and Technology, Taxila, Pakistan.
Syed M. Adnan University of Engineering and Technology, Taxila, Pakistan.
Wakeel Ahmad University of Engineering and Technology, Taxila, Pakistan.
Ishtiaque Mahmood Oman College of Management and Technology, Oman.

Keywords:

SAM2, Transfer Learning, Vision Transformer Model, Bounding Box Prompts, Computed Tomography (CT) scans

Abstract

Lung cancer is one of the most perilous diseases worldwide with high incidence and low survival rates due to late diagnosis. Accurate detection and diagnosis of lung nodules is important for early-stage detection. Machine learning and deep learning techniques have greatly improved the precision of lung nodule segmentation and classification in Computed Tomography (CT) images. The study presents a novel approach to segmenting and classifying nodules by combining foundational models with deep learning architectures. We have used the Segment Anything Model (SAM2) to segment lung nodules and Dense Net to classify them as benign and malignant. SAM2 has been tested on the datasets using different prompts to achieve better results. Foundational Models and Deep Learning architecture’s integration significantly improved Computer-Aided Detection (CADe) and Computer-Aided Diagnosis (CADx) in medical images. Experimental results proved the effectiveness of the proposed model for early-stage detection and classification of lung nodules from CT scans. SAM2 model achieves a Dice Similarity Coefficient (DSC) of 97.87% and an Intersection over Union (IoU) of 95.82% for segmentation, and the Dense Net model's classification accuracy is 97.34%. The experimental results demonstrate the performance of our model compared to existing techniques.

References

B. F. Ferlay J, E.M., Lam F, Laversanne M, Colombet M, Mery L, Piñeros M, Znaor A, Soerjomataram I, “Global Cancer Observatory: Cancer Today,” Int. Agency Res. Cancer, 2024, [Online]. Available: https://gco.iarc.fr/today/en

G. D. Rubin, “Lung nodule and cancer detection in computed tomography screening,” J. Thorac. Imaging, vol. 30, no. 2, pp. 130–138, Mar. 2015, doi: 10.1097/RTI.0000000000000140.

Ted Way, Heang-Ping Chan, Lubomir Hadjiiski , Berkman Sahiner, Aamer Chughtai, Thomas K. Song, Chad Poopat, Jadranka Stojanovska , Luba Frank, Anil Attili, Naama Bogot, and E. A. K. Philip N. Cascade, “Computer-Aided Diagnosis of Lung Nodules on CT Scans:: ROC Study of Its Effect on Radiologists’ Performance,” Acad. Radiol., vol. 17, no. 3, pp. 323–332, 2010, doi: https://doi.org/10.1016/j.acra.2009.10.016.

G. Litjens et al., “A survey on deep learning in medical image analysis,” Med. Image Anal., vol. 42, pp. 60–88, 2017, doi: https://doi.org/10.1016/j.media.2017.07.005.

Lulu Wang, “Deep Learning Techniques to Diagnose Lung Cancer,” Cancers (Basel), vol. 14, no. 22, p. 5569, 2022, doi: 10.3390/cancers14225569.

Md. Eshmam Rayed, S.M. Sajibul Islam, Sadia Islam Niha and M. F. M. Jamin Rahman Jim, Md Mohsin Kabir, “Deep learning for medical image segmentation: State-of-the-art advancements and challenges,” Informatics Med. Unlocked, vol. 47, p. 101504, 2024, doi: https://doi.org/10.1016/j.imu.2024.101504.

B. O. A. Ademola E. Ilesanmi, Taiwo O. Ilesanmi, “Reviewing 3D convolutional neural network approaches for medical image segmentation,” Heliyon, vol. 10, no. 6, p. e27398, 2024, [Online]. Available: https://www.cell.com/heliyon/fulltext/S2405-8440(24)03429-7?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2405844024034297%3Fshowall%3Dtrue

B. G. Satya P Singh, Lipo Wang, Sukrit Gupta, Haveesh Goli, Parasuraman Padmanabhan, “3D Deep Learning on Medical Images: A Review,” Sensors (Basel), vol. 20, no. 18, p. 5097, 2020, doi: 10.3390/s20185097.

Y.-G. S. Muhammad Usman, “DEHA-Net: A Dual-Encoder-Based Hard Attention Network with an Adaptive ROI Mechanism for Lung Nodule Segmentation,” Sensors, vol. 23, no. 4, p. 1989, 2023, doi: https://doi.org/10.3390/s23041989.

Z. Wang, J. Men, and F. Zhang, “Improved V-Net lung nodule segmentation method based on selective kernel,” Signal, Image Video Process., vol. 17, no. 5, pp. 1763–1774, Jul. 2023, doi: 10.1007/S11760-022-02387-W/METRICS.

F. L. J. Hou, C. Yan, R. Li, Q. Huang, X. Fan, “Lung Nodule Segmentation Algorithm With SMR-UNet,” IEEE Access, vol. 11, pp. 34319–3433, 2023, doi: 10.1109/ACCESS.2023.3264789.

Y.-T. W. Ting-Wei Wang, Chih-Keng Wang, Jia-Sheng Hong, Heng-Sheng Chao, Yuh-Min Chen, “Deep Learning in Thoracic Oncology: Meta-Analytical Insights into Lung Nodule Early-Detection Technologies,” Cancers (Basel)., vol. 17, no. 4, p. 621, 2025, doi: https://doi.org/10.3390/cancers17040621.

J. Z. Xiaoguang Tu, Zhi He, Yi Huang, Zhi-Hao Zhang, Ming Yang, “An overview of large AI models and their applications,” Vis. Intell., vol. 2, no. 34, 2024, doi: https://doi.org/10.1007/s44267-024-00065-8.

R. G. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, “Segment Anything,” Proc. IEEE/CVF Int. Conf. Comput. Vis., 2023, doi: https://doi.org/10.48550/arXiv.2304.02643.

Y. J. Junde Wu, Wei Ji, Yuanpei Liu, Huazhu Fu, Min Xu, Yanwu Xu, “Medical sam adapter: Adapting segment anything model for medical image segmentation,” arXiv Prepr. arXiv2304.12620, vol. 1, 2023, doi: https://doi.org/10.48550/arXiv.2304.12620.

C. F. Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, “Sam 2: Segment anything in images and videos,” arXiv Prepr. arXiv2408.00714, 2024, doi: https://doi.org/10.48550/arXiv.2408.00714.

M. O. Marjolein A. Heuvelmans, Peter M.A. van Ooijen, Sarim Ather, Carlos Francisco Silva, Daiwei Han, Claus Peter Heussel, William Hickes, Hans-Ulrich Kauczor, Petr Novotny, Heiko Peschl, Mieneke Rook, Roman Rubtsov, Oyunbileg von Stackelberg, Maria T. Tsakok, “Lung cancer prediction by Deep Learning to identify benign lung nodules,” Lung Cancer, vol. 154, pp. 1–4, 2021, doi: https://doi.org/10.1016/j.lungcan.2021.01.027.

H. Huang, R. Wu, Y. Li, and C. Peng, “Self-Supervised Transfer Learning Based on Domain Adaptation for Benign-Malignant Lung Nodule Classification on Thoracic CT,” IEEE J. Biomed. Heal. Informatics, vol. 26, no. 8, pp. 3860–3871, Aug. 2022, doi: 10.1109/JBHI.2022.3171851.

K. S. Mundher Al-Shabi and M. Tan, “ProCAN: Progressive growing channel attentive non-local network for lung nodule classification,” Pattern Recognit., vol. 122, p. 108309, 2022, doi: https://doi.org/10.1016/j.patcog.2021.108309.

S. A. Mahmood and H. A. Ahmed, “An improved CNN-based architecture for automatic lung nodule classification,” Med. Biol. Eng. Comput., vol. 60, no. 7, pp. 1977–1986, Jul. 2022, doi: 10.1007/S11517-022-02578-0/METRICS.

S. Dodia, A. Basava, and M. Padukudru Anand, “A novel receptive field-regularized V-net and nodule classification network for lung nodule detection,” Int. J. Imaging Syst. Technol., vol. 32, no. 1, pp. 88–101, Jan. 2022, doi: 10.1002/IMA.22636.

A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ICLR 2021 - 9th Int. Conf. Learn. Represent., Oct. 2020, Accessed: May 16, 2024. [Online]. Available: https://arxiv.org/abs/2010.11929v2

Arnaud Arindra Adiyoso Setio , Alberto Traverso , Thomas de Bel, Moira S.N. Berens, Cas van den Bogaard, Cerello Piergiorgio, Hao Chen and R. van der G. Qi Dou, Maria Evelina Fantacci, Bram Geurts, “Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge,” Med. Image Anal., vol. 42, pp. 1–13, 2017, doi: https://doi.org/10.1016/j.media.2017.06.015.

S. Armato et al., “The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Public Database of CT Scans for Lung Nodule Analysis,” Med. Phys., vol. 37, no. 6Part6, pp. 3416–3417, Jun. 2010, doi: 10.1118/1.3469350.

Haichao Cao, Hong Liu, C.-C. H. Enmin Song, X. X. Guangzhi Ma, Renchao Jin, and J. Lu, “Dual-branch residual network for lung nodule segmentation,” Appl. Soft Comput., vol. 86, p. 105934, 2020, doi: https://doi.org/10.1016/j.asoc.2019.105934.

S. Dodia, B. Annappa, and M. A. Padukudru, “A Novel Artificial Intelligence-Based Lung Nodule Segmentation and Classification System on CT Scans,” Commun. Comput. Inf. Sci., vol. 1568 CCIS, pp. 552–564, 2022, doi: 10.1007/978-3-031-11349-9_48.

B. K. Asha V, “Advanced Lung Nodule Segmentation and Classification for Early Detection of Lung Cancer using SAM and Transfer Learning,” arXiv:2501.00586v1, 2024, doi: https://doi.org/10.48550/arXiv.2501.00586.

F. Shaukat, S. M. Anwar, A. Parida, V. K. Lam, M. G. Linguraru, and M. Shah, “Lung-CADex: Fully Automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) , vol. 15241 LNCS, pp. 73–82, 2025, doi: 10.1007/978-3-031-73284-3_8.

H. Y. Wenju Wang, Shuya Yin, Fang Ye, Yinan Chen , Lin Zhu, “3D global coordinate attention wide inverted ResNet network for pulmonary nodules classification,” BMC Pulm Med, vol. 24, no. 1, p. 465, 2024, doi: 10.1186/s12890-024-03272-7.

A. H. Alshamrani K, “Classification of Chest CT Lung Nodules Using Collaborative Deep Learning Model,” J. Multidiscip. Healthc., p. 1459—1472, 2024, doi: https://doi.org/10.2147/JMDH.S456167.