A Framework for Skin Cancer Classification using Ensemble Transfer Learning and Data Fusion

Shabib Aftab; Tuba Mubarik; Muhammad Anwaar Saeed

doi:10.33411/IJIST/1991

Authors

Shabib Aftab Department of Computer Science, Virtual University of Pakistan
Tuba Mubarik Department of Computer Science, Virtual University of Pakistan
Muhammad Anwaar Saeed Department of Computer Science, Virtual University of Pakistan

DOI:

https://doi.org/10.33411/IJIST/1991

Keywords:

Skin Cancer Classification, Deep Learning, Ensemble Learning, EfficientNetB3, Xception

Abstract

Skin cancer is a serious and growing health problem worldwide. Its cases keep rising every year. Early diagnosis plays a key role in improving survival. Manual diagnosis through visual checks and dermoscopy depends on the doctor's experience. It can also give inconsistent results. Biopsy is the most reliable method. But it is invasive and time-consuming. These limits have pushed researchers toward deep learning-based systems. Most past studies use only one model. This lowers accuracy when skin lesions vary in colour, texture, and shape. To fix this, this study proposes an ensemble model combining EfficientNetB3 and Xception. The ISIC 2016 and ISIC 2017 datasets were combined for more diverse training data. Class imbalance was fixed using SMOTE. Both models were trained using transfer learning with ImageNet weights. They were trained under 3-fold cross-validation on a fixed test set. Mixup augmentation was also used during training. Predictions from both models were combined using equal-weighted averaging. The model was evaluated using accuracy, precision, recall, ROC-AUC, and confusion matrices. The ensemble reached a test accuracy of 95.22%, outperforming several existing methods. The ensemble also reached a sensitivity of 92.57%, a specificity of 97.88%, and a ROC-AUC of 0.9878. It improved over the standalone Xception model, which reached 95.01% accuracy, by 0.21 percentage points. It also improved over the standalone EfficientNetB3 model, which reached 94.90% accuracy, by 0.32 percentage points. These results confirm that this ensemble approach, with SMOTE-based balancing, offers a reliable solution for automated skin cancer classification.

References

B. F. . ung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, “Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries,” CA Cancer J Clin, vol. 71, no. 3, pp. 209–249, 2021, doi: 10.3322/caac.21660.

S. J. . Fried L, Tan A, Bajaj S, Liebman TN, Polsky D, “Technological advances for the detection of melanoma: Advances in diagnostic techniques,” J Am Acad Dermatol, vol. 83, no. 4, pp. 983–992, doi: 10.1016/j.jaad.2020.03.121.

M. Janda et al., “Early detection of melanoma: a consensus report from the Australian Skin and Skin Cancer Research Centre Melanoma Screening Summit,” Aust. N. Z. J. Public Health, vol. 44, no. 2, pp. 111–115, Apr. 2020, doi: 10.1111/1753-6405.12972.

H. S. . Goyal M, Knackstedt T, Yan S, “Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities,” Comput Biol Med, doi: 10.1016/j.compbiomed.2020.104065.

D. K. . Davis S, Piggott C, Lyon C, “Effectiveness of dermoscopy in skin cancer diagnosis,” Can Fam Physician, vol. 66, no. 10, pp. 739–740, [Online]. Available: https://pmc.ncbi.nlm.nih.gov/articles/PMC7571636/

K. H. . Sinz C, Tschandl P, Rosendahl C, Akay BN, Argenziano G, Blum A, Braun RP, Cabo H, Gourhant JY, Kreusch J, Lallas A, Lapins J, Marghoob AA, Menzies SW, Paoli J, Rabinovitz HS, Rinner C, Scope A, Soyer HP, Thomas L, Zalaudek I, “Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin.,” J Am Acad Dermatol, vol. 77, no. 6, pp. 1100–1109, doi: 10.1016/j.jaad.2017.07.022.

J. M. Jung, J. Y. Cho, W. J. Lee, S. E. Chang, M. W. Lee, and C. H. Won, “Emerging Minimally Invasive Technologies for the Detection of Skin Cancer,” J. Pers. Med., vol. 11, no. 10, Oct. 2021, doi: 10.3390/JPM11100951.

T. J. Brinker et al., “A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task,” Eur. J. Cancer, vol. 111, pp. 148–154, Apr. 2019, doi: 10.1016/j.ejca.2019.02.005.

H. A. Haenssle et al., “Man against Machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists,” Ann. Oncol., vol. 29, no. 8, pp. 1836–1842, Aug. 2018, doi: 10.1093/annonc/mdy166.

M. A. Kassem, K. M. Hosny and M. M. Fouad ., “Skin Lesions Classification Into Eight Classes for ISIC 2019 Using Deep Convolutional Neural Network and Transfer Learning,” IEEE Access, vol. 8, pp. 114822–114832, 2020, doi: 10.1109/ACCESS.2020.3003890.

N. Tajbakhsh et al., “Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1299–1312, May 2016, doi: 10.1109/TMI.2016.2535302.

E. I. . Mahbod A, Schaefer G, Wang C, Dorffner G, Ecker R, “Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification,” Comput Methods Programs Biomed, 2020, doi: 10.1016/j.cmpb.2020.105475.

A. F. 5 Munir Ahmad 1 , Majed Alfayad 2 , Shabib Aftab 1, 3 , Muhammad Adnan Khan 4, * and Bilal Shoaib 6 , Mohammad Sh. Daoud 7 and Nouh Sabri Elmitwally., “Data and Machine Learning Fusion Architecture for Cardiovascular Disease Prediction,” Mater. Contin., vol. 69, no. 2, doi: http://dx.doi.org/10.32604/cmc.2021.019013.

Samia Akhtar,Shabib Aftab., “A Framework for Diabetic Retinopathy Detection using Transfer Learning and Data Fusion,” Res. gate, vol. 16, no. 6, pp. 61–73, doi: 10.5815/ijitcs.2024.06.05.

Samia Akhtar,Shabib Aftab., “Diabetic Retinopathy Severity Classification Using Data Fusion and Ensemble Transfer Learning,” J. Softw. Eng. Appl., vol. 18, no. 1, pp. 1–23, doi: 10.4236/jsea.2025.181001.

Aftab, S., Abbas, S., Ghazal, T. M., Ahmad, M., Hamadi, H. A., Yeun, C. Y., & Khan, M. A ., “A Cloud-Based Software Defect Prediction System Using Data and Decision-Level Machine Learning Fusion,” Mathematics, vol. 11, no. 3, p. 632, 2023, doi: https://doi.org/10.3390/math11030632.

C. R. & H. K. Philipp Tschandl, “The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Sci. Data Vol., vol. 5, no. 180161, 2018, doi: https://doi.org/10.1038/sdata.2018.161.

A. G. C. Pacheco et al., “PAD-UFES-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones,” Data Br., vol. 32, p. 106221, Oct. 2020, doi: 10.1016/J.DIB.2020.106221.

David Gutman, Noel C. F. Codella, Emre Celebi, Brian Helba, Michael Marchetti, Nabin Mishra, Allan Halpern., “Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC),” arXiv:1605.01397, doi: https://doi.org/10.48550/arXiv.1605.01397.

N. C. F. Codella et al., “Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC),” Proc. - Int. Symp. Biomed. Imaging, vol. 2018-April, pp. 168–172, Oct. 2017, doi: 10.1109/ISBI.2018.8363547.

L. Yu, H. Chen, Q. Dou, J. Qin and P. -A. Heng., “Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks,” IEEE Trans. Med. Imaging, vol. 36, no. 4, pp. 994–1004, 2017, doi: 10.1109/TMI.2016.2642839.

T. G. Dietterich, “Ensemble methods in machine learning,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 1857 LNCS, pp. 1–15, 2000, doi: 10.1007/3-540-45014-9_1/SAVE-RESEARCH.

Q. V. Le Mingxing Tan, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” arXiv:1905.11946, 2019, doi: https://doi.org/10.48550/arXiv.1905.11946.

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-January, pp. 1800–1807, Nov. 2017, doi: 10.1109/CVPR.2017.195.

N. V. Chawla, K. W. Bowyer, “SMOTE: Synthetic Minority Over-sampling Technique,” arXiv:1106.1813, 2011, [Online]. Available: https://arxiv.org/abs/1106.1813

C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” J. Big Data, vol. 6, no. 1, pp. 1–48, Dec. 2019, doi: 10.1186/S40537-019-0197-0/FIGURES/33.

Halder, A., Dalal, A., Gharami, S. et al., “A fuzzy rank-based deep ensemble methodology for multi-class skin cancer classification,” Sci Rep, vol. 15, no. 6268, 2025, doi: https://doi.org/10.1038/s41598-025-90423-3.

N. L. B. & N. A. H. Mayar Ashraf Saeed, Yasmine M. Afify and Scientific Reports volume ., “Multimodal deep learning ensemble framework for skin cancer detection,” Sci Rep, vol. 15, no. 45660, doi: https://doi.org/10.1038/s41598-025-30534-z.

Dorathi Jayaseeli, J.D., Briskilal, J., Fancy, C. et al., “An intelligent framework for skin cancer detection and classification using fusion of Squeeze-Excitation-DenseNet with metaheuristic-driven ensemble deep learning models,” Sci. Rep., vol. 15, no. 7425, 2025, doi: https://doi.org/10.1038/s41598-025-92293-1.

Priya Natha, Sivarama Prasad Tera, Ravikumar Chinthaginjala, Safia Obaidur Rab, C. Venkata Narasimhulu & Tae Hoon Kim., “Boosting skin cancer diagnosis accuracy with an ensemble approach,” Sci. Rep., doi: https://www.nature.com/articles/s41598-024-84864-5.

A. Asaduzzaman, C. C. Thompson, F. N. Sibai, and M. J. Uddin, “Application of ensemble learning models in computer-aided diagnosis of skin diseases,” Neural Comput. Appl. 2025 3721, vol. 37, no. 21, pp. 16735–16751, Jun. 2025, doi: 10.1007/S00521-025-11336-W.

K. Matsunaga, A. Hamada, A. Minagawa, and H. Koga, “Image Classification of Melanoma, Nevus and Seborrheic Keratosis by Deep Neural Network Ensemble,” Mar. 2017, Accessed: Aug. 15, 2026. [Online]. Available: https://arxiv.org/pdf/1703.03108

Yang, G.; Luo, S.; Greer, P., “Boosting Skin Cancer Classification: A Multi-Scale Attention and Ensemble Approach with Vision Transformers,” Sensors, vol. 25, no. 8, 2025, doi: https://doi.org/10.3390/s25082479.

Vijaya, J., Roy, A., Gyanchandani, B. et al., “A novel particle swarm optimization-based ensemble efficientnet learning model for imbalanced image classification in medical diagnosis: case study of melanoma skin cancer prediction,” Netw Model Anal Heal. Inf. Bioinforma, vol. 14, no. 119, 2025, doi: https://doi.org/10.1007/s13721-025-00619-w.

Khan, Irfan Ullah, Aslam, Nida, Anwar, Talha, Aljameel, Sumayh S., Ullah, Mohib, Khan, Rafiullah, Rehman, Abdul, Akhtar, Nadeem., “Remote diagnosis and triaging model for skin cancer using EfficientNet and extreme gradient boosting,” Complexity, p. 13, 2021, doi: https://doi.org/10.1155/2021/5591614.

Burhanettin Ozdemir & Ishak Pacal ., “A robust deep learning framework for multiclass skin cancer classification,” Sci. Reports Vol., vol. 15, 2025, doi: https://doi.org/10.1038/s41598-025-89230-7.

A. I. Aboulmira A, Hrimech H, Lachgar M, Hanine M, Garcia CO, Mezquita GM, “Hybrid model with wavelet decomposition and EfficientNet for accurate skin cancer classification,” J Cancer, vol. 16, no. 2, pp. 506–520, 2016, doi: 10.7150/jca.101574.

Krohling, A. G. C. Pacheco and R. A., “An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification,” IEEE J. Biomed. Heal. Informatics, vol. 25, no. 9, pp. 3554–3563, 2021, doi: 10.1109/JBHI.2021.3062002.

C. J. and J. Y. (2022) A. study on skin tumor classification based on dense convolutional networks with fused metadata. Yin W, Huang J, “A study on skin tumor classification based on dense convolutional networks with fused metadata,” Front. Oncol., 2022, doi: https://doi.org/10.3389/fonc.2022.989894.

D. L. Ha, S. Jeong, D. Woo, J. Cho, and W. J. Lee, “Integrating Anatomical Site Information Into Vision Transformer Models for Skin Cancer Classification,” Ann. Dermatol., vol. 37, no. 5, pp. 314–316, Oct. 2025, doi: 10.5021/AD.24.181.

Leandro M. de Lima, Renato A. Krohling., “Exploring Advances in Transformers and CNN for Skin Lesion Diagnosis on Small Datasets,” IEEE Access, 2022, doi: https://doi.org/10.1007/978-3-031-21689-3_21.

M. L. Battle, A. Atapour-Abarghouei and A. S. McGough., “"Siamese Neural Networks for Skin Cancer Classification and New Class Detection using Clinical and Dermoscopic Image Datasets,” EEE Int. Conf. Big Data, pp. 4346–4355, 2022, doi: 10.1109/BigData55660.2022.10020820.

M. Azeem, K. Kiani, T. Mansouri, and N. Topping, “SkinLesNet: Classification of Skin Lesions and Detection of Melanoma Cancer Using a Novel Multi-Layer Deep Convolutional Neural Network,” Cancers 2024, Vol. 16, Page 108, vol. 16, no. 1, p. 108, Dec. 2023, doi: 10.3390/CANCERS16010108.

Bechelli S, Delhommelle J., “Machine learning and deep learning algorithms for skin cancer classification from dermoscopic images,” Bioengineering, vol. 9, no. 3, p. 97, 2022, doi: 10.3390/bioengineering9030097.

G. M. S. Himel, M. M. Islam, K. A. Al-Aff, S. I. Karim, and M. K. U. Sikder, “Skin Cancer Segmentation and Classification Using Vision Transformer for Automatic Analysis in Dermatoscopy-Based Noninvasive Digital System,” Int. J. Biomed. Imaging, vol. 2024, 2024, doi: 10.1155/2024/3022192.

H. T. Halawani, E. M. Senan, Y. Asiri, I. Abunadi, A. M. Mashraqi, and E. A. Alshari, “Enhanced early skin cancer detection through fusion of vision transformer and CNN features using hybrid attention of EViT-Dens169,” Sci. Reports 2025 151, vol. 15, no. 1, pp. 34776-, Oct. 2025, doi: 10.1038/s41598-025-18570-1.

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2017-October, pp. 618–626, Dec. 2017, doi: 10.1109/ICCV.2017.74.

A. M. Fayyaz, S. J. Abdulkadir, N. Talpur, S. M. Al-Selwi, S. U. Hassan, and E. H. Sumiea, “Grad-CAM (Gradient-weighted Class Activation Mapping): A systematic literature review,” Comput. Biol. Med., vol. 198, p. 111200, Nov. 2025, doi: 10.1016/J.COMPBIOMED.2025.111200.

S. Yan et al., “A multimodal vision foundation model for clinical dermatology,” Nat. Med. 2025 318, vol. 31, no. 8, pp. 2691–2702, Jun. 2025, doi: 10.1038/s41591-025-03747-y.

H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond Empirical Risk Minimization,” 6th Int. Conf. Learn. Represent. ICLR 2018 - Conf. Track Proc., Oct. 2017, Accessed: Aug. 15, 2026. [Online]. Available: https://arxiv.org/pdf/1710.09412