SkinDxFusion: An Integrated VGG19 and Vision Transformer Architecture for Dermatological Cancer Detection and GenAI-Based Reporting

Abstract

Skin cancer is one of the most common and rapidly increasing forms of cancer worldwide, making early and accurate diagnosis essential for improving patient outcomes. This paper presents SkinDxFusion, an intelligent dermatological analysis system that integrates a hybrid VGG19 and Vision Transformer (ViT) architecture with YOLOv11-based lesion detection and automated GenAI-driven medical report generation. The proposed hybrid model combines the local feature extraction capability of VGG19 with the global contextual learning ability of Vision Transformers, resulting in improved classification performance across multiple skin lesion categories. YOLOv11 is employed to accurately detect and localize lesion regions in both dermoscopic images and real-time video streams, enabling efficient clinical screening and practical deployment. To enhance usability, the system automatically generates structured dermatological reports containing predicted lesion type, confidence score, severity level, lesion characteristics, and clinical recommendations, reducing documentation effort and improving interpretability. The dataset consists of publicly available skin lesion images with appropriate preprocessing, normalization, augmentation, and class balancing techniques to improve model generalization. Experimental evaluation demonstrates that the proposed framework achieves superior accuracy, precision, and macro F1-score compared to standalone deep learning models while maintaining reliable detection performance. The developed web-based application provides a scalable, accessible, and decision-support solution for early skin cancer screening, assisting healthcare professionals in achieving faster, more consistent, and clinically meaningful dermatological diagnosis.