A GenAI-Driven Hybrid CNN–Vision Transformer Framework for Multi-Class Knee Osteoarthritis Classification from X-Ray Images

Abstract

Knee osteoarthritis is one of the most common degenerative musculoskeletal disorders, causing progressive cartilage deterioration, joint pain, stiffness, and reduced mobility. Accurate assessment of disease severity from knee X-ray images is essential for timely diagnosis and effective treatment planning, yet manual interpretation often depends on radiologist expertise and may lead to inconsistent grading. This research presents a hybrid deep learning framework that combines the feature extraction capability of VGG19 with the contextual learning strength of a Vision Transformer (ViT) for automated multi-class classification of knee osteoarthritis. The proposed framework classifies X-ray images into six categories: Healthy, Doubtful, Minimal, Moderate, Severe, and Invalid. A comprehensive preprocessing pipeline involving image resizing, normalization, contrast enhancement, and data augmentation improves image quality and model robustness. The fused CNN and transformer features enable effective learning of both local anatomical characteristics and global structural relationships, resulting in improved classification performance. The framework further integrates a Generative Artificial Intelligence module to automatically generate structured diagnostic reports based on predicted classes and confidence scores, enhancing clinical usability. Additionally, a blockchain-enabled storage mechanism ensures secure, tamper-resistant management of diagnostic records. A user-friendly web application allows real-time image upload, prediction, report generation, and history management. Experimental evaluation demonstrates that the proposed hybrid framework provides reliable, accurate, and scalable computer-aided diagnosis for supporting orthopedic clinical decision-making.