On-Screen Activity Classification In E-Learning: A Federated Learning Approach For Privacy Preservation

Abstract

With the rapid rise of online education, students often use the same devices for both learning and entertainment. This blurred boundary between productive and non-productive activities causes distraction and reduced learning efficiency. Existing solutions such as screen monitoring software or activity tracking tools either lack accuracy or severely compromise user privacy by transmitting raw screen data to centralized servers.
This project proposes a privacy-preserving approach that tracks on-screen activities, classifies them using deep learning, and ensures data privacy via federated learning. Screenshots are captured locally every minute and classified into categories such as programming, youtube_edu, or youtube_entertainment. The system provides users with a real-time, interactive dashboard to visualize how much time is spent on each activity type. Federated learning ensures that the deep learning model is trained and updated across devices without ever sharing raw data, thus offering both accuracy and privacy.