A Big Data Based Emotion Detection Framework for Social Media Using Apache Spark.

Abstract

YouTube is considered as one of the most widely used video-sharing platforms in the world. Users can express their reactions to videos through comments, which often convey emotions that can be automatically identified using computational techniques. Emotion detection on YouTube data presents a challenging task due to the heterogeneity, unstructured nature, and large scale of user generated contents. In this study, we develop an emotion detection framework implemented on Apache Spark, an open-source platform for distributed Big Data processing. The proposed system integrates Machine Learning algorithms with Natural Language Processing (NLP) techniques and leverages Spark’s MLlib library to classify emotions expressed in YouTube comments. To efficiently deal with the complexity and noise inherent in largescale multimedia data, several preprocessing and feature extraction steps are introduced. The K-Means clustering algorithm is used after data preparation for the corpus automatic annotation, the resulting labeled Dataset is labeled according to the Ekman emotional model with six basic emotions. The selected classifiers are trained using the resulting labeled Dataset. Experimental results demonstrate that the proposed approach improves both scalability and accuracy, making it suitable for leveraging the emotion detection in social Big Data environments.