There are many application fields to predict the emotion in multimedia content automatically such as offering personalized media options to the users, indexing media. With the developments in deep learning, the issues have become more popular. In this study, it is aimed to predict the emotion elicited from movies by using convolutional neural network approaches from the visual based features. In addition, the effect of movie genre in emotion prediction in terms of valence and arousal score is analyzed separately by using the LIRIS-ACCEDE movie dataset. As the main contribution, the dataset is deeply analyzed according to film genres. After that, it is categorized into the training groups in a way that the same genre movies are proportionally distributed, and well-known CNN networks are utilized for regression training.