News websites sometimes use titles which are not directly related to news content to get more interest. In this study, news articles are classified as clickbait or non-clickbait by using machine learning methods. To classify news articles, some features are extracted from both title and body paragraphs. Some machine learning methods are applied into these features and their results are examined. According to experiment result, the best score is 0.85 which is taken with KNN-3 dataset in the dataset. Moreover, to open public usage of this classifier, a web service has been created with the model. With this web service, a client can classify any news article as clickbait or non-clickbait by providing title and body paragraphs of article.