Predicting Defects with Latent and Semantic Features from Commit Logs in an Industrial Setting

EKEN B., Atar R., Sertalp S., TOSUN A.

First International Workshop onSoftware Engineering Intelligence, 15 November 2019 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/asew.2019.00038
  • Keywords: Software defect prediction, matrix factorization, topic modeling, XGBoost, NONNEGATIVE MATRIX FACTORIZATION
  • Istanbul Technical University Affiliated: Yes

Abstract

Software defect prediction is still a challenging task in industrial settings. Noisy data and/or lack of data make it hard to build successful prediction models. In this study, we aim to build a change-level defect prediction model for a software project in an industrial setting. We combine various probabilistic models, namely matrix factorization and topic modeling, with the expectation of overcoming the noisy and limited nature of industrial settings by extracting hidden features from multiple resources. Commit level process metrics, latent features from commits, and semantic features from commit messages are combined to build the defect predictors with the use of Log Filtering and feature selection techniques, and two machine learning algorithms Naive Bayes and Extreme Gradient Boosting (XGBoost). Collecting data from various sources and applying data pre-processing techniques show a statistically significant improvement in terms of probability of detection by up to 24% when compared to a base model with process metrics only.