A deep learning approach for automatic detection, segmentation and classification of breast lesions from thermal images

Civilibal S., ÇEVİK K. K., Bozkurt A.

Expert Systems with Applications, vol.212, 2023 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 212
  • Publication Date: 2023
  • Doi Number: 10.1016/j.eswa.2022.118774
  • Journal Name: Expert Systems with Applications
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Aerospace Database, Applied Science & Technology Source, Communication Abstracts, Computer & Applied Sciences, INSPEC, Metadex, Public Affairs Index, Civil Engineering Abstracts
  • Keywords: Thermal breast images, Detection, Classification, Segmentation, Transfer learning
  • Istanbul Technical University Affiliated: Yes


Purpose: This study investigates implementation of deep learning (DL) approaches to breast tumor recognition based on thermal images. We propose to utilize Mask R-CNN technique on images by first assigning bounding boxes and then creating a border for each tumor volume to differentiate it from adjacent tissues and structures. In this manner, thermal images can be handled by a single DL model to successfully perform detection, classification, and segmentation of normal and abnormal breast tissues. Methods: This study employs Mask R-CNN technique along with transfer learning models to accurately delineate breast volumes from adjacent tissues and structures. It is a novel study that uses a single DL model to carry out three steps of breast tumor diagnosis, namely detection, segmentation and classification of normal and abnormal tissues based on thermal images. Results: Two network architectures (ResNet-50 and ResNet-101) were trained for 60 epochs and were then evaluated based on their classification and segmentation performances. The testing process resulted in higher classification success for ResNet-50 backbone pretrained on COCO images (%97.1 accuracy). Detection and segmentation performances of this model were also higher with mAP of 0.921 and overlap score of 0.868. Conclusions: Our results indicate that the classification and segmentation performances of Mask R-CNN method on ResNet-50 architecture are better than the data reported in the literature for thermal breast image studies. It should be emphasized that our approach employs a single DL model that successfully performs detection, classification, and segmentation procedures for diagnosing normal and abnormal breast tissues.