COSMOS on Steroids: a Cheap Detector for Cheapfakes

Akgül T., Civelek T. E., Ugur D., Begen A. C.

12th ACM Multimedia Systems Conference (MMSys), İstanbul, Turkey, 28 September - 01 October 2021, pp.327-331 identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1145/3458305.3479968
  • City: İstanbul
  • Country: Turkey
  • Page Numbers: pp.327-331
  • Keywords: Cheapfakes, RNN, BERT, SBERT, IoU, differential sensing, fake
  • Istanbul Technical University Affiliated: Yes


The growing prevalence of visual disinformation has become an important problem to solve nowadays. Cheapfake is a new term used for the altered media generated by non-AI techniques. In their recent COSMOS work, the authors developed a self-supervised training strategy that detected whether different captions for a given image were out-of-context, meaning that even though pointing to the same object(s) in the image, the captions implied different meanings. In this paper, we propose four methods to improve the detection accuracy of COSMOS. These methods range from differential sensing and fake-or-fact checking that detect contradicting or fake captions to object-caption matching and threshold adjustment that modify the baseline algorithm for improved accuracy.