A Discriminative Long Short Term Memory Network with Metric Learning Applied to Multispectral Time Series Classification

Bozo M., Aptoula E., Çataltepe Z.

JOURNAL OF IMAGING, vol.6, no.7, 2020 (ESCI) identifier identifier identifier

  • Publication Type: Article / Article
  • Volume: 6 Issue: 7
  • Publication Date: 2020
  • Doi Number: 10.3390/jimaging6070068
  • Journal Name: JOURNAL OF IMAGING
  • Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus, INSPEC, Directory of Open Access Journals
  • Istanbul Technical University Affiliated: Yes


In this article, we propose an end-to-end deep network for the classification of multi-spectral time series and apply them to crop type mapping. Long short-term memory networks (LSTMs) are well established in this regard, thanks to their capacity to capture both long and short term temporal dependencies. Nevertheless, dealing with high intra-class variance and inter-class similarity still remain significant challenges. To address these issues, we propose a straightforward approach where LSTMs are combined with metric learning. The proposed architecture accommodates three distinct branches with shared weights, each containing a LSTM module, that are merged through a triplet loss. It thus not only minimizes classification error, but enforces the sub-networks to produce more discriminative deep features. It is validated viaBreizhcrops, a very recently introduced and challenging time series dataset for crop type mapping.