This paper presents a dedicated approach to detect loop closures using visually salient patches. We introduce a novel, energy maximization based saliency detection technique which has been used for unsupervised landmark extraction. We explain how to learn the extracted landmarks on-the-fly and re-identify them. Furthermore, we describe the sparse location representation we use to recognize previously seen locations in order to perform reliable loop closure detection. The performance of our method has been analyzed both on an indoor and an outdoor dataset, and it has been shown that our approach achieves quite promising results on both datasets.