In this paper, we propose an Incremental Non-negative Matrix Factorization (INMF) method which can be effectively used for dynamic background modeling in surveillance applications. The proposed factorization method is derived from Non-negative Matrix Factorization (NW), and models the dynamic content of the video by controlling contribution of the subsequent observations to the existing model adaptively. Unlike the batch nature of NMF, INMF is an on-line content representation scheme which is capable of extracting moving foreground objects. Test results are reported in order to compare background modeling performances of INMF, NMF and Incremental Principal Components Analysis (IPCA). It is concluded that INMF outperforms both NMF and IPCA and its robustness to illumination changes makes it as a powerful representation tool in surveillance applications.