Human activity recognition using new generation depth sensors are particularly important for application that require human activity recognition. In this paper, a deep learning based algorithm is developed human activity recognition using RGB-D video sequences. Based on the assumption that every human activity is composed of many smaller actions, a temporal structure is being learnt in order to improve the classification of human activities. Since our approach is an attempt to develop a deep learning structure to the problem, it can be considered as a deep structural arhitecture. A deep neural network is obtained manipulating the activitation functions which yield hidden variables at every hidden layer. Our approach outperforms the methods that are constructed upon engineered features, since it uses the skeleton coordinates extracted from depth images. Tested on a new dataset, it is observed that our appproach outputs better recognition rates compared to those of other state-of-art methods.