In this paper, we propose a CNN model based spermatozoa motility analysis, which is an important part of complete semen analysis. Sperm motility analysis is a good example of a multiple object tracking and video surveillance problem when viewed from engineering viewpoint. Our proposed system takes the video and images from a CCD camera, applies the front edge preprocessing tasks that uses uses CNN algorithms for spatial enhancement and preparation of image frames, combined with an appropriately designed cost function and a greedy assignment algorithm, that determines the objects-spermatozoa, traces their trajectories and classifies the obtained information for the use of biologists. The system composed of a digital CCD camera connected to the evaluation system. Here we showed the results by a simulation software running under a PC system. For the determination of sperm cells and and tracking the trajectories, we utilized the heuristic rules deduced from the dynamics of spermatozoa and investigation of the video obtained from real samples.