The existing tracking and recognition methods concentrate mainly on single-class targets; however, systems for traffic management or intelligent transport often require multi-class target tracking and recognition in real time. This study proposes an effective multi-class moving target recognition method that is based on Gaussian mixture part-based model, which accurately locates objects of interest and recognises their corresponding categories. The method is multi-threaded and combines soft clustering approach with multiple mixture part based models to provide stable multi-class target tracking and recognition in video sequences. The highlight of the method is its ability to recognise multi-class moving targets and to count their numbers in the video sequence captured by a stationary camera with fixed focal length. Another contribution of this study is that an extended part based model is developed for object recognition in real-world environments, which can improve the overall system performance, lower time costs, and better meet the actual demand of a video system. Experimental results show that the proposed method is viable in real-time multi-class moving target tracking and recognition.