The tracking consensus problem for stochastic Markovian multi-agent systems (MASs) with mismatched uncertainty is investigated in this study. A restrictive assumption for stochastic systems in existing results is removed in this study. Under the directed interacted topology, the sliding mode control method is utilised to achieve leader-following consensus for general linear MASs in the sense of mean square. Integral sliding mode surfaces are employed to tackle the mismatched uncertainties. Then, with the transition rates of Markov process being completely known or partly unknown, two kinds of distributed sliding mode protocols are given, respectively. Based on the sliding mode control theory, algebraic graph theory and stochastic differential equation theory, mean square stability of sliding mode dynamics are ensured by LMIs and reachability of sliding mode surfaces almost surely from the initial time are obtained. Finally, simulations are given to illustrate the effectiveness of the results.