One of the main obstacles that prevents practical applications of antiferromagnets is the difficulty of manipulating the magnetic order parameter. Recently, following the theoretical prediction J. Železný et al., Phys. Rev. Lett. 113, 157201 (2014), the electrical switching of magnetic moments in an antiferromagnet was demonstrated P. Wadley et al., Science 351, 587 (2016). The switching is due to the so-called spin-orbit torque, which has been extensively studied in ferromagnets. In this phenomena a nonequilibrium spin-polarization exchange coupled to the ordered local moments is induced by current, hence exerting a torque on the order parameter. Here we give a general systematic analysis of the symmetry of the spin-orbit torque in locally and globally noncentrosymmetric crystals. We study when the symmetry allows for a nonzero torque, when is the torque effective, and its dependence on the applied current direction and orientation of magnetic moments. For comparison, we consider both antiferromagnetic and ferromagnetic orders. In two representative model crystals we perform microscopic calculations of the spin-orbit torque to illustrate its symmetry properties and to highlight conditions under which the spin-orbit torque can be efficient for manipulating antiferromagnetic moments.