The effect of grain size and alloying elements on the yield strength in compression was investigated using pure magnesium and its diluted solid solution alloys (Mg–Zn and Mg–Y alloys). The yield strength in compression was closely related to the {10–12} twinning, and followed the Hall–Petch relation in all the alloys. The value of σ0 increased in the following order, Mg–Y>Mg–Zn>pure magnesium, indicating solid solution strengthening; however, the slope in the Hall–Petch relation was not influenced much by the alloying elements. The twinning formed frequently at the grain boundaries with a low misorientation angle, and even occurred in the parent grains with a low Schmid factor in all the alloys. These results indicated that one of the most influential parameters for the Hall–Petch slope was not the Schmid factor but the grain boundary characteristics. All the stress and strain curves in the compression showed a plateau region referring to the expansion/propagation of {10–12} twinning after the yielding behavior. The length of the plateau region increased with a decrease in the grain size and the Schmid factor of the basal slip in the parent grains.