AZ31 Mg alloy was processed by accumulative roll-bonding (ARB) and hot rolling (HR), respectively, followed by annealing. Layered bimodal structures characterized by an alternative distribution of fine-grained layers and coarse-grained layers were obtained in the ARB samples, while mixed bimodal structures were achieved in the HR samples. The ARB samples have superior combinations of high strength and good elongation compared to the HR samples, indicating a clear effect of layered bimodal structures on mechanical properties of the alloy. The strength of the ARB samples is related to the grain size; while the ductility is attributed to the activity of non-basal slip and the strong backstress.