Transformations of the four polymorphs and magnetic transition of BaRuO3 under high pressure have been studied using first-principles calculations. We search for the low-enthalpy structures by relaxing four initial structural configurations under high pressure. We find that BaRuO3 exhibits sequential structural changes from nine-layer rhombohedral (9R), four-layer hexagonal (4H), six-layer hexagonal (6H), and finally to cubic perovskite (3C) structure, consistent well with the experiments. The three hexagonal crystallographic forms (9R, 4H and 6H) of BaRuO3 show paramagnetic states, whereas 3C-BaRuO3 displays a ferromagnetic ground state. Along with the pressure increasing, the magnetic moment of Ru in the 3C-BaRuO3 first come through a discontinuous drop around the critical pressure 26aaGPa, then becomes smaller gradually until zero at 180aaGPa. According to the theory of Moruzzi, this magnetic-nonmagnetic transition belongs to the type-III transition. The calculated pressure dependence of the enthalpies (a) and the volumes (b) of BaRuO3, reveals the pressure-induced phase transformation of BaRuO3. Display Omitted