We report high-field magnetization, high-frequency electron spin resonance (ESR), and Se77 nuclear magnetic resonance (NMR) measurements on the linear spin tetramer system CuSeO3, consisting of strongly interacting Cu(1) dimers and weakly coupled Cu(2) spins. The magnetization exhibits anisotropic half-step magnetization plateaus at μ0H=45 T, depending on a crystallographic orientation. A temperature dependence of the ESR linewidth ΔHpp in a paramagnetic phase points towards the significance of anisotropic exchange interactions. Below TN=9–10 K long-range magnetic order is evidenced by the observation of a critical divergence of both ΔHpp(T) and the nuclear spin-lattice relaxation rate 1/T1. In addition, we identify a magnetic anomaly at T*=6.0(5) K below TN, which is caused by a spin reorientation. The nuclear spin-spin relaxation rate 1/T2 unveils the development of site-specific spin correlations. The intriguing magnetism of CuSeO3 is discussed in terms of the energy hierarchy of Cu(1) and Cu(2) spins in concert with additional intertetramer interactions.