The most intriguing properties of non-Hermitian systems are found near the exceptional points (EPs) at which the Hamiltonian matrix becomes defective. Because of the complex topological structure of the energy Riemann surfaces close to an EP and the breakdown of the adiabatic theorem due to non-Hermiticity, the state evolution in non-Hermitian systems is much more complex than that in Hermitian systems. For example, recent experimental work Doppler et al., Nature (London) 537, 76 (2016) demonstrated that dynamically encircling an EP can lead to chiral behaviors; i.e., encircling an EP in different directions results in different output states. Here, we propose a coupled ferromagnetic waveguide system that carries two EPs and design an experimental setup in which the trajectory of state evolution can be controlled in situ using a tunable external field, allowing us to dynamically encircle zero, one, or even two EPs experimentally. The tunability allows us to control the trajectory of encircling in the parameter space, including the size of the encircling loop and the starting/end point. We discovered that whether or not the dynamics is chiral actually depends on the starting point of the loop. In particular, dynamically encircling an EP with a starting point in the parity-time-broken phase results in nonchiral behaviors such that the output state is the same no matter which direction the encircling takes. The proposed system is a useful platform to explore the topology of energy surfaces and the dynamics of state evolution in non-Hermitian systems and will likely find applications in mode switching controlled with external parameters.