In the present study, the welding thermal cycle of magnesium-treated EH36 steel was simulated by a thermal simulation experiment machine. The effects of welding heat input on the microstructure and low temperature toughness of the welded joint were studied, and the mechanism of acicular ferrite nucleation induced by MgAl2O4 inclusion was revealed. The results showed that, when the welding heat input of 150 kJ/cm, 250 kJ/cm and 350 kJ/cm was utilized, the microstructure of heat affected zone in the experimental EH36 steel consisted of grain boundary ferrite, acicular ferrite, granular bainite, and a small amount of pearlite, but the volume fraction and grain size of each independent microstructure were different. With the increase of welding heat input, the content of acicular ferrite and granular bainite decreased, the content of grain boundary ferrite increased, and the average grain size increased, whereas the low temperature toughness of the experimental EH36 steel welded joint reduced from 174 to 67 J. Furthermore, MgAl2O4 inclusions can induce acicular ferrite nucleation, whereas the nucleation mechanism is a combination of low lattice mismatch and low interfacial energy.