Massive stars are crucial to galactic chemical evolution for elements heavier than iron. Their contribution at early times in the evolution of the Universe, however, is unclear due to poorly constrained nuclear reaction rates. The competing $^{17}$O($\alpha,\gamma$)$^{21}$Ne and $^{17}$O($\alpha,n$)$^{20}$Ne reactions strongly impact weak s-process yields from rotating massive stars at low metallicities. Abundant $^{16}$O absorbs neutrons, removing flux from the s-process, and producing $^{17}$O. The $^{17}$O($\alpha,n$)$^{20}$Ne reaction releases neutrons, allowing continued s-process nucleosynthesis, if the $^{17}$O($\alpha,\gamma$)$^{21}$Ne reaction is sufficiently weak. While published rates are available, they are based on limited indirect experimental data for the relevant temperatures and, more importantly, no uncertainties are provided. The available nuclear physics has been evaluated, and combined with data from a new study of astrophysically relevant $^{21}$Ne states using the $^{20}$Ne($d,p$)$^{21}$Ne reaction. Constraints are placed on the ratio of the ($\alpha,n$)/($\alpha,\gamma$) reaction rates with uncertainties on the rates provided for the first time. The new rates favour the ($\alpha,n$) reaction and suggest that the weak s-process in rotating low-metallicity stars is likely to continue up to barium and, within the computed uncertainties, even to lead.