Galactose is an abundant and essential sugar used for the biosynthesis of many macromolecules in different organisms, including plants. Galactose metabolism is tightly and finely controlled, since excess galactose and its derivatives are inhibitory to plant growth. In Arabidopsis (Arabidopsis thaliana), root growth and pollen germination are strongly inhibited by excess galactose. However, the mechanism of galactose-induced inhibition during pollen germination remains obscure. In this study, we characterized a plasma membrane-localized transporter, Arabidopsis Sugars Will Eventually be Exported Transporter 5, that transports glucose and galactose. SWEET5 protein levels started to accumulate at the tricellular stage of pollen development and peaked in mature pollen, before rapidly declining after pollen germinated. SWEET5 levels are responsible for the dosage-dependent sensitivity to galactose, and galactokinase is essential for these inhibitory effects during pollen germination. However, sugar measurement results indicate that galactose flux dynamics and sugar metabolism, rather than the steady-state galactose level, may explain phenotypic differences between sweet5 and Col-0 in galactose inhibition of pollen germination.