Potassium (K+) channels can regulate ionic conduction through their pore by a mechanism, involving the selectivity filter, known as C-type inactivation. This process is rapid in the hERG K+ channel and is fundamental to its physiological role. Although mutations within hERG are known to remove this process, a structural basis for the inactivation mechanism has yet to be characterized. Using MD simulations based on homology modeling, we observe that the carbonyl of the filter aromatic, Phe627, forming the S0 K+ binding site, swiftly rotates away from the conduction axis in the wild-type channel. In contrast, in well-characterized non-inactivating mutant channels, this conformational change occurs less frequently. In the non-inactivating channels, interactions with a water molecule located behind the selectivity filter are critical to the enhanced stability of the conducting state. We observe comparable conformational changes in the acid sensitive TASK-1 channel and propose a common mechanism in these channels for regulating efflux of K+ ions through the selectivity filter.