In recent years, genetic programming has achieved impressive results on evolutionary feature construction tasks. To increase search effectiveness, researchers have developed many semantic-based crossover and mutation operators to guide genetic programming searches toward the target semantics. However, semantics has not yet been explored for the hoist mutation operator, which is an operator designed for controlling the bloat effect. Although the hoist mutation operator can significantly reduce model sizes, the most informative subtree may be disrupted by the randomness in mutation. To address this issue, we develop a semantic-based hoist mutation operator in this paper to preserve the most informative subtree that has the largest cosine similarity between its semantics and the target semantics. Experimental results on 98 regression datasets from the Penn Machine Learning Benchmark show that using this operator not only significantly reduces model size, but also improves the test accuracy of features constructed by genetic programming. A comparison with seven bloat control methods shows that the proposed operator achieves the best trade-off between accuracy and model size. Moreover, an experiment on the state-of-the-art symbolic regression benchmark shows that genetic programming with the semantic-based hoist mutation operator achieves the best test accuracy and competitive model sizes compared with 22 symbolic regression and machine learning algorithms.