Metazoan guts are in permanent contact with microbial communities. However, the host mechanisms that have developed to manage the dynamic changes of these microorganisms and maintain homeostasis remain largely unknown. Serotonin (5-hydroxytryptamine 5-HT) was found to modulate gut microbiome homeostasis via regulation of a dual oxidase (Duox) gene expression in both Bactrocera dorsalis and Aedes aegypti. The knockdown of the peripheral 5-HT biosynthetic gene phenylalanine hydroxylase (TPH) increased the expression of Duox and the activity of reactive oxygen species, leading to a decrease in the gut microbiome load. Moreover, the TPH knockdown reduced the relative abundance of the bacterial genera Serratia and Providencia, including the opportunistic pathogens, S. marcescens and P. alcalifaciens in B. dorsalis. Treatment with 5-hydroxytryptophan, a precursor of 5-HT synthesis, fully rescued the TPH knockdown-induced phenotype. The findings reveal the important contribution of 5-HT in regulating gut homeostasis, providing new insights into gut-microbe interactions in metazoans.