It is assumed that diet influences the composition of gut microbiota, which in turn may affect human health status. This systematic review aimed to summarize associations of a vegan or vegetarian diet with the composition of microbiota. A literature search was conducted in PubMed and Embase for eligible human studies with vegan or vegetarian diets as an exposure and microbiota composition as an outcome in healthy adults. Furthermore, data from our cross-sectional study with vegan participants were included. Out of sixteen included studies, six investigated the association between gut microbiota composition in both vegans and in vegetarians, six in vegans and four studies in vegetarians compared to omnivores, respectively. Among 5 different phyla, 28 families, 96 genera and 177 species, Bacteroides, Bifidobacterium and Prevotella were the most reported genera, followed by the species Prevotella copri, Faecalibacterium prausnitzii and Escherichia coli in all diets. No consistent association between a vegan diet or vegetarian diet and microbiota composition compared to omnivores could be identified. Moreover, some studies revealed contradictory results. This result could be due to high microbial individuality, and/or differences in the applied approaches. Standardized methods with high taxonomical and functional resolutions are needed to clarify this issue.