Abstract Background Female reproductive tract microbiota may affect human reproduction. The current study considered whether a more detailed characterization of the vaginal microbiota could improve prediction of risk of poor reproductive outcome in patients undergoing in vitro fertilization (IVF). Methods Vaginal samples from 120 patients undergoing IVF were sequenced using the V4 region of the 16S ribosomal RNA gene with clustering of Gardnerella vaginalis genomic clades. Abnormal vaginal microbiota was defined by microscopy and quantitative polymerase chain reaction (qPCR) for G. vaginalis and/or Atopobium vaginae above a threshold. Results Three major community state types with abundance of Lactobacillus crispatus, Lactobacillus iners, and a diverse community type were identified, including 2 subtypes, characterized by a high abundance of L. crispatus and L. iners, respectively, but in combination with common diversity type operational taxonomic units. No significant association between community state type and the reproductive outcome could be demonstrated; however, abnormal vaginal microbiota by qPCR and a grouping based on high Shannon diversity index predicted the reproductive outcome equally well. Conclusions The predictive value of 16S ribosomal RNA gene sequencing was not superior to the simpler and less expensive qPCR diagnostic approach in predicting the risk of a poor reproductive outcome in patients undergoing IVF. Clinical Trials Registration NCT02042352 Analysis of diagnostic methods for defining abnormal vaginal microbiota in patients undergoing in vitro fertilization treatment shows that quantitative polymerase chain reaction targeting Gardnerella vaginalis and Atopobium vaginae provides clinicians with a valid, robust, and accessible method of diagnosis.