Germline pathogenic variants in the exonuclease domain (ED) of polymerases POLE and POLD1 predispose to adenomatous polyps, colorectal cancer (CRC), endometrial tumors, and other malignancies, and exhibit increased mutation rate and highly specific associated mutational signatures. The tumor spectrum and prevalence of POLE and POLD1 variants in hereditary cancer are evaluated in this study. POLE and POLD1 were sequenced in 2813 unrelated probands referred for genetic counseling (2309 hereditary cancer patients subjected to a multigene panel, and 504 patients selected based on phenotypic characteristics). Cosegregation and case-control studies, yeast-based functional assays, and tumor mutational analyses were performed for variant interpretation. Twelve ED missense variants, 6 loss-of-function, and 23 outside-ED predicted-deleterious missense variants, all with population allele frequencies <1%, were identified. One ED variant (POLE p.Met294Arg) was classified as likely pathogenic, four as likely benign, and seven as variants of unknown significance. The most commonly associated tumor types were colorectal, endometrial and ovarian cancers. Loss-of-function and outside-ED variants are likely not pathogenic for this syndrome. Polymerase proofreading-associated syndrome constitutes 0.1-0.4% of familial cancer cases, reaching 0.3-0.7% when only CRC and polyposis are considered. ED variant interpretation is challenging and should include multiple pieces of evidence.