Three Pol X family members have been linked to nonhomologous end joining (NHEJ) in mammals. Template-independent TdT promotes diversity during NHEJ-dependent repair of V(D)J recombination intermediates, but the roles of the template-dependent polymerases μ and λ in NHEJ remain unclear. We show here that pol μ and pol λ are similarly recruited by NHEJ factors to fill gaps when ends have partially complementary overhangs, suggesting equivalent roles promoting accuracy in NHEJ. However, only pol μ promotes accuracy during immunoglobulin kappa recombination. This distinctive in vivo role correlates with the TdT-like ability of pol μ, but not pol λ, to act when primer termini lack complementary bases in the template strand. However, unlike TdT, synthesis by pol μ in this context is primarily instructed by a template from another DNA molecule. This apparent gradient of template dependence is largely attributable to a small structural element that is present but different in all three polymerases.