Objective The aim of this study was to explore the occurrence and prognostic value of perineural invasion (PNI) as a classic tumor pathological feature in esophageal squamous cell carcinoma (ESCC). Methods We retrospectively enrolled 794 ESCC patients who underwent radical esophagectomy at Shanghai Chest Hospital from 2017 to 2018. The incidence, associated factors, and prognosis of PNI were analyzed. Results PNI was identified in 15.7% (125/794) of patients. The presence of PNI was significantly associated with depth of invasion ( p < 0.001), pN stage ( p = 0.008), tumor stage ( p < 0.001), and lymphovascular invasion (LVI; p < 0.001). Multivariate logistic regression analysis demonstrated that advanced pT stage and LVI were independently associated with the presence of PNI, while multivariate Cox regression analysis demonstrated that PNI was not an independent risk factor for poor overall survival (OS) or recurrence-free survival (RFS) in ESCC patients (OS hazard ratio HR 0.688, 95% confidence interval CI 0.448–1.056, p = 0.087; RFS HR 0.837, 95% CI 0.551–1.273, p = 0.406). In the PNI-positive patient subgroup, adjuvant therapy was associated with better OS and RFS. Conclusion PNI correlates with, and may be a concomitant consequence of, LVI and advanced tumor invasion (T3–4) in ESCC patients. Although PNI was not identified as an independent prognostic indicator, our results suggest ESCC patients with PNI should be considered for adjuvant therapy.