The diagnosis of achalasia is associated with an average delay of two years. Endoscopic features may prompt an earlier diagnosis. We aimed to develop and test a novel endoscopic CARS score for the prediction of achalasia. Part 1: Twenty endoscopic videos were taken from patients undergoing endoscopy for dysphagia or reflux. A survey with videos and endoscopic criteria options was distributed to 6 esophagologists and 6 general gastroenterologists. Inter-rater reliability (IRR) was measured and logistic regression was used to evaluate predictive performance. Three rounds of review were conducted to select the final score of four components. A retrospective review was conducted for consecutive patients who had comprehensive esophageal testing. Each patient had a CARS endoscopic score calculated based on findings reported at endoscopy. From a video review and analysis of score components, IRR ranged from 0.23 to 0.57 for score components. The final CARS score was selected based on the following four components: Contents, Anatomy, Resistance, and Stasis. In a mixed effects model, the mean score across raters was higher for achalasia compared to non-achalasia subjects (4.44 vs. 0.87, p = < 0.01). In part 2 of the study, achalasia patients had a higher mean CARS score compared to those with no / ineffective motility disorder (mean 4.1 vs 1.3, p = < 0.01). We developed a CARS score based on reliability performance in a video-based survey and tested the score in clinical setting. The CARS score performed well in predicting achalasia.