Objectives An evaluation of the effectiveness of a new computational system proposed for automatic classification, developed based on a Siamese network combined with Convolutional Neural Networks (CNNs), is presented. It aims to identify endodontic technical errors using Cone Beam Computed Tomography (CBCT). The study also aims to compare the performance of the automatic classification system with that of dentists. Methods One thousand endodontically treated maxillary molars sagittal and coronal reconstructions were evaluated for the quality of the endodontic treatment and the presence of periapical hypodensities by three board-certified dentists and by an oral and maxillofacial radiologist. The proposed classification system was based on a Siamese network combined with EfficientNet B1 or EfficientNet B7 networks. Accuracy, sensivity, precision, specificity, and F1-score values were calculated for automated artificial systems and dentists. Chi-square tests were performed. Results The performances were obtained for EfficienteNet B1, EfficientNet B7 and dentists. Regarding accuracy, sensivity and specificity, the best results were obtained with EfficientNet B1. Concerning precision and F1-score, the best results were obtained with EfficientNet B7. The presence of periapical hypodensity lesions was associated with endodontic technical errors. In contrast, the absence of endodontic technical errors was associated with the absence of hypodensity. Conclusions Quality evaluation of the endodontic treatment performed by dentists and by Siamese Network combined with EfficientNet B7 or EfficientNet B1 networks was comparable with a slight superiority for the Siamese Network. Clinical relevance CNNs have the potential to be used as a support and standardization tool in assessing endodontic treatment quality in clinical practice.