The Measure of Acceptance of the Theory of Evolution (MATE) has been extensively used in science education research for more than two decades. This study examines the fairness of MATE items based on religious convictions and academic majors. The multidimensional item response theory and differential item functioning analyses were run on data collected from 619 Korean undergraduate students. These participants included non-religious and Christian groups, as well as biology and non-biology majors. The results supported the two-dimensional model of MATE, consisting of the constructs of fact and credibility. The results indicated that all items were psychometrically sound. Further, the MATE instrument was a reliable measure of students' acceptance of the evolutionary theory. Additionally, DIF analysis showed that several MATE items were interpreted differently by students from different subgroups. Specifically, we found four religion-related DIF items and two academic major-related DIF items. The effect of DIF items on the estimated individual scores showed that DIF items widened the gaps between the MATE scores of the subgroups. The findings underscore the importance of the generalizability of an instrument, particularly the acceptance of the evolutionary theory, to achieve more accurate interpretations of the scores generated by the instrument.