Cronbach's alpha estimates the internal consistency of responses in multi-item bipolar scales. This article examined three research questions (RQ): (1) To what extent do inconsistencies exist in data (e.g., responses of −2 −2 2 2)? (2) Does the number of scale items influence the amount of inconsistency? (3) Does Cronbach's alpha mask inconsistencies? Data were obtained from 29 research projects (n = 10,616). Each survey had place attachment questions comprising two concepts: place identity and place dependence. Respondents were classified as consistent or inconsistent based on their responses to the place attachment questions. Results demonstrated that: (a) inconsistent response patterns existed in the data (RQ1), (b) number of scale items influenced amount of inconsistency (RQ2), and (c) alpha masked these inconsistencies (RQ3). Discussion focused on implications of these findings.