Summary The paper focuses on the identification of atypical fractures (AFFs). This paper examines the concordance between objective classification and expert subjective review. We believe the paper adds critical information about how to apply the American Society of Bone and Mineral Research (ASBMR) criteria to diagnose AFFs and is of high interest to the field. Introduction Assess American Society of Bone and Mineral Research (ASBMR) criteria for identifying atypical femoral fractures (AFFs). Methods Two orthopedic surgeons independently evaluated radiographs of 372 fractures, applying ASBMR criteria. We assessed ease of applying ASBMR criteria and whether criteria-based assessment matched qualitative expert assessment. Results There was up to 27% uncertainty about how to classify specific features. 84% of films were classified similarly for the presence of AFF according to ASBMR criteria; agreement increased to 94% after consensus meeting. Of 37 fractures categorized as AFFs based on ASBMR criteria, 23 (62.2%) were considered AFFs according to expert assessment (not relying on criteria). Only one (0.5%) femoral shaft fracture that did not meet ASBMR criteria was considered an AFF per expert assessment. The number of major ASBMR features present (four vs five) and whether there was periosteal or endosteal thickening (“beaking” or “flaring”) played major roles in the discrepancies between ASBMR criteria-based and expert-based determinations. Conclusions ASBMR AFF criteria were useful for reviewers but several features were difficult to interpret. Expert assessments did not agree with the ASBMR classification in almost one-third of cases, but rarely identified an AFF when a femoral shaft fracture did not meet ASBMR AFF criteria. Experts identified lateral cortical transverse fracture line and associated new-bone formation along with no or minimal comminution as crucial features necessary for the definition of atypical femoral fractures.