Abstract Objective The High-Alert Medication Stratification Tool‒Revised (HAMST-R) was originally designed to standardize the identification of high-alert medications (HAMs) according to safety risk. The primary objective of this multisite study was to assess interrater reliability of the HAMST-R PRO, a version of the tool designed to prospectively evaluate safety risk of medications during evaluation for formulary addition. Methods HAMST-R was designed as an objective tool to evaluate HAMs at a single site during the HAMST-R phase I study. Phase II of the study demonstrated the validity of the tool in a multisite, national study. In this third study, 11 medication safety experts from 8 health systems across the United States and 1 in Canada facilitated evaluation of medications prospectively with the HAMST-R PRO during the formulary review process for 27 medications. At each site, at least 5 individuals were asked to review each medication. Interrater reliability was evaluated using Kendall’s coefficient of concordance. Ease of use was determined by participant interviews. Results Overall interrater reliability for HAMST-R PRO was found to be 0.76 (P < 0.001) across all sites, indicating substantial agreement between users. Interrater reliability among individual sites ranged from 0.52 to 0.82 (P < 0.05 for all sites). Conclusion Interrater reliability of HAMST-R PRO is substantial, indicating consistency and agreement among pharmacists utilizing this tool to evaluate safety risk of medications before their addition to a health-system formulary. This information can be used to identify potential interventions for each step of the medication-use process that institutions may implement to decrease a medication’s potential safety risk.