Purpose Oncology electronic health record (EHR) databases have increased in quality and availability over the past decade, yet it remains unclear whether these clinical practice data can be used to conduct reliable comparative effectiveness studies. We sought to emulate a clinical trial with EHR data in the advanced breast cancer population and compare our results against the trial. Methods This cohort study used EHR data from US oncology practices. All elements of the study were defined to mimic the PALOMA‐2 trial as closely as possible. Patients with hormone‐positive, HER‐2 negative metastatic breast cancer with no prior treatment for metastatic disease were included. Patients initiating palbociclib and letrozole on the same day following the earliest record of metastasis were compared to those initiating letrozole only. The primary associational measure was the conditional hazard ratio for time‐to‐next treatment (TTNT). TTNT is well‐measured in our data source and amenable for calibration against the randomized study results of the PALOMA‐2 trial. We used multiple imputation for several patient characteristics with missing values. Results There were 3836 study‐eligible women with advanced breast cancer. The hazard ratio for TTNT in the observational study (HR: 0.62; 95% CI: 0.56–0.68) was closely aligned with that of the randomized trial (HR: 0.64; 95% CI: 0.52–0.78). Conclusions Under our assumptions on missing data and comparability of the two study populations, results from our non‐randomized study closely matched that of the randomized trial. Further studies are needed to determine whether EHR data can yield reliable conclusions on treatment effects in oncology.