Video-based telemedicine (vs audio only) is less frequently used in diverse, low socioeconomic status settings. Few prior studies have evaluated the impact of telemedicine modality (ie, video vs audio-only visits) on clinical quality metrics. The aim of this study was to assess telemedicine uptake and impact of visit modality (in-person vs video and phone visits) on primary care quality metrics in diverse, low socioeconomic status settings through an implementation science lens. Informed by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, we evaluated telemedicine uptake, assessed targeted primary care quality metrics by visit modality, and described provider-level qualitative feedback on barriers and facilitators to telemedicine implementation. We found marginally better quality metrics (ie, blood pressure and depression screening) for in-person care versus video and phone visits; de-adoption of telemedicine was marked within 2 years in our population. Following the widespread implementation of telemedicine during the COVID-19 pandemic, the impact of visit modality on quality outcomes, provider and patient preferences, as well as technological barriers in historically marginalized settings should be considered.