•Inflammation has been identified as a biological process relevant in depression.•C-reactive protein is a biomarker of inflammation.•Methodological inconsistences are widespread in the field.•Methodological inconsistences in the literature hinder reproducibility.•Methodologically rigorous studies are needed to drive progress in the field. One of the most common inflammatory markers examined in depression is C-reactive protein (CRP). However, the magnitude of the association between CRP and depression when controlling for potentially confounding factors such as age, sex, socio-economic status, body mass index, medication and other substance use, and medical illness, is unclear. Inconsistencies in other methodological practices, such as sample collection, assaying, and data cleaning and transformation, may contribute to variations in results. We aggregate studies that examined the association between CRP and depression in two ways. First, a systematic review summarizes how studies of CRP and depression have reported on methodological issues. Second, a tiered meta-analysis aggregates studies that have adhered to various levels of methodological rigor. Findings from the systematic review indicate a lack of protocol detail provided. The effect between depression and CRP was small, but highly significant across all stages of the meta-analysis (p < 0.01). The effect size in the most methodologically rigorous stage of the meta-analysis, which included studies controlling for age, sex, obesity, medical conditions and substance, medication, or psychosocial factors, was small (r = 0.05). There were also only 26 articles in this stage (13% of studies from the systematic review), suggesting that more studies that consistently account for these confounding factors are needed. Additionally, an a priori quality score of methodological rigor was a significant moderator in this stage of the meta-analysis. The effect size was strikingly attenuated (r = 0.005) and non-significant in studies with higher quality scores. We describe a set of recommended guidelines for future research to consider, including sample collection and assaying procedures, data cleaning and statistical methods, and control variables to assess.