To develop and validate Clinical Diversity In Meta-analyses (CDIM), a new tool for assessing clinical diversity between trials in meta-analyses of interventions. The development of CDIM was based on consensus work informed by empirical literature and expertise. We drafted the CDIM tool, refined it, and validated CDIM for interrater scale reliability and agreement in three groups. CDIM measures clinical diversity on a scale that includes four domains with 11 items overall: setting (time of conduct/country development status/units type); population (age, sex, patient inclusion criteria/baseline disease severity, comorbidities); interventions (intervention intensity/strength/duration of intervention, timing, control intervention, cointerventions); and outcome (definition of outcome, timing of outcome assessment). The CDIM is completed in two steps: first two authors independently assess clinical diversity in the four domains. Second, after agreeing upon scores of individual items a consensus score is achieved. Interrater scale reliability and agreement ranged from moderate to almost perfect depending on the type of raters. CDIM is the first tool developed for assessing clinical diversity in meta-analyses of interventions. We found CDIM to be a reliable tool for assessing clinical diversity among trials in meta-analysis.