Background Mitochondrial genetics are important in the etiology of Alzheimer’s disease (AD). Based on the polymorphisms in mitochondrial DNA (mtDNA) the population can be classified into subgroups called the haplogroups. Previous studies have explored the association of haplogroups and genetic variants in APOE ε4 with AD but the results are inconsistent. Also, patients with mild cognitive impairment (MCI) at baseline often progress differently over several years of follow up. Method We carried out a cross‐sectional study to examine the association of the rate of incidence of AD with APOE ε4 genotype and haplogroups. The data collected at the University of Kansas AD Center (KU ADC) consisting of 146 AD and 265 normal subjects, and the data collected by AD Neuroimaging Initiative (ADNI) consisting of 244 AD and 242 normal subjects were used for these analyses. Logistic regression and Fisher Exact tests were used to explore the associations. We also carried out the association analyses of the differences in the disease progression rates from MCI to AD among the haplogroups using both datasets. Result Rates of incidence of AD were statistically significantly different among the haplogroups (2.2 × 10−8), with differing strengths of association across haplogroups. APOE ε4 was significantly associated with AD overall (p‐value = 1.9 × 10−11) and the association further varied by haplogroups. The results of the KU ADC population were consistent with those of the ADNI data. Out of 112 MCI subjects at baseline in KU ADC, 26 progressed to AD and 20 reverted back to normal at the last clinical visit. In ADNI, 172 progressed to AD and 42 reverted back out of 482 MCI at baseline. For both datasets, the proportion of subjects who progressed to AD differed by haplogroups. APOE ε4 genotype was significantly associated with increased risk of progression from MCI to AD (p‐value = 8.6 × 10−7 KUADC and p‐value = 1.33 × 10−9 ADNI). Conclusion The rate of incidence of AD differed by mtDNA haplogroup and the strength of association further differed by APOE ε4 genotype. The rates of progression of MCI subjects to AD also differed by haplogroups and APOE ε4 genotype.