Abstract Background The relationship between glycemic variability, another component of glycemic disorders as well as chronic sustained hyperglycemia, and cardiovascular autonomic neuropathy (CAN) has not been clarified. Our aim is to investigate the association between glycemic variability and CAN in newly diagnosed type 2 diabetic patients. Methods Ewing tests were performed in 90 newly diagnosed type 2 diabetic patients and 37 participants with normal glucose tolerance as control from May 1, 2009, through September 30, 2010. According to the scores from Ewing tests, diabetic patients were divided into two groups: without CAN (CAN −) and with CAN (CAN +). All participants underwent a 48-h to 72-h continuous glucose monitoring (CGM). Coefficient of variability of glycemia (%CV), mean amplitude of glycemic excursions (MAGE) and means of daily differences (MODD) were calculated with the CGM data. Results The prevalence of CAN in patients with newly diagnosed type 2 diabetes was 22.2%. An increasing trend of glycemic variability was found from control group, CAN − group to CAN + group. MAGE in CAN + group was significantly higher than that in CAN − group (5.27 ± 1.99 mmol/L vs. 4.04 ± 1.39 mmol/L, P = 0.001). In the Logistic regression analysis, a significant relationship was shown between MAGE and CAN odds ratio (OR): 1.73, 95% confidence interval (CI): 1.01–2.73, P = 0.018). The area under the receiver-operating characteristic curve for MAGE was superior to those for other dysglycemic indices in detecting CAN. Conclusions Glycemic variability is associated with CAN in patients with newly diagnosed type 2 diabetes. Among the glycemic variability indices, MAGE is a significant indicator for detecting CAN.