To investigate whether extent and severity of white matter (WM) damage, as measured with diffusion tensor imaging (DTI), can distinguish cognitively preserved (CP) from cognitively impaired (CI) multiple sclerosis (MS) patients. Conventional MRI and DTI data were acquired from 55 MS patients (35 CP, 20 CI) and 30 healthy controls (HC). Voxelwise analyses were used to investigate fractional anisotropy (FA), mean diffusivity, radial diffusivity, and axial diffusivity of a WM skeleton. Regional gray matter volume was quantified and lesion probability maps were generated. Compared to HCs, decreased FA was found in 49% of the investigated WM skeleton in CP patients and in 76% of the investigated WM in CI patients. Several brain areas that showed reduced FA in both patient groups were significantly worse in CI patients, i.e, corpus callosum, superior and inferior longitudinal fasciculus, corticospinal tracts, forceps major, cingulum, and fornices. In CI patients, WM integrity damage was additionally seen in cortical brain areas, thalamus, uncinate fasciculus, brainstem, and cerebellum. These findings were independent of lesion location and regional gray matter volume, since no differences were found between the groups. CI patients diverged from CP patients only on DTI metrics. WM integrity changes were found in areas that are highly relevant for cognition in the CI patients but not in the CP patients. These WM changes are therefore thought to be related to the cognitive deficits and suggest that DTI might be a powerful tool when monitoring cognitive impairment in MS.