We investigate the long-term optical-infrared variability of SV Cep, and explain it in the context of an existing UX Ori (UXOR) model. A 25-month monitoring programme was completed with the Infrared Space Observatory in the 3.3-100 um wavelength range. Following a careful data reduction, the infrared light curves were correlated with the variations of SV Cep in the V-band. A remarkable correlation was found between the optical and the far-infrared light curves. In the mid-infrared regime the amplitude of variations is lower, with a hint for a weak anti-correlation with the optical changes. In order to interpret the observations, we modelled the spectral energy distribution of SV Cep assuming a self-shadowed disc with a puffed-up inner rim, using a 2-dimensional radiative transfer code. We found that modifying the height of the inner rim, the wavelength-dependence of the long-term optical-infrared variations is well reproduced, except the mid-infrared domain. The origin of variation of the rim height might be fluctuation in the accretion rate in the outer disc. In order to model the mid-infrared behaviour we tested to add an optically thin envelope to the system, but this model failed to explain the far-infrared variability. Infrared variability is a powerful tool to discriminate between models of the circumstellar environment. The proposed mechanism of variable rim height may not be restricted to UXOR stars, but might be a general characteristic of intermediate-mass young stars.