We present a detailed analysis of the Galaxy Stellar Mass Function (GSMF) of galaxies up to z =2.5 as obtained from the VIMOS VLT Deep Survey (VVDS). Our survey offers the possibility to investigate the GSMF using two different samples: (1) an optical ( I-selected 17.5 <I_{\rm AB}<24) main spectroscopic sample of about 6500 galaxies over 1750 arcmin super(2) and (2) a near-IR ( K-selected K_{\rm AB}<22.34 similar to {\rm and} similar to K_{\rm AB}<22.84) sample of about 10 200 galaxies, with photometric redshifts accurately calibrated on the VVDS spectroscopic sample, over 610 arcmin super(2). We apply and compare two different methods to estimate the stellar mass {\cal M}_{\rm stars} from broad-band photometry based on different assumptions about the galaxy star-formation history. We find that the accuracy of the photometric stellar mass is satisfactory overall, and show that the addition of secondary bursts to a continuous star formation history produces systematically higher (up to 40%) stellar masses. We derive the cosmic evolution of the GSMF, the galaxy number density and the stellar mass density in different mass ranges. At low redshift ( z\simeq0.2) we find a substantial population of low-mass galaxies (<10 { similar to M_\odot) composed of faint blue galaxies ( M_I-M_K \simeq 0.3). In general the stellar mass function evolves slowly up to z\sim0.9 and more rapidly above this redshift, in particular for low mass systems. Conversely, a massive population is present up to z =2.5 and has extremely red colours ( M_I-M_K\simeq 0.7-0.8). We find a decline with redshift of the overall number density of galaxies for all masses (59\pm5% for 10 proportional to similar to M_\odot$--> {\cal M}_{\rm stars} > 10 proportional to similar to M_\odot at z =1), and a mild mass-dependent average evolution ("mass-downsizing"). In particular our data are consistent with mild/negligible (<30%) evolution up to z\sim0.7 for massive galaxies ( }6\times10 6M_\odot$--> {>}6\times10 6M_\odot). For less massive systems the no-evolution scenario is excluded. Specifically, a large fraction ({\ge}50\%) of massive galaxies have been assembled and converted most of their gas into stars at z\sim1, ruling out "dry mergers" as the major mechanism of their assembly history below z\simeq1. This fraction decreases to {\sim}33\% at z\sim2. Low-mass systems have decreased continuously in number density (by a factor of up to 4.1\pm0.9) from the present age to z =2, consistent with a prolonged mass assembly also at z <1. The evolution of the stellar mass density is relatively slow with redshift, with a decrease of a factor of 2.3\pm0.1 at z =1 and about 4.5\pm0.3 at z =2.5.