The effective medium theory of one-dimensional and two-dimensional periodic structures are investigated. A method based on a Fourier decomposition of the wave propagating along the direction perpendicular to the periodic structures allows one to determine the zeroth-, first- and second-order effective indices. For one-dimensional problems, we derive closed-form expressions of the effective indices for both TE and TM polarization. Our result can be applied to arbitrary periodic structure with symmetric or non-symmetric lamellar or continuously varying index profiles. The theoretical predictions are carefully validated using rigorous coupled-wave analysis. For the two-dimensional case, only symmetric structures are discussed and the computation of the zeroth-, first-, and second-order effective indices requires the inversion of an infinite matrix which can be truncated and simply solved numerically. The EMT prediction is qualitatively validated using rigorous computation for small period-to-wavelength ratios. It is shown that for large period-to-wavelength ratios near the cutoff value, no analogy between 2-D periodic structures and homogeneous media holds for highly modulated lamellar gratings.