It is well known that the discretization of fractional diffusion equations with fractional derivatives α∈(1,2)$$ \alpha \in \left(1,2\right) $$, using the so‐called weighted and shifted Grünwald formula, leads to linear systems whose coefficient matrices show a Toeplitz‐like structure. More precisely, in the case of variable coefficients, the related matrix sequences belong to the so‐called generalized locally Toeplitz class. Conversely, when the given FDE has constant coefficients, using a suitable discretization, we encounter a Toeplitz structure associated to a nonnegative function ℱα$$ {\mathcal{F}}_{\alpha } $$, called the spectral symbol, having a unique zero at zero of real positive order between one and two. For the fast solution of such systems by preconditioned Krylov methods, several preconditioning techniques have been proposed in both the one‐ and two‐dimensional cases. In this article we propose a new preconditioner denoted by 𝒫ℱα which belongs to the τ$$ \tau $$‐algebra and it is based on the spectral symbol ℱα$$ {\mathcal{F}}_{\alpha } $$. Comparing with some of the previously proposed preconditioners, we show that although the low band structure preserving preconditioners are more effective in the one‐dimensional case, the new preconditioner performs better in the more challenging multi‐dimensional setting.