The glass state of matter represents a frozen state of an atomically disordered system with local order only. Instead of atoms, systems with glassy states of magnetic and electric dipole moments in solids are known as spin and dipole glasses, respectively. In these conventional glasses, slow dynamics, such as relaxation and memory phenomena, are characteristics of their magnetic/dielectric properties. Here we propose a new glassy state in solids, a 'toroidal glass', in which toroidal moments-vector-like electromagnetic multipole moments breaking both space inversion and time reversal symmetries, and producing a linear magnetoelectric coupling-are randomly oriented and frozen. We investigate the dynamics of a linear magnetoelectric effect in Ni0.4Mn0.6TiO3 and find that the magnetoelectric responses strongly depend on the magnetoelectric cooling history and show striking memory effects. These unusual magnetoelectric dynamical features can be explained in the framework of a toroidal glass in which the toroidal frozen state can be controlled magnetoelectrically.