In this Letter we consider a general quadratic parity-preserving theory for a general flat connection. Imposing a local symmetry under the general linear group singles out the general teleparallel equivalent of General Relativity carrying both torsion and non-metricity. We provide a detailed discussion on the teleparallel equivalents of General Relativity and how the two known equivalents, formulated on Weitzenböck and symmetric teleparallel geometries respectively, can be interpreted as two gauge-fixed versions of the general teleparallel equivalent. We then explore the viability of the general quadratic theory by studying the spectrum around Minkowski. The linear theory generally contains two symmetric rank-2 fields plus a 2-form and, consequently, extra gauge symmetries are required to obtain potentially viable theories.