A bstract We provide the all-loop structure of gauge-variant operators required for the renormalisation of Green’s functions with insertions of twist-two operators in Yang-Mills theory. Using this structure we work out an explicit basis valid up to 4-loop order for an arbitrary compact simple gauge group. To achieve this we employ a generalised gauge symmetry, originally proposed by Dixon and Taylor, which arises after adding to the Yang-Mills Lagrangian also operators proportional to its equation of motion. Promoting this symmetry to a generalised BRST symmetry allows to generate the ghost operator from a single exact operator in the BRST-generalised sense. We show that our construction complies with the theorems by Joglekar and Lee. We further establish the existence of a generalised anti-BRST symmetry which we employ to derive non-trivial relations among the anomalous dimension matrices of ghost and equation-of-motion operators. For the purpose of demonstration we employ the formalism to compute the N = 2 , 4 Mellin moments of the gluonic splitting function up to 4 loops and its N = 6 Mellin moment up to 3 loops, where we also take advantage of additional simplifications of the background field formalism.