This paper is devoted to the study of multi-agent deterministic optimal control problems. We initially provide a thorough analysis of the Lagrangian, Eulerian and Kantorovich formulations of the problems, as well as of their relaxations. Then we exhibit some equivalence results among the various representations and compare the respective value functions. To do it, we combine techniques and ideas from optimal transportation, control theory, Young measures and evolution equations in Banach spaces. We further exploit the connections among Lagrangian and Eulerian descriptions to derive consistency results as the number of particles/agents tends to infinity. To that purpose we prove an empirical version of the Superposition Principle and obtain suitable Gamma-convergence results for the controlled systems.