In neurons, the distinct molecular composition of axons and dendrites is established through polarized targeting mechanisms, but it is currently unclear how nonpolarized cargoes, such as mitochondria, become uniformly distributed over these specialized neuronal compartments. Here, we show that TRAK family adaptor proteins, TRAK1 and TRAK2, which link mitochondria to microtubule-based motors, are required for axonal and dendritic mitochondrial motility and utilize different transport machineries to steer mitochondria into axons and dendrites. TRAK1 binds to both kinesin-1 and dynein/dynactin, is prominently localized in axons, and is needed for normal axon outgrowth, whereas TRAK2 predominantly interacts with dynein/dynactin, is more abundantly present in dendrites, and is required for dendritic development. These functional differences follow from their distinct conformations: TRAK2 preferentially adopts a head-to-tail interaction, which interferes with kinesin-1 binding and axonal transport. Our study demonstrates how the molecular interplay between bidirectional adaptor proteins and distinct microtubule-based motors drives polarized mitochondrial transport. ► Kinesin-1 and dynein drive mitochondrial transport to axons and dendrites ► TRAK1 binds to both kinesin-1 and dynein/dynactin and is localized in axons ► TRAK2 predominantly interacts with dynein/dynactin and is present in dendrites ► TRAK2 backfolding inhibits kinesin-1 binding and controls mitochondrial transport van Spronsen et al. show that mitochondria utilize different machineries to steer their transport into axons and dendrites. The molecular interplay between mitochondrial adaptor protein family TRAK/Milton and distinct microtubule-based motors drives polarized mitochondrial transport.