The development of modern technology has posed greater and more urgent needs for thermal management materials. Aligned carbon nanotube arrays and carbon/carbon composites have aroused extensive interest as ideal lightweight and stable thermal management materials because of their low thermal expansion coefficient, excellent thermal conduction and high-temperature resistance. Here, we first review the thermal conducting mechanism of carbon materials. We then describe the general fabrication methods, the main factors affecting the thermal conductivity of aligned carbon nanotube arrays and carbon/carbon composites as well as their use in thermal management. The preparation-structure-performance relationships are outlined and the strategies for achieving high thermal conductivity are summarized. Finally, a critical consideration of the challenges and prospects in the thermal management applications of aligned carbon nanotubes and carbon/carbon composites is given.