Nanocarbon materials, such as carbon nanotube and graphene, are reckoned as the ideal reinforcement for composites due to their extraordinary intrinsic properties. However, the commonly-used homogenous architecture could not guarantee an effective enhancement in the mechanical and physical properties and become the major technological barrier for practical applications. Fabrication of nanocarbon reinforced composites with special architecture such as laminate, alignment and network architecture has been reported to break through the limitations and improve the overall performance of the composites. In this review, we aim to: (1) systematically summarize the current preparation methods to achieve different types of architecture design in three classes of matrices (polymer, ceramic and metal); (2) analyze the mechanisms and the influence factors for the mechanical and physical properties (electrical and thermal conductivities) of the composites with different architecture types. Finally, we outline the main advances and outlooks for architecture design in nanocarbon reinforced composites.