Since the 1960s, it has been a common practice worldwide to pursue a homogeneous distribution of reinforcements within a matrix material, discontinuous metal matrix composites (DMMCs) in particular. Taking an overview of the worldwide activities in DMMC research, despite many favourable attributes such as improved specific strength, stiffness and superior wear resistance, DMMCs with a homogeneous microstructure tend to exhibit a very low room temperature damage tolerance even with a highly ductile matrix material such as aluminium. In this review, a range of uniquely multi-scale hierarchical structures have been successfully designed and fabricated by tailoring reinforcement distribution for DMMCs in order to obtain superior performance. A variety of specific microstructures that were developed in Al, Mg, Cu, Fe, Co and TiAl matrices indicate that there must be adequate plastic regions among the reinforcements to blunt or deflect cracks if one wants to toughen DMMCs. Following this path, aided by theoretical analyses, the most recent success is the design and fabrication of a network distribution of in situ reinforcing TiB whiskers (TiBw) in titanium matrix composites (TMCs), where a tailored three-dimensional (3D) quasi-continuous network microstructure displays significant improvements in mechanical properties. This resolves the brittleness surrounding TMCs fabricated by powder metallurgy. It is the large reinforcement-lean regions that remarkably improve the composite's ductility by bearing strain, blunting the crack and decreasing the crack-propagation rate. The fracture, strengthening and toughening mechanisms are comprehensively elucidated in order to further understand the advantages of such an inhomogeneous microstructure, and to justify the development of novel techniques to produce such inhomogeneous microstructures. This approach opens up a new horizon of research and applications of DMMCs and can be easily extended to general multi-phase composites with enhanced physical and mechanical properties.