Significance It is widely recognized that studying the detailed anatomy of the human brain is of great importance for neuroscience and medicine. The principal means for achieving this goal is presently diffusion magnetic resonance imaging (dMRI) tractography, which uses the local diffusion of water throughout the brain to estimate the course of long-range anatomical projections. Such projections connect gray matter regions through axons that travel in the deep white matter. The present study combines dMRI tractography with histological analysis to investigate where in the brain this method succeeds and fails. We conclude that certain superficial white matter systems pose challenges for measuring cortical connections that must be overcome for accurate determination of detailed neuroanatomy in humans. In vivo tractography based on diffusion magnetic resonance imaging (dMRI) has opened new doors to study structure–function relationships in the human brain. Initially developed to map the trajectory of major white matter tracts, dMRI is used increasingly to infer long-range anatomical connections of the cortex. Because axonal projections originate and terminate in the gray matter but travel mainly through the deep white matter, the success of tractography hinges on the capacity to follow fibers across this transition. Here we demonstrate that the complex arrangement of white matter fibers residing just under the cortical sheet poses severe challenges for long-range tractography over roughly half of the brain. We investigate this issue by comparing dMRI from very-high-resolution ex vivo macaque brain specimens with histological analysis of the same tissue. Using probabilistic tracking from pure gray and white matter seeds, we found that ∼50% of the cortical surface was effectively inaccessible for long-range diffusion tracking because of dense white matter zones just beneath the infragranular layers of the cortex. Analysis of the corresponding myelin-stained sections revealed that these zones colocalized with dense and uniform sheets of axons running mostly parallel to the cortical surface, most often in sulcal regions but also in many gyral crowns. Tracer injection into the sulcal cortex demonstrated that at least some axonal fibers pass directly through these fiber systems. Current and future high-resolution dMRI studies of the human brain will need to develop methods to overcome the challenges posed by superficial white matter systems to determine long-range anatomical connections accurately.