Protogalaxies forming in low-mass dark matter haloes are thought to provide the majority of ionizing photons needed to reionize the Universe, due to their high escape fractions of ionizing photons. We study how the escape fraction in high-redshift galaxies relates to the physical properties of the halo in which the galaxies form, by computing escape fractions in more than 75 000 haloes between redshifts 27 and 6 that were extracted from the First Billion Years project, high-resolution cosmological hydrodynamical simulations of galaxy formation. We find that the main constraint on the escape fraction is the gas column density in a radius of 10 pc around the stellar populations, causing a strong mass dependence of the escape fraction. The lower potential well in haloes with M 200 ≲ 108 M⊙ results in low column densities that can be penetrated by radiation from young stars (age <5 Myr). In haloes with M 200 ≳ 108 M⊙ supernova feedback is important, but only ∼30 per cent of the haloes in this mass range have an escape fraction higher than 1 per cent. We find a large range of escape fractions in haloes with similar properties, caused by different distributions of the dense gas in the halo. This makes it very hard to predict the escape fraction on the basis of halo properties and results in a highly anisotropic escape fraction. The strong mass dependence, the large spread and the large anisotropy of the escape fraction may strongly affect the topology of reionization and is something current models of cosmic reionization should strive to take into account.