Infrared radiation emitted from an asteroid surface causes a torque that can significantly affect rotational state of the asteroid. The influence of small topographic features on this phenomenon, called the YORP effect, seems to be of utmost importance. In this work, we show that a lateral heat diffusion in boulders of suitable sizes leads to an emergence of a local YORP effect which magnitude is comparable to the YORP effect due to the global shape. We solve a three-dimensional heat diffusion equation in a boulder and its surroundings by the finite element method, using the freefem++ code. The contribution to the total torque is inferred from the computed temperature distribution. Our general approach allows us to compute the torque induced by a realistic irregular boulder. For an idealized boulder, our result is consistent with an existing one-dimensional model. We also estimated (and extrapolated) a size distribution of boulders on (25143) Itokawa from close-up images of its surface. We realized that topographic features on Itokawa can potentially induce a torque corresponding to a rotational acceleration of the order of 10−7 rad d−2 and can therefore explain the observed phase shift in light curves.