Transshipments, monitored movements of material at the same echelon of a supply chain, represent an effective pooling mechanism. Earlier papers dealing with transshipments either do not incorporate replenishment lead times into their analysis, or only provide a heuristic algorithm where optimality cannot be guaranteed beyond settings with two locations. This paper uses infinitesimal perturbation analysis by combining with a stochastic approximation method to examine the multi-location transshipment problem with positive replenishment lead times. It demonstrates the computation of optimal base stock quantities through sample path optimization. From a methodological perspective, this paper deploys a duality-based gradient computation method to improve computational efficiency. From an application perspective, it solves transshipment problems with non-negligible replenishment lead times. A numerical study illustrates the performance of the proposed approach.