In 2011, Fang et al. in 9 posed the following problem: Classify non-normal locally primitive Cayley graphs of finite simple groups of valency d, where eitherd≤20or d is a prime number. The only case for which the complete solution of this problem is known is of d=3. Except this, a lot of efforts have been made to attack this problem by considering the following problem: Characterize finite nonabelian simple groups which admit non-normal locally primitive Cayley graphs of certain valencyd≥4. Even for this problem, it was only solved for the cases when either d≤5 or d=7 and the vertex stabilizer is solvable. In this paper, we make crucial progress towards the above problems by completely solving the second problem for the case when d≥11 is a prime and the vertex stabilizer is solvable.