•The continuous wake model describes well the wake profile behind a wind turbine.•The expected wind farm power is expressed as a differentiable function.•SCP can be employed to efficiently optimize the layout of a large-scale wind farm.•The optimized wind farm layout increases the wind farm power efficiency by 7.3%. This paper describes an efficient method for optimizing the placement of wind turbines to maximize the expected wind farm power. In a wind farm, the energy production of the downstream wind turbines decreases due to reduced wind speed and increased level of turbulence caused by the wakes formed by the upstream wind turbines. As a result, the wake interference among wind turbines lower the overall power efficiency of the wind farm. To improve the overall efficiency of a wind farm, researchers have studied the wind farm layout optimization problem to find the placement locations of wind turbines that maximize the expected wind farm power. Most studies on wind farm layout optimization employ heuristic search-based optimization algorithms. In spite of their simplicity, optimization algorithms based on heuristic search are computationally expensive and have limitation in optimizing the locations of a large number of wind turbines since the computational time for the search tends to increase exponentially with increasing number of wind turbines. This study employs a mathematical optimization scheme to efficiently and effectively optimize the locations of a large number of wind turbines with respect to maximizing the wind farm power production. To formulate the mathematical optimization problem, we derive a continuous wake model and express the expected wind farm power as a continuous and smooth function in terms of the locations of the wind turbines. The constructed wind farm power function is then maximized using sequential convex programming (SCP) for the nonlinear mathematical problem. We show how SCP can be used to evaluate the efficiency of an existing wind farm and to optimize a wind farm layout consisting of 80 wind turbines.