Existing researches for studying the influences of mill's helix angle on peak cutting force were mainly carried out by using simulation or experimental means. Thus, the obtained conclusions were just empirical judgments or qualitative observations, which cannot be further used for optimally designing the helix angle of mills. This paper presents a theoretical method for the first time to reveal the working mechanism of peripheral milling tool's helix angle on peak cutting force through strict analytical formulation. To facilitate well understanding the effect of helix angle, variation tendency of peak cutting force versus helix angle is comprehensively studied by analyzing the formation principle of peak cutting force from the point view of both geometrical explanation and theoretical proof. Both infinitesimal element method and differential theory are used in the investigation procedure. It is proved that the peak value of cutting forces decreases with the increase of helix angle for a single engaged cutting edge. Combining this conclusion with the overlapping effect of multiple engaged cutting edges, optimal helix angle corresponding to the minimum of peak cutting forces is analytically derived to be the function of axial depth of cut, number of flutes and cutter diameter. Experimental verifications have been carried out to validate the proposed theory.