Investigations on nonlinear optics are active, with the applications in the modulators, fiber lasers, optical sensors, etc. In this paper, we focus our attention on a system for the ultra-short pulses in an inhomogeneous multi-component nonlinear optical medium. Starting from the existing Lax pair and one-fold Darboux transformation (DT), we construct the N-fold DT of that system, which involves N distinct spectral parameters, where N is a positive integer. The N-fold generalized DT with one spectral parameter is obtained through resorting to the Taylor-series-expansion coefficients of a special solution to that Lax pair. Double-pole soliton solutions of that system are derived via that N-fold generalized DT with N=2. With the aid of the N-fold DT, an N-fold Darboux matrix is constructed, based on which the multi-pole soliton solutions in the determinant form with respect to the electromagnetic field E are determined. Graphically, we find that those double-pole soliton solutions are a kind of the bound-state soliton solutions which represent the elastic interactions between the two solitons. Effects of the coefficients in that system on the double-pole soliton are shown via choosing the trigonometric, linear and quadratic functions. Furthermore, we present the triple-pole soliton and quadruple-pole soliton with respect to E. Our results might be useful in understanding the ultra-short pulses in the nonlinear optical media.