A parallel direct-forcing fictitious domain method is employed to perform fully resolved numerical simulations of turbulent channel flow laden with finite-size particles. The effects of the particle-fluid density ratio on the turbulence modulation in the channel flow are investigated at the friction Reynolds number of 180, the particle volume fraction of 0.84%, and the particle-fluid density ratio ranging from 1 to 104.2. The results show that the variation of the flow drag with the particle-fluid density ratio is not monotonic, with a larger flow drag for the density ratio of 10.42, compared to those of unity and 104.2. A significant drag reduction by the particles is observed for large particle-fluid density ratios during the transient stage, but not at the statistically stationary stage. The intensity of particle velocity fluctuations generally decreases with increasing particle inertia, except that the particle streamwise root-mean-square velocity and streamwise-transverse velocity correlation in the near-wall region are largest at the density ratio of the order of 10. The averaged momentum equations are derived with the spatial averaging theorem and are used to analyze the mechanisms for the effects of the particles on the flow drag. The results indicate that the drag-reduction effect due to the decrease in the fluid Reynolds shear stress is counteracted by the drag-enhancement effect due to the increase in the total particle stress or the interphase drag force for the large particle-inertia case. The sum of the total Reynolds stress and particle inner stress contributions to the flow drag is largest at the density ratio of the order of 10, which is the reason for the largest flow drag at this density ratio. The interphase drag force obtained from the averaged momentum equation (the balance theory) is significantly smaller than (but agrees qualitatively with) that from the empirical drag formula based on the phase-averaged slip velocity for large density ratios. For the neutrally buoyant case, the balance theory predicts a positive interphase force on the particles arising from the negative gradient of the particle inner stress, which cannot be predicted by the drag formula based on the phase-averaged slip velocity. In addition, our results show that both particle collision and particle-turbulence interaction play roles in the formation of the inhomogeneous distribution of the particles at the density ratio of the order of 10.