Background: Fear-avoidance is one of the factors associated with chronic pain. However, it remains unclear whether the association between fear-avoidance and pain depends on sex. The present study aimed to investigate whether the association between fear-avoidance and pain intensity differed between men and women in chronic pain patients. Additionally, the potential confounding effect of affective experiences on the association between fear-avoidance and pain intensity was analyzed. Method: This cohort study included hospital referred chronic pain patients (n = 45). Short momentary assessment questions according to the experience sampling method (ESM) were used to repeatedly assess patients’ pain intensity, level of fear-avoidance and positive as well as negative affect during their daily life. Linear mixed-effects models were applied in the statistical analysis. Unadjusted and adjusted models were made, in which the latter corrected for statistically significant affective experiences and baseline variables, taking the Aikake Information Criterion into account to assess a better model of fit. Results: The results demonstrated an association between fear-avoidance and pain intensity that differed for men and women. In men (n = 13), no association between these variables was found (−0.04 (95% CI: −0.14, 0.06) with a p-value of 0.48), whereas in women (n = 32), an increase in fear-avoidance was associated with a (slight) increase in pain intensity (0.18 (95% CI 0.06, 0.30) with a p-value of 0.003). Affect did not confound the above-mentioned findings. Conclusion: Our data supports previous research highlighting the importance of sex differences in pain experience. These findings may be relevant for clinicians to consider more personalized (i.e., gender specific) pain management in chronic pain patients.