Individuals with inflammatory bowel disease (IBD) often hold strong beliefs on how diet impacts their disease. This study aimed to compare dietary intakes between IBD patients and individuals from the general population and to assess associations of dietary factors with relapse in patients with IBD. Patients with longstanding Crohn's disease and ulcerative colitis (n = 165) filled out food frequency questionnaires, as did individuals participating in a Dutch population-based study (controls) (n = 1469). Multivariable regression analyses were used to assess differences in habitual dietary intake between IBD patients and controls, and to estimate associations of dietary factors in patients in remission with the development of disease relapse at follow-up. IBD patients had higher intakes of animal protein (3.50 g/d, 95% confidence interval (CI) 1.67–5.34) and carbohydrate (10.1 g/d, 95% CI 5.23–14.9) than controls and lower intakes of (unsaturated) fat (−3.53 g/d, 95% CI −5.57– −1.50), dietary fiber (−2.19 g/d, 95% CI −3.05– −1.32) and alcohol (−0.84 g/d, 95% CI −1.46– −0.22). This was explained by a higher consumption of carbonated beverages, meat and poultry and lower consumption of fruit, vegetables and dairy products (except cheese) by IBD patients compared with controls. Several dietary factors, particularly (saturated) fat and fiber, appeared to be associated with risk of relapse. IBD patients had higher dietary intakes of (animal) protein and carbohydrate and lower intakes of (unsaturated) fat, dietary fiber and alcohol compared with a general Dutch population. The results further underscore that dietary factors may have a role in disease course.