We prepare mixtures of ultracold CaF molecules and Rb atoms in a magnetic trap and study their inelastic collisions. When the atoms are prepared in the spin-stretched state and the molecules in the spin-stretched component of the first rotationally excited state, they collide inelastically with a rate coefficient k_{2}=(6.6±1.5)×10^{-11}  cm^{3}/s at temperatures near 100  μK. We attribute this to rotation-changing collisions. When the molecules are in the ground rotational state we see no inelastic loss and set an upper bound on the spin-relaxation rate coefficient of k_{2}<5.8×10^{-12}  cm^{3}/s with 95% confidence. We compare these measurements to the results of a single-channel loss model based on quantum defect theory. The comparison suggests a short-range loss parameter close to unity for rotationally excited molecules, but below 0.04 for molecules in the rotational ground state.