Background Fatigue, inflammation, and physical activity (PA) are all independently associated with gait speed, but their directionality is not fully elucidated. Aims Evaluate the bidirectional associations amongst fatigue, inflammation, and PA on gait speed. Methods This cross sectional study included probands ( n  = 1280, aged 49–105) and offspring ( n  = 2772, aged 24–88) in the Long Life Family Study. We assessed gait speed, fatigue with the question “I could not get going”, inflammation using fasting interleukin-6 (IL-6) and high sensitivity C-reactive protein (CRP), and self-reported PA as walking frequency in the past two weeks. The two generations were examined separately using linear mixed modeling. Results Lower fatigue, lower IL-6, and greater PA were all associated with faster gait speed in both generations (all p  < 0.05); lower CRP was only associated with faster gait speed in the offspring. PA explained the association of fatigue and gait speed via a 16.1% (95% CI 9.7%, 26.7%) attenuation of the direct associations for the probands and 9.9% (95% CI 6.3%, 18.8%) in the offspring. In addition, IL-6 explained more of the association of fatigue and gait speed than the association between PA and gait speed, via a 14.9% (95% CI 9.2%, 23.4%) attenuation of the direct association in the offspring only. Discussion Results revealed a potential directionality from fatigue to IL-6 to PA that may lead to faster gait speed. Future work should examine these relationships longitudinally to establish temporality and causality. Conclusions Our findings support a signal that lowering fatigue and inflammation and increasing physical activity may delay functional decline.