Robust environmental decision support tools are critical to maximise the ecological benefit of management decisions. However the models that underpin these rarely undergo rigorous evaluation. Here, we evaluated components of a scenario-based habitat suitability model, the Murray Flow Assessment Tool, by correlating model outputs against fish monitoring data collected since its development. Overall, we detected a low correlation between habitat suitability scores for fish and fish assemblages during low-flow conditions, including when lags in fish response to hydrological inputs were introduced. Scores specific to fish functional groups were also poorly correlated with data for those groups. Finally, model outcomes were highly sensitive to methods used to combine both individual indices and weightings for each component. Thus, we recommend using constant weightings, simple and consistent combination methods and reconsidering the number of fish functional groups as simplifications to this model and in the development of similar habitat suitability models elsewhere. •We evaluated a widely-used ecological model to predict fish assemblages.•Model scores were poorly correlated with fish data collected from the same basin.•Functional-group specific scores did not correspond with fish composition.•Scores were highly sensitive to weighting and preference-curve combination methods.•There is a need for ongoing model evaluation and refinement as new data are available.