Rainfall‐runoff models are often deficient under changing climatic conditions, yet almost no recent studies propose new or improved model structures, instead focusing on model intercomparison, input sensitivity, and/or quantification of uncertainty. This paucity of progress in model development is (in part) due to the difficulty of distinguishing which cases of model failure are truly caused by structural inadequacy. Here we propose a new framework to diagnose the salient cause of poor model performance in changing climate conditions, be it structural inadequacy, poor parameterization, or data errors. The framework can be applied to a single catchment, although larger samples of catchments are helpful to generalize and/or cross‐check results. To generate a diagnosis, multiple historic periods with contrasting climate are defined, and the limits of model robustness and flexibility are explored over each period separately and for all periods together. Numerous data‐based checks also supplement the results. Using a case study catchment from Australia, improved inference of structural failure and clearer evaluation of model structural improvements are demonstrated. This framework enables future studies to (i) identify cases where poor simulations are due to poor calibration methods or data errors, remediating these cases without recourse to structural changes; and (ii) use the remaining cases to gain greater clarity into what structural changes are needed to improve model performance in changing climate. Plain Language Summary Rainfall runoff models are tools used by hydrologists in climate change assessments to estimate how future streamflow might change in response to a given (often hypothetical) climate scenario. For example, suppose we can assume that rainfall in a particular location is going to reduce by 20% in the future. Does this mean that streamflow will also reduce by 20%? Or will it be 10% less or 40% less? Although rainfall runoff models are among the best tools available, they are often not very good at answering this question. When tested on historical multiyear droughts, they often perform poorly, and we are unsure why. One problem is that when a model fails in this task, it is difficult to know what went wrong. Perhaps there was a problem with the data, since environmental monitoring is often subject to large errors. Perhaps the problem lay not with the model itself but with the way it was trained, or calibrated, to the data. Lastly, perhaps the model itself—its mathematical equations—need to be changed. To improve our estimates, we need a method to test which cause is behind the model failure; otherwise, we might make changes where none are warranted. This paper proposes such a method, in the form of a multistep framework that can isolate the causes of model failures. By ensuring that our attention is focused in the correct direction, this framework will help us to understand and make better estimates of how river flow will be altered by a changing climate. Key Points Diagnosing model deficiency is difficult as there are many possible causes of poor simulations Split sample failure does not mean model structural inadequacy—further tests are required This framework diagnoses the cause of poor performance and prioritizes remedial action