Rainfall-runoff models must be calibrated and validated before they can be used for urban stormwater management. Manual calibration is very difficult and time-consuming due to the large number of model parameters that must be estimated concurrently. Automatic calibration offers as a promising alternative, ideally supporting a user-independent and time-efficient approach to model parameters estimation. In this article, we test the use of a state-of-the-art standard package (PEST, Parameter ESTimation, http://www.pesthomepage.org/ ) for the automatic calibration of a rainfall-runoff EPA-SWMM (Storm Water Management Model) model developed for a small suburban catchment. Results reported in the paper demonstrate that the performance of automatically calibrated models still depends on a number of user-dependent choices (the level of catchment discretization, the selection of significant parameters, the optimization techniques adopted). Through a systematic analysis of the results, we try to identify the guidelines for the effective use of automatic calibration procedures based on modeling assumptions and target of the analysis.