► Planning poker is compared to the mean of individual estimates. ► Estimates provided by ten student teams and a group of experts are analyzed. ► Planning poker increases the over-optimism of inexperienced developers. ► The optimism bias diminishes or disappears by increasing the expertise. ► Planning poker can improve estimates of motivated experienced professionals. While most studies in psychology and forecasting stress the possible hazards of group processes when predicting effort and schedule, agile software development methods recommend the use of a group estimation technique called planning poker for estimating the size of user stories and developing release and iteration plans. It is assumed that the group discussion through planning poker helps in identifying activities that individual estimators could overlook, thus providing more accurate estimates and reducing the over-optimism that is typical for expert judgment-based methods. In spite of the widespread use of agile methods, there is little empirical evidence regarding the accuracy of planning poker estimates. In order to fill this gap a study was conducted requiring 13 student teams to develop a Web-based student records information system. All teams were given the same set of user stories which had to be implemented in three Sprints. Each team estimated the stories using planning poker and the estimates provided by each team member during the first round were averaged to obtain the statistical combination for further comparison. In the same way the stories were estimated by a group of experts. The study revealed that students’ estimates were over-optimistic and that planning poker additionally increased the over-optimism. On the other hand, the experts’ estimates obtained through planning poker were much closer to actual effort spent and tended to be more accurate than the statistical combination of their individual estimates. The results indicate that the optimism bias caused by group discussion diminishes or even disappears as the expertise of the people involved in the group estimation process increases.