Animal welfare in pig production is frequently a topic of debate and is sensitive in nature. This debate is partly due to differences in values, forms, convictions, interests and knowledge among the stakeholders that constitute differences among their frames of reference with respect to pigs and their welfare. Differences in frames of reference by stakeholder groups are studied widely, but not specifically with respect to animal behaviour or welfare. We explored this phenomenon using a qualitative behaviour assessment (QBA). Participating stakeholders were classified into two expert groups consisting of pig farmers (N = 11) and animal scientists (N = 18) and a lay-group consisting of urban citizens (N = 15). The stakeholders were asked to observe the behaviour of a specific pig in each of the nine videos and to assign a score for each video using 21 predefined terms describing the mood, such as ‘happy’ or ‘irritated’. They were asked to complete two additional questionnaires to obtain information on their frames of reference. Results from the QBA showed that the pig farmers observed the behaviour of pigs more positively than the urban citizens and the animal scientists. This was evident from the consistently higher scores on the positive terms to assess pig behaviour. The questionnaires revealed that the farmers had a different frames of reference regarding pigs and different understanding of welfare, which might explain the differences in assessment. In a follow-up stakeholder workshop, which focussed on differences in observation, QBA showed to be an effective tool to stimulate mutual learning among stakeholders, which is necessary to find shared solutions.