UP - logo
E-resources
Peer reviewed Open access
  • Assessing the influence of ...
    Sedrakyan, Gayane; Poelmans, Stephan; Snoeck, Monique

    Information and software technology, 02/2017, Volume: 82
    Journal Article

    •The work extends previous research on simulation benefits for model comprehension.•Our previous work shows positive effects for comprehension of structural aspects.•The extension includes assessment of the effectiveness for behavioral aspects.•Findings show significant positive effect on understanding system behavior.•The findings suggest that simulation is preferable over manual model inspection. UML diagrams are the de facto standard for analysing, communicating and designing software systems, as well as automated code generation. However there is a certain degree of difficulty in understanding a system represented by means of UML diagrams. Our previous research demonstrates a significant improvement in understanding the structural aspects of a system represented as a UML class diagram when using a feedback-inclusive prototype of a model. This paper extends our previous work with an empirical validation study for the effectiveness of the feedback-inclusive rapid prototyping (FIRP) method, on the comprehension of system dynamics represented as multiple interacting UML statecharts. Because models often combine structural and behavioural views that are highly intertwined, we additionally evaluate the effectiveness of the proposed method with respect to comprehension of the between-view consistency. The FIRP environment was built following the principles of Design Science Research in Information Systems. This study targets the empirical validation of the effectiveness of the proposed technique using an experimental study method. Two experiments were conducted with the participation of 65 final-year master students in the context of different modelling courses from different study programs at KU Leuven using two two-group factorial experimental designs. The effectiveness of the FIRP method was measured by comparing students’ performance between the cycles with and without the use of the method, using the understandability (comprehension test results) as the dependent variable and the use of FIRP as the independent variable. Effects from unknown variables were neutralized by means of randomized group compositions. The effectiveness of FIRP was additionally assessed with respect to personal characteristics (age, gender, previous knowledge, self-efficacy) and user acceptance (perceived ease of use, perceived utility, preference, satisfaction). The findings reveal a significant positive impact of the use of the prototyping technique on students’ comprehension of system dynamics represented as multiple interacting statecharts. The findings provide empirical support for the advantage of the use of FIRP over manual inspection of interacting statecharts. The findings also suggest that the method is suitable for training system's analysis and modelling skills when UML statecharts are involved.