Predictive modelling of academic success and retention has been a key research theme in Learning Analytics. While the initial work on predictive modelling was focused on the development of general ...predictive models, portable across different learning settings, later studies demonstrated the drawbacks of not considering the specificities of course design and disciplinary context. This study builds on the methods and findings of related earlier studies to further explore factors predictive of learners' academic success in blended learning. In doing so, it differentiates itself by (i) relying on a larger and homogeneous course sample (15 courses, 50 course offerings in total), and (ii) considering both internal and external conditions as factors affecting the learning process. We apply mixed effect linear regression models, to examine: i) to what extent indicators of students' online learning behaviour can explain the variability in the final grades, and ii) to what extent that variability is attributable to the course and students' internal conditions, not captured by the logged data. Having examined different types of behaviour indicators (e.g., indicators of the overall activity level, those indicative of regularity of study, etc), we found little difference, if any, in their predictive power. Our results further indicate that a low proportion of variance is explained by the behaviour-based indicators, while a significant portion of variability stems from the learners' internal conditions. Hence, when variability in external conditions is largely controlled for (the same institution, discipline, and nominal pedagogical model), students’ internal state is the key predictor of their course performance.
•Low portability of trace-based student success predictors across homogenous courses.•Low predictive power of online learning behaviour indicators in blended courses.•Activity level and regularity of study indicators have comparable predictive power.•Students' internal state can explain a large portion of variance in the course grades.•In similar course settings, study success relates most to students' internal state.
In this paper, we propose a novel approach to the analysis of collaborative learning. The approach posits that different dimensions of collaborative learning emerging from social ties and content ...analysis of discourse can be modeled as networks. As such, the combination of social network analysis (SNA) and epistemic network analysis (ENA) analysis can detect information about a learner's enactment of what the literature on collaborative learning has described as a role: an ensemble of cognitive and social dimensions that is marked by interacting with the appropriate people about appropriate content. The proposed approach is named social epistemic network signature (SENS) and is defined as a combination of these two complementary network analytic techniques. The proposed SENS approach is examined on data produced in collaborative activities performed in a massive open online course (MOOC) delivered via a major MOOC platform. The results of a study conducted on a data set collected in a MOOC suggest SNA and ENA produce complementary results which can i) explain collaboration processes that shaped the creation of social ties and that were associated with different network roles; ii) describe differences between low and high performing groups of learners; and iii) show how combined properties derived from SNA and ENA predict academic performance.
•A network analytics approach for collaborative learning is proposed.•The approach combines social network analysis and epistemic network analysis.•The approach is validated with a dataset from a massive open online course.•Prediction of the structure of social ties and network roles with discourse is shown.•Prediction of performance (groups) with discourse is demonstrated.
Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes ...remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis of learning-related constructs used in the prediction and measurement of student engagement and learning outcome. Based on our literature review, we identify current gaps in the research, including a lack of solid frameworks to explain learning in open online setting. Finally, we put forward a novel framework suitable for open online contexts based on a well-established model of student engagement. Our model is intended to guide future work studying the association between contextual factors (i.e., demographic, classroom, and individual needs), student engagement (i.e., academic, behavioral, cognitive, and affective engagement metrics), and learning outcomes (i.e., academic, social, and affective). The proposed model affords further interstudy comparisons as well as comparative studies with more traditional education models.
This paper describes a study that looked at the effects of different technology-use profiles on educational experience within communities of inquiry, and how they are related to the students' levels ...of cognitive presence in asynchronous online discussions. Through clustering of students (N=81) in a graduate distance education engineering course, we identified six different profiles: 1) task-focused users, 2) content-focused no-users, 3) no-users, 4) highly intensive users, 5) content-focused intensive users, and 6) socially-focused intensive users. Identified profiles significantly differ in terms of their use of learning platform and their levels of cognitive presence, with large effect sizes of 0.54 and 0.19 multivariate η2, respectively. Given that several profiles are associated with higher levels of cognitive presence, our results suggest multiple ways for students to be successful within communities of inquiry. Our results also emphasize a need for a different instructional support and pedagogical interventions for different technology-use profiles.
•We investigated technology-use profiles using six offerings of a graduate level course with the total of 81 students.•Our MANOVA results revealed six technology-use profiles, associated with different levels of cognitive presence.•Building on the previous research we describe identified clusters and discuss the impact of our findings.
Data are fundamental to Learning Analytics (LA) research and practice. However, the ethical use of data, particularly in terms of respecting learners' privacy rights, is a potential barrier that ...could hinder the widespread adoption of LA in the education industry. Despite the policies and guidelines of privacy protection being available worldwide, this does not guarantee successful implementation in practice. It is necessary to develop practical approaches that would allow for the translation of the existing guidelines into practice. In this study, we examine an initial set of privacy-preserving mechanisms on a large-scale education dataset. The data utility is evaluated before and after privacy-preserving mechanisms are applied by fitting into commonly used LA models, providing an evaluation of the utility loss. We further explore the balance between preserving data privacy and maintaining data utility in LA. The results prove the compatibility between preserving learners' privacy and LA, providing a benchmark of utility loss to practitioners and researchers in the education sector. Our study reminds an imminent concern of data privacy and advocates that privacy preserving can and should be an integral part of the design of any LA technique.
Tools for Educational Data Mining: A Review Slater, Stefan; Joksimović, Srećko; Kovanovic, Vitomir ...
Journal of educational and behavioral statistics,
02/2017, Letnik:
42, Številka:
1
Journal Article
Recenzirano
Odprti dostop
In recent years, a wide array of tools have emerged for the purposes of conducting educational data mining (EDM) and/or learning analytics (LA) research. In this article, we hope to highlight some of ...the most widely used, most accessible, and most powerful tools available for the researcher interested in conducting EDM/LA research. We will highlight the utility that these tools have with respect to common data preprocessing and analysis steps in a typical research project as well as more descriptive information such as price point and user-friendliness. We will also highlight niche tools in the field, such as those used for Bayesian knowledge tracing (BKT), data visualization, text analysis, and social network analysis. Finally, we will discuss the importance of familiarizing oneself with multiple tools—a data analysis toolbox—for the practice of EDM/LA research.
Contemporary literature on online and distance education almost unequivocally argues for the importance of interactions in online learning settings. Nevertheless, the relationship between different ...types of interactions and learning outcomes is rather complex. Analyzing 204 offerings of 29 courses, over the period of six years, this study aimed at expanding the current understanding of the nature of this relationship. Specifically, with the use of trace data about interactions and utilizing the multilevel linear mixed modeling techniques, the study examined whether frequency and duration of student–student, student–instructor, student–system, and student–content interactions had an effect of learning outcomes, measured as final course grades. The findings show that the time spent on student–system interactions had a consistent and positive effect on the learning outcome, while the quantity of student–content interactions was negatively associated with the final course grades. The study also showed the importance of the educational level and the context of individual courses for the interaction types supported. Our findings further confirmed the potential of the use of trace data and learning analytics for studying learning and teaching in online settings. However, further research should account for various qualitative aspects of the interactions used while learning, different pedagogical/media features, as well as for the course design and delivery conditions in order to better explain the association between interaction types and the learning achievement. Finally, the results might imply the need for the development of the institutional and program-level strategies for learning and teaching that would promote effective pedagogical approaches to designing and guiding interactions in online and distance learning settings.
•We examined the relationship between interaction types and learning out-come.•The findings show significant positive effect of student–system interactions.•Student-content interactions were negatively associated with the learning outcome.•Educational level and course context are important for interaction types supported.
This paper describes a study that looked at the effects of different teaching presence approaches in communities of inquiry, and ways in which student–student online discussions with high levels of ...cognitive presence can be designed. Specifically, this paper proposes that high-levels of cognitive presence can be facilitated in online courses, based on the community of inquiry model, by building upon existing research in i) self-regulated learning through externally-facilitated regulation scaffolding and ii) computer-supported collaborative learning through role assignment. We conducted a quasi-experimental study in a fully-online course (N=82) using six offerings of the course. After performing a quantitative content analysis of online discussion transcripts, a multilevel linear modeling analysis showed the significant positive effects of both externally-facilitated regulation scaffolding and role assignment on the level of cognitive presence. Specifically, the results showed that externally-facilitated regulation scaffolding had a higher effect on cognitive presence than extrinsically induced motivation through grades. The results showed the effectiveness of role assignment to facilitate a high-level of cognitive presence. More importantly, the results showed a significant effect of the interaction between externally-facilitated regulation scaffolding and role assignment on cognitive presence. The paper concludes with a discussion of practical and theoretical implications.
•Externally-facilitated regulated (EFR) learning and role scripts for online discussions•Design-based study conducted in a fully-online master's level course•Multi-level linear modeling showed significant effects of EFR and role scripting.•Motivation needs to be complemented with EFR for high level of cognitive presence.•EFR can offer equitable opportunities for cognitive presence of different roles.
Research and development in learning analytics has established viable solutions for scaling personalised feedback to all students. However, questions remain regarding how such feedback is perceived, ...interpreted and acted upon by stakeholders. The present study reports on the analysis of focus group data from four courses to understand students' sensemaking of personalised, learning analytics-based feedback. Deductive analysis and epistemic network analysis were carried out to identify students' perceptions and associated emotional responses to the feedback. The results demonstrate that students mostly held positive perceptions of their feedback - especially in regard to quality, task-focus and instructor care. While emotions to feedback were often negative, students also noted this increased their motivation to invest more effort in the learning tasks. The results of the study provide further evidence of the interpersonal communication variables influencing students' proactive recipience of feedback. Importantly, the findings show that students were mostly actively engaged with their personalised feedback, and that the instructor-mediated approach to feedback based on learning analytics is aligned to the principles of effective feedback.
Student-facing visualisations have attracted increased attention with recent developments in data-driven tools to support individual and group work. Learning analytics dashboards (LADs), a ...data-enhanced feedback tool that allows students to make sense of their learning by providing insights into their learning behaviours represents one of the prominent examples of this trend. While these visualisation tools are increasingly used to study and enhance students' learning in academic contexts, current research is limited regarding the effects of the LADs in K-12 environments. There is a missed opportunity to empower teams and allow instructors and researchers to understand how teams use the LAD to regulate their learning. In this study, we developed a K-12 LAD for supporting students' collaborative work and evaluated with respect to students' perceived usefulness of the proposed LAD and the association between its use and course performance. The study followed a mixed-methods approach, combining quantitative analysis of log data from the dashboard and qualitative analysis of students’ perceptions using surveys and focus groups. Our results show that different roles within teams have distinguished engagement patterns with the LAD and that the tool improves the collaborative learning experience. We postulate that the implications of this study will aid future research work when investigating the behaviours of teams and optimising their learning using LADs.
•Learning analytics dashboard in K-12 collaborative learning promotes engagement and collaboration.•Team interactions with a dashboard can support instructors in assessing and improving collaborative learning behaviours.•Visuals illustrating workflow processes of teams fosters more engagement with the dashboard compared to traditional visuals.•Discovered behavioural differences between team members and leader engagement with the learning analytics dashboard.•K-12 teams perceived the learning analytics dashboard as useful in supporting team regulation in collaborative settings.
PDF
