This paper surveys the state of the art on prediction in MOOCs through a systematic literature review (SLR). The main objectives are: first, to identify the characteristics of the MOOCs used for ...prediction, second, to describe the prediction outcomes, third, to classify the prediction features, fourth, to determine the techniques used to predict the variables, and, fifth, to identify the metrics used to evaluate the predictive models. Results show there is strong interest in predicting dropouts in MOOCs. A variety of predictive models are used, though regression and support vector machines stand out. There is also wide variety in the choice of prediction features, but clickstream data about platform use stands out. Future research should focus on developing and applying predictive models that can be used in more heterogeneous contexts (in terms of platforms, thematic areas, and course durations), on predicting new outcomes and making connections among them (e.g., predicting learners’ expectancies), on enhancing the predictive power of current models by improving algorithms or adding novel higher-order features (e.g., efficiency, constancy, etc.).
The advancement of learning analytics has enabled the development of predictive models to forecast learners' behaviors and outcomes (e.g., performance). However, many of these models are only ...applicable to specific learning environments and it is usually difficult to know which factors influence prediction results, including the predictor variables as well as the type of prediction outcome. Knowing these factors would be relevant to generalize to other contexts, compare approaches, improve the predictive models and enhance the possible interventions. In this direction, this work aims to analyze how several factors can make an influence on the prediction of students' performance. These factors include the effect of previous grades, forum variables, variables related to exercises, clickstream data, course duration, type of assignments, data collection procedure, question format in an exam, and the prediction outcome (considering intermediate assignment grades, including the final exam, and the final grade). Results show that variables related to exercises are the best predictors, unlike variables about forum, which are useless. Clickstream data can be acceptable predictors when exercises are not available, but they do not add prediction power if variables related to exercises are present. Predictive power was also better for concept-oriented assignments and best models usually contained only the last interactions. In addition, results showed that multiple-choice questions were easier to predict than coding questions, and the final exam grade (actual knowledge at a specific moment) was harder to predict than the final grade (average knowledge in the long term), based on different assignments during the course.
MOOCs (Massive Open Online Courses) have usually high dropout rates. Many articles have proposed predictive models in order to early detect learners at risk to alleviate this issue. Nevertheless, ...existing models do not consider complex high-level variables, such as self-regulated learning (SRL) strategies, which can have an important effect on learners' success. In addition, predictions are often carried out in instructor-paced MOOCs, where contents are released gradually, but not in self-paced MOOCs, where all materials are available from the beginning and users can enroll at any time. For self-paced MOOCs, existing predictive models are limited in the way they deal with the flexibility offered by the course start date, which is learner dependent. Therefore, they need to be adapted so as to predict with little information short after each learner starts engaging with the MOOC. To solve these issues, this paper contributes with the study of how SRL strategies could be included in predictive models for self-paced MOOCs. Particularly, self-reported and event-based SRL strategies are evaluated and compared to measure their effect for dropout prediction. Also, the paper contributes with a new methodology to analyze self-paced MOOCs when carrying out a temporal analysis to discover how early prediction models can serve to detect learners at risk. Results of this article show that event-based SRL strategies show a very high predictive power, although variables related to learners' interactions with exercises are still the best predictors. That is, event-based SRL strategies can be useful to predict if e.g., variables related to learners' interactions with exercises are not available. Furthermore, results show that this methodology serves to achieve early powerful predictions from about 25 to 33% of the theoretical course duration. The proposed methodology presents a new approach to predict dropouts in self-paced MOOCs, considering complex variables that go beyond the classic trace-data directly captured by the MOOC platforms.
•Event-based self-regulated learning (SRL) strategies are good predictors of dropout.•Self-reported SRL strategies are useless to predict dropout.•A new approach is proposed for prediction in self-paced MOOCs.•It is possible to achieve powerful predictions from 25 to 33% of the course duration.
The learning process in a MOOC (Massive Open Online Course) can be improved from knowing in advance learners' grades on different assignments. This would be very useful to detect problems with enough ...time to take corrective measures. In this work, the aim is to analyse how different course scores can be predicted, what elements or variables affect the predictions and how much and in which way it is possible to anticipate scores. To do that, data from a MOOC about Java programming have been used. Results show the importance of indicators over the algorithms and that forum-related variables do not add power to predict grades, unlike previous scores. Furthermore, the type of task can vary the results. Regarding the anticipation, it was possible to use data from previous topics but with worse performance, although values were better than those obtained in the first seven days of the current topic.
Celotno besedilo
Dostopno za:
BFBNIB, DOBA, GIS, IJS, IZUM, KILJ, KISLJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
Learning analytics (LA) as a research field has grown rapidly over the last decade. However, adoption of LA is mostly found to be small in scale and isolated at the instructor level. This paper ...presents an exploratory study on institutional approaches to LA in European higher education and discusses prominent challenges that impede LA from reaching its potential. Based on a series of consultations with senior managers from 83 different higher education institutions in 24 European countries, we observe that LA is primarily perceived as a tool to enhance teaching and institutional management. As a result, teaching and support staff are found to be the main users of LA and the target audience of training support. In contrast, there is little evidence of active engagement with students or using LA to develop self-regulated learning skills. We highlight the importance of grounding LA in learning sciences and including students as a key stakeholder in the design and implementation of LA. This paper contributes to our understanding of the development of LA in European higher education and highlights areas to address in both practice and research.
•Teaching and support staff are the main users of learning analytics (LA).•Managers prioritise using LA to influence institutional and teaching decisions.•There is little evidence of active engagement with students.•LA is primarily implemented at small scales in European higher education.•Few institutions have a dedicated strategy, policy, or evaluation framework for LA.
Potential benefits of learning analytics (LA) for improving students’ performance, predicting students’ success, and enhancing teaching and learning practice have increasingly been recognized in ...higher education. However, the adoption of LA in higher education institutions (HEIs) to date remains sporadic and predominantly small in scale due to several socio-technical challenges. To better understand why HEIs struggle to scale LA adoption, it is needed to untangle adoption challenges and their related factors. This paper presents the findings of a study that sought to investigate the associations of adoption factors with challenges HEIs face in the adoption of LA and how these associations are compared among HEIs at different scopes of adoption. The study was based on a series of semi-structured interviews with senior managers in HEIs. The interview data were thematically analysed to identify the main challenges in LA adoption. The connections between challenges and other factors related to LA adoption were analysed using epistemic network analysis (ENA). From senior managers’ viewpoints, ethical issues of informed consent and resistance culture had the strongest links with challenges of learning analytic adoption in HEI; this was especially true for those institutions that had not adopted LA or who were in the initial phase of adoption (i.e., preparing for or partially implementing LA). By contrast, among HEIs that had fully adopted LA, the main challenges were found to be associated with centralized leadership, gaps in the analytic capabilities, external stakeholders, and evaluations of technology. Based on the results, we discuss implications for LA strategy that can be useful for institutions at various stages of LA adoption, from early stages of interest to the full adoption phase.
In education, several studies have tried to track student persistence (i.e., students’ ability to keep on working on the assigned tasks) using different definitions and self-reported data. However, ...self-reported metrics may be limited, and currently, online courses allow collecting many low-level events to analyze student behaviors based on logs and using learning analytics. These analyses can be used to provide personalized and adaptative feedback in Smart Learning Environments. In this line, this work proposes the analysis and measurement of two types of persistence based on students’ interactions in online courses: (1) local persistence (based on the attempts used to solve an exercise when the student answers it incorrectly), and (2) global persistence (based on overall course activity/completion). Results show that there are different students’ profiles based on local persistence, although medium local persistence stands out. Moreover, local persistence is highly affected by course context and it can vary throughout the course. Furthermore, local persistence does not necessarily relate to global persistence or engagement with videos, although it is related to students’ average grade. Finally, predictive analysis shows that local persistence is not a strong predictor of global persistence and performance, although it can add some value to the predictive models.
One of the characteristics of massive open online courses (MOOCs) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social ...learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners' behavior and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analyzing learners' interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This paper presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analyzing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments, and skill dimensions that can be extracted from forum data. We also introduce LAT∃S, a learning analytics tool for edX/Open edX related to the three dimensions (3S), which includes visualizations to guide the proposed methodology. We apply the 3S methodology and the tool to an MOOC on Java programming. Results showed, among others, the action-reaction effect produced when learners increase their participation after instructor's events. Moreover, a decrease of positive sentiments over time and before deadlines of open-ended assignments was also observed and that there were certain skills, which caused more troubles (e.g., arrays and loops). These results acknowledge the importance of using a learning analytics methodology to detect problems in MOOCs.
MOOCs (massive open online courses) have a built-in forum where learners can share experiences as well as ask questions and get answers. Nevertheless, the work of the learners in the MOOC forum is ...usually not taken into account when calculating their grade in the course, due to the difficulty of automating the calculation of that grade in a context with a very large number of learners. In some situations, discussion forums might even be the only available evidence to grade learners. In other situations, forum interactions could serve as a complement for calculating the grade in addition to traditional summative assessment activities. This paper proposes an algorithm to automatically calculate learners’ grades in the MOOC forum, considering both the quantitative dimension and the relevance in their contributions. In addition, the algorithm has been implemented within a web application, providing instructors with a visual and a numerical representation of the grade for each learner. An exploratory analysis is carried out to assess the algorithm and the tool with a MOOC on programming, obtaining a moderate positive correlation between the forum grades provided by the algorithm and the grades obtained through the summative assessment activities. Nevertheless, the complementary analysis conducted indicates that this correlation may not be enough to use the forum grades as predictors of the grades obtained through summative assessment activities.
PDF
