According to cognitive load theory, instructions can impose three types of cognitive load on the learner: intrinsic load, extraneous load, and germane load. Proper measurement of the different types ...of cognitive load can help us understand why the effectiveness and efficiency of learning environments may differ as a function of instructional formats and learner characteristics. In this article, we present a ten-item instrument for the measurement of the three types of cognitive load. Principal component analysis on data from a lecture in statistics for PhD students (
n
= 56) in psychology and health sciences revealed a three-component solution, consistent with the types of load that the different items were intended to measure. This solution was confirmed by a confirmatory factor analysis of data from three lectures in statistics for different cohorts of bachelor students in the social and health sciences (
n
s = 171, 136, and 148), and received further support from a randomized experiment with university freshmen in the health sciences (
n
= 58).
Context
Although work‐based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to ...validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold. In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings.
Methods
Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio‐cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under‐stood ‘in situ’. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act.
Discussion
We propose constructivist–interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these strategies generate trustworthy evidence that is needed to develop the validity argument in WBA, allowing for in‐depth and meaningful information about professional competence.
Discuss ideas arising from the article at ‘discuss’
Where do we stand now?
In the 30 years that have passed since The Edinburgh Declaration on Medical Education, we have made tremendous progress in research on fostering ‘self‐directed and independent ...study’ as propagated in this declaration, of which one prime example is research carried out on problem‐based learning. However, a large portion of medical education happens outside of classrooms, in authentic clinical contexts. Therefore, this article discusses recent developments in research regarding fostering active learning in clinical contexts.
Self‐regulated, lifelong learning in medical education
Clinical contexts are much more complex and flexible than classrooms, and therefore require a modified approach when fostering active learning. Recent efforts have been increasingly focused on understanding the more complex subject of supporting active learning in clinical contexts. One way of doing this is by using theory regarding self‐regulated learning (SRL), as well as situated learning, workplace affordances, self‐determination theory and achievement goal theory. Combining these different perspectives provides a holistic view of active learning in clinical contexts.
Entry to practice, vocational training and continuing professional development
Research on SRL in clinical contexts has mostly focused on the undergraduate setting, showing that active learning in clinical contexts requires not only proficiency in metacognition and SRL, but also in reactive, opportunistic learning. These studies have also made us aware of the large influence one's social environment has on SRL, the importance of professional relationships for learners, and the role of identity development in learning in clinical contexts. Additionally, research regarding postgraduate lifelong learning also highlights the importance of learners interacting about learning in clinical contexts, as well as the difficulties that clinical contexts may pose for lifelong learning. However, stimulating self‐regulated learning in undergraduate medical education may also make postgraduate lifelong learning easier for learners in clinical contexts.
In this article the authors review developments in supporting active learning in medical education with particular emphasis on translating the ideas from a variety of important theories to the clinical context.
Feedback in medical education has traditionally showcased techniques and skills of giving feedback, and models used in staff development have focused on feedback providers (teachers) not receivers ...(learners). More recent definitions have questioned this approach, arguing that the impact of feedback lies in learner acceptance and assimilation of feedback with improvement in practice and professional growth. Over the last decade, research findings have emphasized that feedback conversations are complex interpersonal interactions influenced by a multitude of sociocultural factors. However, feedback culture is a concept that is challenging to define, thus strategies to enhance culture are difficult to pin down. In this twelve tips paper, we have attempted to define elements that constitute a feedback culture from four different perspectives and describe distinct strategies that can be used to foster a learning culture with a growth mind-set.
Medical Education 2012:
46
: 192–200
Context: How learners interpret their clinical experiences to create meaningful learning has not been well studied. We explored experiences considered by doctors ...to be influential in their learning in order to better understand this process.
Methods: Using a grounded theory approach, we interviewed 22 academic doctors who had been in practice for ≤ 5 years. Participants were asked to reflect on experiences they considered to have been influential during their training. Constant comparative analysis for emerging themes was conducted iteratively with data collection.
Results: A model of clinical learning emerged in which the clinical work itself is central. As they observe and participate in clinical work, learners can attend to a variety of sources of information that facilitate the interpretation of the experience and the construction of knowledge from it. These ‘learning cues’ include feedback, role models, clinical outcomes, patient or family responses, and comparisons with peers. The integration of a cue depends on the learner’s judgement of its credibility. Certain cues, such as clinical outcomes or feedback from patients, are seen as innately credible, whereas other cues, particularly feedback from supervisors, are subjected to critical judgement.
Conclusions: Learners make complex judgements regarding the credibility of information about clinical performance. Credibility judgements influence the learning that arises from the clinical experience. Further understanding of how such judgements are made could guide educators in providing credible information to learners.
Context
In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet ...the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional ‘fix‐the‐problem’ approaches (i.e. either–or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes.
Methods
Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its ‘both–and’ approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment.
Discussion
Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its ‘both–and’ approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.
The authors argue that traditional ‘either‐or’ solutions to assessment problems need to be replaced with ‘both‐and’ strategies if we are to advance our efforts to transform assessment in the era of competency‐based education.
A history of assessment in medical education Schuwirth, Lambert W. T.; van der Vleuten, Cees P. M.
Advances in health sciences education,
12/2020, Letnik:
25, Številka:
5
Journal Article
Recenzirano
Odprti dostop
The way quality of assessment has been perceived and assured has changed considerably in the recent 5 decades. Originally, assessment was mainly seen as a measurement problem with the aim to tell ...people apart, the competent from the not competent. Logically, reproducibility or reliability and construct validity were seen as necessary and sufficient for assessment quality and the role of human judgement was minimised. Later, assessment moved back into the authentic workplace with various workplace-based assessment (WBA) methods. Although originally approached from the same measurement framework, WBA and other assessments gradually became assessment processes that included or embraced human judgement but based on good support and assessment expertise. Currently, assessment is treated as a whole system problem in which competence is evaluated from an integrated rather than a reductionist perspective. Current research therefore focuses on how to support and improve human judgement, how to triangulate assessment information meaningfully and how to construct fairness, credibility and defensibility from a systems perspective. But, given the rapid changes in society, education and healthcare, yet another evolution in our thinking about good assessment is likely to lurk around the corner.
Context
Single‐best‐answer questions (SBAQs) have been widely used to test knowledge because they are easy to mark and demonstrate high reliability. However, SBAQs have been criticised for being ...subject to cueing.
Objectives
We used a novel assessment tool that facilitates efficient marking of open‐ended very‐short‐answer questions (VSAQs). We compared VSAQs with SBAQs with regard to reliability, discrimination and student performance, and evaluated the acceptability of VSAQs.
Methods
Medical students were randomised to sit a 60‐question assessment administered in either VSAQ and then SBAQ format (Group 1, n = 155) or the reverse (Group 2, n = 144). The VSAQs were delivered on a tablet; responses were computer‐marked and subsequently reviewed by two examiners. The standard error of measurement (SEM) across the ability spectrum was estimated using item response theory.
Results
The review of machine‐marked questions took an average of 1 minute, 36 seconds per question for all students. The VSAQs had high reliability (alpha: 0.91), a significantly lower SEM than the SBAQs (p < 0.001) and higher mean item–total point biserial correlations (p < 0.001). The VSAQ scores were significantly lower than the SBAQ scores (p < 0.001). The difference in scores between VSAQs and SBAQs was attenuated in Group 2. Although 80.4% of students found the VSAQs more difficult, 69.2% found them more authentic.
Conclusions
The VSAQ format demonstrated high reliability and discrimination and items were perceived as more authentic. The SBAQ format was associated with significant cueing. The present results suggest the VSAQ format has a higher degree of validity.
The authors present a novel method of marking short‐answer questions and demonstrate high reliability, discrimination and acceptability
Assessment in the context of problem-based learning van der Vleuten, Cees P. M.; Schuwirth, Lambert W. T.
Advances in health sciences education : theory and practice,
12/2019, Letnik:
24, Številka:
5
Journal Article
Recenzirano
Odprti dostop
Arguably, constructive alignment has been the major challenge for assessment in the context of problem-based learning (PBL). PBL focuses on promoting abilities such as clinical reasoning, team skills ...and metacognition. PBL also aims to foster self-directed learning and deep learning as opposed to rote learning. This has incentivized researchers in assessment to find possible solutions. Originally, these solutions were sought in developing the right instruments to measure these PBL-related skills. The search for these instruments has been accelerated by the emergence of competency-based education. With competency-based education assessment moved away from purely standardized testing, relying more heavily on professional judgment of complex skills. Valuable lessons have been learned that are directly relevant for assessment in PBL. Later, solutions were sought in the development of new assessment strategies, initially again with individual instruments such as progress testing, but later through a more holistic approach to the assessment program as a whole. Programmatic assessment is such an integral approach to assessment. It focuses on optimizing learning through assessment, while at the same gathering rich information that can be used for rigorous decision-making about learner progression. Programmatic assessment comes very close to achieving the desired constructive alignment with PBL, but its wide adoption—just like PBL—will take many years ahead of us.
Context
Although feedback is widely considered essential to learning, its actual influence on learners is variable. Research on responsivity to feedback has tended to focus on individual rather than ...social or cultural influences on learning. In this study, we explored how feedback is handled within different professional cultures, and how the characteristics and values of a profession shape learners' responses to feedback.
Methods
Using a constructivist grounded theory approach, we conducted 12 focus groups and nine individual interviews (with a total of 50 participants) across three cultures of professional training in, respectively, music, teacher training and medicine. Constant comparative analysis for recurring themes was conducted iteratively.
Results
Each of the three professional cultures created a distinct context for learning that influenced how feedback was handled. Despite these contextual differences, credibility and constructiveness emerged as critical constants, identified by learners across cultures as essential for feedback to be perceived as meaningful. However, the definitions of credibility and constructiveness were distinct to each professional culture and the cultures varied considerably in how effectively they supported the occurrence of feedback with these critical characteristics.
Conclusions
Professions define credibility and constructiveness in culturally specific ways and create contexts for learning that may either facilitate or constrain the provision of meaningful feedback. Comparison with other professional cultures may offer strategies for creating a productive feedback culture within medical education.
Discuss ideas arising from the article at ‘discuss’
PDF
