The authors of this article reviewed the methodology of three common consensus methods: nominal group process, consensus development panels, and the Delphi technique. The authors set out to determine ...how a majority of researchers are conducting these studies, how they are analyzing results, and subsequently the manner in which they are reporting their findings. The authors conclude with a set of guidelines and suggestions designed to aid researchers who choose to use the consensus methodology in their work.Overall, researchers need to describe their inclusion criteria. In addition to this, on the basis of the current literature the authors found that a panel size of 5 to 11 members was most beneficial across all consensus methods described. Lastly, the authors agreed that the statistical analyses done in consensus method studies should be as rigorous as possible and that the predetermined definition of consensus must be included in the ultimate manuscript. More specific recommendations are given for each of the three consensus methods described in the article.
As a result of the coronavirus pandemic, the feasibility of holding secure closed-book examinations in medical education is compromised. In this Personal View, we compare the underlying reasoning for ...using open-book and closed-book exams. We rethink the role of open-book assessment and offer ways in which we believe they can complement closed-book exams. We highlight the gap in research, highlight future directions, and call on medical educators to seize our current golden opportunity to explore the impact of open-book exams - on their own or combined to closed-book tests, as a blended approach - on learners, educators, and licensing bodies.
Robust response rates are essential for effective survey-based strategies. Researchers can improve survey validity by addressing both response rates and nonresponse bias. In this AMEE Guide, we ...explain response rate calculations and discuss methods for improving response rates to surveys as a whole (unit nonresponse) and to questions within a survey (item nonresponse). Finally, we introduce the concept of nonresponse bias and provide simple methods to measure it.
An unprecedented rise in health professions education (HPE) research has led to increasing attention and interest in knowledge syntheses. There are many different types of knowledge syntheses in ...common use, including systematic reviews, meta-ethnography, rapid reviews, narrative reviews, and realist reviews. In this Perspective, the authors examine the nature, purpose, value, and appropriate use of one particular method: scoping reviews. Scoping reviews are iterative and flexible and can serve multiple main purposes: to examine the extent, range, and nature of research activity in a given field; to determine the value and appropriateness of undertaking a full systematic review; to summarize and disseminate research findings; and to identify research gaps in the existing literature. Despite the advantages of this methodology, there are concerns that it is a less rigorous and defensible means to synthesize HPE literature. Drawing from published research and from their collective experience with this methodology, the authors present a brief description of scoping reviews, explore the advantages and disadvantages of scoping reviews in the context of HPE, and offer lessons learned and suggestions for colleagues who are considering conducting scoping reviews. Examples of published scoping reviews are provided to illustrate the steps involved in the methodology.
Objectives
This paper reviews why tensions between service and education persist and highlights that this is an area of medical education research (MER) that, to date, lacks a robust body of ...theory‐driven research. After carrying out a review of the literature on service–education tensions in medical education and training, we turn to consider how theory can help provide new insights into service–education tensions.
Methods
We conducted a search of the literature on service–education tensions since 1998 to examine the use of theory in studies on this topic.
Results
We identified 44 out of 603 relevant papers. Their focus fell into four broad categories: time residents spent on ‘service’ and ‘education’; perceptions of the balance between service and education; considerations of how best to define service and education, and the impact of structural and systems changes on education/training. Of the papers reporting primary research, the dominant methodology was the bespoke survey. Rarely were the precise natures of tensions or how different factors interact to cause tensions examined in detail.
Discussion
Through discussion and reflection, we then agreed on the applicability of four sociocultural theories for illuminating some examples of service–education tensions. We present four sociocultural theories: Holland's figured worlds, Kemmis et al.'s practice architectures, Lave and Wenger's situated learning and Engeström's cultural‐historical activity theory (CHAT or AT). We describe each and then briefly illustrate how each theory can support new ways of thinking and potential directions for research focusing on education–service tensions.
Conclusions
The use of theory in research studies will not resolve service–education tensions. However, what theory can do is illuminate and magnify different aspects of service–education tensions, to generate new insight and knowledge that can then be used to inform future research and changes in practice.
How might theory provide new insights into the service vs education tension? Cleland and Durning examine four specific possibilities.
Continuing professional development (CPD) programs, which aim to enhance health professionals' practice and improve patient outcomes, are offered to practitioners across the spectrum of health ...professions through both formal and informal learning activities. Various knowledge syntheses (or reviews) have attempted to summarize the CPD literature; however, these have primarily focused on continuing medical education or formal learning activities. Through this scoping review, the authors seek to answer the question, What is the current landscape of knowledge syntheses focused on the impact of CPD on health professionals' performance, defined as behavior change and/or patient outcomes?
In September 2019, the authors searched PubMed, Embase, CINAHL, Scopus, ERIC, and PsycINFO for knowledge syntheses published between 2008 and 2019 that focused on independently practicing health professionals and reported outcomes at Kirkpatrick's level 3 and/or 4.
Of the 7,157 citations retrieved from databases, 63 satisfied the inclusion criteria. Of these 63 syntheses, 38 (60%) included multicomponent approaches, and 29 (46%) incorporated eLearning interventions-either standalone or in combination with other interventions. While a majority of syntheses (n = 42 67%) reported outcomes affecting health care practitioners' behavior change and/or patient outcomes, most of the findings reported at Kirkpatrick level 4 were not statistically significant. Ten of the syntheses (16%) mentioned the cost of interventions though this was not their primary focus.
Across health professions, CPD is an umbrella term incorporating formal and informal approaches in a multicomponent approach. eLearning is increasing in popularity but remains an emerging technology. Several of the knowledge syntheses highlighted concerns regarding both the financial and human costs of CPD offerings, and such costs are being increasingly addressed in the CPD literature.
An evidence-based approach to assessment is critical for ensuring the development of clinical reasoning (CR) competence. The wide array of CR assessment methods creates challenges for selecting ...assessments fit for the purpose; thus, a synthesis of the current evidence is needed to guide practice. A scoping review was performed to explore the existing menu of CR assessments.
Multiple databases were searched from their inception to 2016 following PRISMA guidelines. Articles of all study design types were included if they studied a CR assessment method. The articles were sorted by assessment methods and reviewed by pairs of authors. Extracted data were used to construct descriptive appendixes, summarizing each method, including common stimuli, response formats, scoring, typical uses, validity considerations, feasibility issues, advantages, and disadvantages.
A total of 377 articles were included in the final synthesis. The articles broadly fell into three categories: non-workplace-based assessments (e.g., multiple-choice questions, extended matching questions, key feature examinations, script concordance tests); assessments in simulated clinical environments (objective structured clinical examinations and technology-enhanced simulation); and workplace-based assessments (e.g., direct observations, global assessments, oral case presentations, written notes). Validity considerations, feasibility issues, advantages, and disadvantages differed by method.
There are numerous assessment methods that align with different components of the complex construct of CR. Ensuring competency requires the development of programs of assessment that address all components of CR. Such programs are ideally constructed of complementary assessment methods to account for each method's validity and feasibility issues, advantages, and disadvantages.
The American Medical Association has called for telehealth to become a core competency of medical students. Studies indicate that a principal reason physicians do not practice telehealth is lack of ...training, yet patient interest in and satisfaction with telehealth are high. No comprehensive U.S. undergraduate medical education curriculum teaching telehealth principles has been published.
In February 2018, the Uniformed Services University of the Health Sciences (USU) provided an innovative telehealth training experience for third-year medical students. USU led an interinstitutional, interprofessional learner-centered course including six segments: (1) multiple-choice pretest; (2) asynchronous lectures covering telehealth history, applications, ethics, safety, military uses, etiquette, and patient considerations; (3) in-person interactive telehealth instruction including patient selection, current uses, and risk management; (4) faculty-supervised mock patient telehealth encounters; (5) hands-on diagnosis and advanced surgical procedures using telehealth equipment; and (6) multiple-choice posttest.
This course was piloted with 149 third-year medical students. Students' improvement in telehealth knowledge was demonstrated through (1) 10.1% average improvement between pre- and posttest scores, (2) completion of competency-based checklists, and (3) postcourse preceptor and student feedback. Faculty feedback indicated that technology use was novel and effective based on student input. Faculty noted that students enjoyed engaging via videoconference. Of participating medical students, 119 (80%) indicated future plans to practice telehealth; several requested to be part of future telehealth courses.
Telehealth will be integrated into clinical rotations in collaboration with other institutions. As the telehealth curriculum is taught at other institutions, lessons learned will inform enhancements at USU.
PDF
