The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. ...This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. ...This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
MRI is an objective and basic method of diagnosing a herniated disc. In terms of hernia size, most specialists decide on the tactics of further treatment (try conservative therapy or surgery). And ...certainly all specialists seeing in the dynamics of an increase in hernia size interpret this as an indication for the surgical intervention. In our study, we show that an increase in the size of a hernia and a simultaneous change in its signal characteristics may precede its further decrease.
To break the paradigm that increase of herniated disc on MRI is always an indication for surgical intervention.
Prospective observational study.
Dynamics of changes in the size of herniated disc and its signal chatacteristics in MRI images.
The study included 135 patients with an intervertebral extrusion present in the L4-5, L5-S1 segments aged 35–55years, 73 of them men and 62 women. In all patients, the intervertebral hernia was diagnosed within 3 months preceding the start of the study. The average extrusion size was 8.8 mm. All patients received gabapentin as an analgetic. An MRI was performed 3times with a 30-day interval. Assessment of the dynamic of patient improvement was performed using short programs: MRI: – T2 sagittal – TR=4,490 ms, TE=121 ms, dist. factor (slice gap) 0%, FOV=300 mm, slice thickness=3 mm; - T2 transversal - TR=7,530 ms, TE=124 ms, dist. factor (slice gap) 0%, FOV=180 mm, slice thickness=3 mm.
Patients were divided into two groups based on the character of the dynamic changes seen in the signal characteristics of disc hernias: (1) increase in signal (diffuse or focal) from the disc, with significant increase in the size of the hernia (2–5 mm), which can be caused by inflammatory edema and neovascularization. The increase in size could be regarded as a negative dynamic, but further observation within 90days after the first MRI showed a decrease in the size of the hernia (3-7 mm) and an improvement of the clinical symptoms in 89 patients; and (2) in 46 patients, the disc signal did not change, neither did the size of the hernia, which resulted in surgical intervention being indicated for herniated disc removal.
The active change in the structure and especially the size of the hernia in a relatively short time interval (1–2 months) should not serve as an absolute indication for surgical intervention, as our study has shown, in the majority of patients the hernia regresses over the next few months. This phenomenon requires further immunobiological studies. It is also necessary to systematize the enormous information obtained in MRI diagnostics (not only in size, but also in structure) into a single classification unified for all specialists dealing with the problem of low back pain, which will reduce the number of unreasonable surgical interventions.
Assessment and evaluation of trainees' clinical performance measures is needed to ensure safe, high-quality patient care. These measures also aid in the development of reflective, high-performing ...clinicians and hold graduate medical education (GME) accountable to the public. Although clinical performance measures hold great potential, challenges of defining, extracting, and measuring clinical performance in this way hinder their use for educational and quality improvement purposes. This article provides a way forward by identifying and articulating how clinical performance measures can be used to enhance GME by linking educational objectives with relevant clinical outcomes. The authors explore four key challenges: defining as well as measuring clinical performance measures, using electronic health record and clinical registry data to capture clinical performance, and bridging silos of medical education and health care quality improvement. The authors also propose solutions to showcase the value of clinical performance measures and conclude with a research and implementation agenda. Developing a common taxonomy of uniform specialty-specific clinical performance measures, linking these measures to large-scale GME databases, and applying both quantitative and qualitative methods to create a rich understanding of how GME affects quality of care and patient outcomes is important, the authors argue. The focus of this article is primarily GME, yet similar challenges and solutions will be applicable to other areas of medical and health professions education as well.
Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine ...whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs).
EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified.
274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay.
For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.
To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance.
In October ...2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessing physicians' professional performance published from inception to October 2016. They included studies reporting on the validity evidence of tools used to assess physicians' clinical, teaching, and research performance. Using Kane's validity framework, they conducted data extraction based on four inferences in the validity argument: scoring, generalization, extrapolation, and implications.
They included 46 articles on 15 tools assessing clinical performance and 72 articles on 38 tools assessing teaching performance. They found no studies on research performance tools. Only 12 of the tools (23%) gathered evidence on all four components of Kane's validity argument. Validity evidence focused mostly on generalization and extrapolation inferences. Scoring evidence showed mixed results. Evidence on implications was generally missing.
Based on the argument-based approach to validity, not all questionnaire-based tools seem to support their intended use. Evidence concerning implications of questionnaire-based tools is mostly lacking, thus weakening the argument to use these tools for formative and, especially, for summative assessments of physicians' clinical and teaching performance. More research on implications is needed to strengthen the argument and to provide support for decisions based on these tools, particularly for high-stakes, summative decisions. To meaningfully assess academic physicians in their tripartite role as doctor, teacher, and researcher, additional assessment tools are needed.
To investigate the association between learning climate and adverse perinatal and maternal outcomes in obstetrics-gynecology departments.
The authors analyzed 23,629 births and 103 learning climate ...evaluations from 16 nontertiary obstetrics-gynecology departments in the Netherlands in 2013. Multilevel logistic regressions were used to calculate the odds of adverse perinatal and maternal outcomes, by learning climate score tertile, adjusting for maternal and department characteristics. Adverse perinatal outcomes included fetal or early neonatal mortality, five-minute Apgar score < 7, or neonatal intensive care unit admission for ≥ 24 hours. Adverse maternal outcomes included postpartum hemorrhage and/or transfusion, death, uterine rupture, or third- or fourth-degree perineal laceration. Bias analyses were conducted to quantify the sensitivity of the results to uncontrolled confounding and selection bias.
Learning climate scores were significantly associated with increased odds of adverse perinatal outcomes (aOR 2.06, 95% CI 1.14-3.72). Compared with the lowest tertile, departments in the middle tertile had 46% greater odds of adverse perinatal outcomes (aOR 1.46, 95% CI 1.09-1.94); departments in the highest tertile had 69% greater odds (aOR 1.69, 95% CI 1.24-2.30). Learning climate was not associated with adverse maternal outcomes (middle vs. lowest tertile: OR 1.04, 95% CI 0.93-1.16; highest vs. lowest tertile: OR 0.98, 95% CI 0.88-1.10).
Learning climate was associated with significantly increased odds of adverse perinatal, but not maternal, outcomes. Research in similar clinical contexts is needed to replicate these findings and explore potential mechanisms behind these associations.
PDF
