Despite wide support for patient involvement in health technology assessments (HTA), determining meaningful engagement is complex. This article explores experiences and perceptions among patient ...groups participating in the Canadian Agency for Drugs and Technologies in Health (CADTH)’s pan-Canadian Oncology Drug Review (pCODR) process.
We created a qualitative interview study comprising 22 semi-structured telephone interviews with individuals representing 21 different patient groups registered with the pCODR process. The analysis used a qualitative descriptive approach employing techniques from grounded theory.
Patient groups view the ability to make submissions to the pCODR process as a meaningful activity closely aligned with organizational priorities. Concurrently, they face substantial resource challenges to prepare submissions, including high opportunity costs and difficulty accessing needed literature and finding relevant patients. Although patient groups felt that CADTH is committed to transparency, they expressed considerable uncertainty around the direct impact of their submissions and desired additional avenues for engagement.
This study suggests a strong commitment by patient groups to participate in the pCODR process despite uncertainty about how their submissions are used to inform HTA recommendations. Identifying opportunities to provide both financial and nonfinancial resources to patient groups is crucial to encouraging and supporting their meaningful participation in HTA processes.
•Although there is widespread support for patient engagement in health technology assessments (HTA), there is limited evidence about why and how patient input is important and a limited understanding of the experiences and perspectives of participating patient groups.•Our study identifies a deep commitment from patient groups to participate in the pan-Canadian Oncology Drug Review (pCODR) process. They especially value the opportunity to provide information not available in trials, such as patient/caregiver priorities, but their participation is limited by substantial challenges including significant opportunity costs (eg, displacing patient support activities), difficulty accessing literature, or trouble finding relevant patients. Patient groups are uncertain regarding the impact of their submissions and expressed a desire for additional avenues of engagement.•Our work makes a substantial contribution to the literature by providing detailed, specific examples of what patient groups experienced as meaningful and what they experienced as challenges when making a pCODR submission. Coupled with our earlier examination of stakeholder perspectives, we believe our work has practical implications for better supporting patient engagement in the drug review process as well as within HTA more generally.
Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured ...approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.
Canadian provinces routinely collect patient-level data for administrative purposes. These real-world data (RWD) can be used to generate real-world evidence (RWE) to inform clinical care and ...healthcare policy. The CanREValue Collaboration is developing a framework for the use of RWE in cancer drug funding decisions. A Data Working Group (WG) was established to identify data assets across Canada for generating RWE of oncology drugs. The mapping exercise was conducted using an iterative scan with informant surveys and teleconference. Data experts from ten provinces convened for a total of three teleconferences and two in-person meetings from March 2018 to September 2019. Following each meeting, surveys were developed and shared with the data experts which focused on identifying databases and data elements, as well as a feasibility assessment of conducting RWE studies using existing data elements and resources. Survey responses were compiled into an interim data report, which was used for public stakeholder consultation. The feedback from the public consultation was used to update the interim data report. We found that databases required to conduct real-world studies are often held by multiple different data custodians. Ninety-seven databases were identified across Canada. Provinces held on average 9 distinct databases (range: 8-11). An Essential RWD Table was compiled that contains data elements that are necessary, at a minimal, to conduct an RWE study. An Expanded RWD Table that contains a more comprehensive list of potentially relevant data elements was also compiled and the availabilities of these data elements were mapped. While most provinces have data on patient demographics (e.g., age, sex) and cancer-related variables (e.g., morphology, topography), the availability and linkability of data on cancer treatment, clinical characteristics (e.g., morphology and topography), and drug costs vary among provinces. Based on current resources, data availability, and access processes, data experts in most provinces noted that more than 12 months would be required to complete an RWE study. The CanREValue Collaboration's Data WG identified key data holdings, access considerations, as well as gaps in oncology treatment-specific data. This data catalogue can be used to facilitate future oncology-specific RWE analyses across Canada.
Second‐line ipilimumab has been publicly funded in Ontario for metastatic melanoma (MM) since September 2012. We examined real‐world toxicity of second‐line ipilimumab compared to standard ...second‐line treatments prior to funding. MM patients who received systemic treatment from April 2005 to March 2015 were included. Patients receiving second‐line ipilimumab after September 2012 were considered as cases, and those who received second‐line treatment prior to the funding date were included as historical controls. Outcomes assessed include treatment‐related mortality, any‐cause hospital visits, ipilimumab‐related hospital visits and specialist visits (eg, endocrinologists, ophthalmologists, gastroenterologists, rheumatologists and respirologists), which were captured from up to 30 and/or 90 days after end of second‐line treatment. Inverse probability of treatment weighting was used to adjust for baseline differences between groups. Odds ratios (ORs) from logistic regressions and rate ratios (RRs) from rate regressions were used to assess differences between groups. We identified 329 MM patients who received second‐line treatments (ipilimumab: 189; controls: 140). Ipilimumab was associated greater any‐cause (60.1% vs 45.7%; OR = 1.81; P value = .019) and ipilimumab‐related (47.2% vs 31.9%; OR = 1.91; P value = .011) hospital visits. Adjusting for different follow‐up days, ipilimumab was associated with higher rates of all‐cause (RR = 1.56 95%CI: 1.12‐2.16), and ipilimumab‐related (RR = 2.18 95% CI: 1.45‐3.27) hospital visits. Patients receiving ipilimumab were more likely to visit specialist involved in immunotherapy toxicity management (23.5% vs 13.7%; P value = .04). Compared to historical second‐line treatments, second‐line ipilimumab was associated with more health service utilization (specifically hospital visits and specialist visits), suggestive of potentially increased toxicity in the real world.
What's new?
Ipilimumab improves overall survival in metastatic melanoma. But how toxic is it in real‐world use? In this population‐based, comparative study, the authors found that patients receiving second‐line ipilimumab required more hospital (all‐cause and drug‐related) and specialist visits compared to historical controls. These findings suggest that ipilimumab is associated with greater toxicity and resource utilization than conventional treatments. Given that patients in actual clinical practice may have additional comorbidities compared to patients selected for clinical trials, it is important to further examine toxicity in a real‐world setting in order to better understand the risks.
Emergency medical response is provided by volunteer emergency medical services (EMS) professionals in many parts of the United States. However, little is known about those who serve as volunteer EMS ...professionals, especially as their main EMS job. Our objective was to compare the characteristics of nationally-certified volunteer versus paid EMS professionals in the U.S.
We conducted a cross-sectional evaluation of EMS professionals who recertified their National EMS Certification between October 1, 2017-March 31, 2018 or October 1, 2018-March 31, 2019. Data were obtained from 2 cycles of the biennial National EMS Certification recertification application. We included currently working, nonmilitary EMS professionals aged 18-85 years at the emergency medical technician (EMT) or higher certification level. Volunteer was defined as receiving nominal or no compensation for the provision of EMS services at an agency as the main EMS job. Comparisons were made, accounting for the large sample size, to determine notable differences between volunteer and paid EMS professionals.
Of the 154,229 EMS professionals that met inclusion criteria (response rate = 86%), 13% volunteered at their main EMS job. More volunteer than paid EMS professionals were female (38% vs. 22%). There was also a significant difference in certification level between volunteer and paid (EMT: 82% vs. 46%). Three-quarters (74%) of volunteer EMS professionals reported working in rural communities compared to 30% of paid. The states with the largest proportion of volunteers to any currently working EMS professionals who recertified were Vermont (47%) and North Dakota (45%).
Approximately 13% of the EMS workforce held a primary job in EMS as a volunteer, and these volunteer EMS professionals differed from their paid counterparts - e.g., volunteers were more likely to be women, have EMT certification, and work in rural areas. We encourage further research to understand motivations for volunteering in EMS.
The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) ...questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.
The COVID-19 pandemic has a significant impact on cancer patients and the delivery of cancer care. To allow clinicians to adapt treatment plans for patients, Ontario Health (Cancer Care Ontario) ...issued a series of interim funding measures for the province's New Drug Funding Program (NDFP), which covers the cost of most hospital-delivered cancer drugs. To assess the utility of the measures and the need for their continuation, we conducted an online survey of Ontario oncology clinicians. The survey was open 3-25 September 2020 and generated 105 responses. Between April and June 2020, 46% of respondents changed treatment plans for more than 25% of their cancer patients due to the pandemic. Clinicians report broad use of interim funding measures. The most frequently reported strategies used were treatment breaks for stable patients (62%), extending dosing intervals (59%), and deferring routine imaging (56%). Most clinicians anticipate continuing to use these interim funding measures in the coming months. The survey showed that adapting cancer drug funding policies has supported clinical care in Ontario during the pandemic.
The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration was established to develop a framework for generating and using real-world evidence (RWE) to inform the ...reassessment of cancer drugs following initial health technology assessment (HTA). The Reassessment and Uptake Working Group (RWG) is one of the five established CanREValue Working Groups. The RWG aims to develop considerations for incorporating RWE for HTA reassessment and strategies for using RWE to reassess drug funding decisions. Between February 2018 and December 2019, the RWG attended four teleconferences (with follow-up surveys) and two in-person meetings to discuss recommendations for the development of a reassessment process and potential barriers and facilitators. Modified Delphi methods were used to gather input. A draft report of recommendations (to December 2018) was shared for public consultation (December 2019 to January 2020). Initial considerations for developing a reassessment process were proposed. Specifically, reassessment can be initiated by diverse stakeholders, including decision makers from public drug plans or industry stakeholders. The reassessment process should be modelled after existing deliberation and recommendation frameworks used by HTA agencies. Proposed reassessment outcome categories include maintaining status quo, revisiting funding criteria, renegotiating price, or disinvesting. Overall, these initial considerations will serve as the basis for future advancements by the Collaboration.
Background
The efficacy‐effectiveness gap between randomized trial and real‐world evidence regarding the clinical benefit of ipilimumab for metastatic melanoma (MM) has been well characterized by ...previous literature, consistent with initial concerns raised by health technology assessment agencies (HTAs). As these differences can significantly impact cost‐effectiveness, it is critical to assess the real‐world cost‐effectiveness of second‐line ipilimumab versus non‐ipilimumab treatments for MM.
Methods
This was a population‐based retrospective cohort study of patients who received second‐line non‐ipilimumab therapies between 2008 and 2012 versus ipilimumab treatment between 2012 and 2015 (after public reimbursement) for MM in Ontario. Using a 5‐year time horizon, censor‐adjusted and discounted (1.5%) costs (from the public payer's perspective in Canadian dollars) and effectiveness were used to calculate incremental cost‐effectiveness ratios (ICERs) in life‐years gained (LYGs) and quality‐adjusted life years (QALYs), with bootstrapping to capture uncertainty. Varying the discount rate and reducing the price of ipilimumab were done as sensitivity analyses.
Results
In total, 329 MM were identified (Treated: 189; Controls: 140). Ipilimumab was associated with an incremental effectiveness of 0.59 LYG, incremental cost of $91,233, and ICER of $153,778/LYG. ICERs were not sensitive to discounting rate. Adjusting for quality of life using utility weights resulted in an ICER of $225,885/QALY, confirming the original HTA estimate prior to public reimbursement. Reducing the price of ipilimumab by 100% resulted in an ICER of $111,728/QALY.
Conclusion
Despite its clinical benefit, ipilimumab as second‐line monotherapy for MM patients is not cost‐effective in the real world as projected by HTA under conventional willingness‐to‐pay thresholds.
In light of concerns regarding the diminished survival benefit of second‐line ipilimumab for metastatic melanoma in the real world, this population‐based cost‐effectiveness analysis validates trial based model estimations by the Canadian Agency for Drugs and Technologies in Health to conclude that ipilimumab is not cost‐effective compared to non‐ipilimumab treatments. The findings of this study highlights the value of incorporating real‐world evidence to inform life‐cycle reassessment.
Introduction: The quality of an Emergency Medical Technician (EMT) or paramedic training program is likely one factor that contributes to a candidate's success on the National Registry Cognitive ...Examination. However, program pass rates and their associated geographic location have not previously been evaluated. Our objective was to evaluate the performance of EMT and paramedic programs in the United States, pass rates on the National Registry Cognitive Examinations, and relationship to geography.
Methods: We conducted a cross-sectional evaluation of EMT and paramedic programs' first and cumulative third attempt pass rates on the National Registry Cognitive Examination in 2015. Included were civilian EMT and paramedic programs with >5 graduates testing for certification. Descriptive statistics were calculated for program pass rates, total graduates testing, and pass rates by National Association of State EMS Officials (NASEMSO) regions.
Results: We included 1,939 EMT programs (6-1,892 graduates testing/program) and 602 paramedic programs (6-689 graduates testing/program). We excluded 262 (11.9%) EMT and 51 (7.8%) paramedic programs with ≤5 graduates testing annually due to unstable estimates of program pass rates. EMT programs in the highest quartile for total number of graduates testing outperformed the lowest quartile in both first attempt (65.7% vs. 61.9%, p < 0.001) and cumulative third attempt pass rates (79.1% vs 72.7%, p < 0.001). This difference was also seen for paramedic programs on first attempt (77.3% vs. 62.5%, p < 0.001) and cumulative third attempt (91.9% vs. 76.9%, p < 0.001). EMT program pass rates for first and cumulative third attempts also varied by NASEMSO region (first: 62-68%; third: 74-78%) with the Great Lakes and West regions outperforming the other regions. Paramedic program pass rates differed by NASEMSO regions as well (first: 65-83%; third: 81-95%) with highest pass rates in the West region.
Conclusions Program performance for both EMT and paramedic programs varies by total number of graduates testing and geographic location. Graduates from larger EMT and paramedic programs have higher first and cumulative third attempt pass rates compared to graduates from smaller programs. Additionally, there is variability in program pass rates across NASEMSO regions for both certification levels. Further evaluation is necessary to better understand the variability in program performance in the United States.
PDF
