Background Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for ...evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs. Methods An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings. Results Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM. Discussion The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.
Purpose The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on ...measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties. Methods For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis. Results Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system. Conclusions The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.
Summary Objective To conduct a systematic review and meta-analysis to synthesise evidence regarding measurement properties of the Knee injury and Osteoarthritis Outcome Score (KOOS). Design A ...comprehensive literature search identified 37 eligible papers evaluating KOOS measurement properties in participants with knee injuries and/or osteoarthritis. Methodological quality was evaluated using the COSMIN checklist. Where possible, meta-analysis of extracted data was conducted for all studies and stratified by age and knee condition; otherwise narrative synthesis was performed. Results KOOS has adequate internal consistency, test-retest reliability and construct validity in young and old adults with knee injuries and/or osteoarthritis. The ADL subscale has better content validity for older patients and Sport/Rec for younger patients with knee injuries, while the Pain subscale is more relevant for painful knee conditions. The five-factor structure of the original KOOS is unclear. There is some evidence that the KOOS subscales demonstrate sufficient unidimensionality, but this requires confirmation. Although measurement error requires further evaluation, the minimal detectable change for KOOS subscales ranges from 14.3 to 19.6 for younger individuals, and ≥20 for older individuals. Evidence of responsiveness comes from larger effect sizes following surgical (especially total knee replacement) than non-surgical interventions. Conclusions KOOS demonstrates adequate content validity, internal consistency, test-retest reliability, construct validity and responsiveness for age- and condition-relevant subscales. Structural validity, cross-cultural validity and measurement error require further evaluation, as well as construct validity of KOOS-PS. Suggested order of subscales for different knee conditions can be applied in hierarchical testing of endpoints in clinical trials.
Purpose Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one ...or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs. Methods Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs. Results A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1-4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5-8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review. Conclusions The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.
Summary
Background
The Harmonising Outcome Measures for Eczema (HOME) initiative has defined four core outcome domains for a core outcome set (COS) to be measured in all atopic eczema (AE) trials to ...ensure cross‐trial comparison: clinical signs, symptoms, quality of life and long‐term control.
Objectives
The aim of this paper is to report on the consensus process that was used to select the core instrument to consistently assess symptoms in all future AE trials.
Methods
Following the HOME roadmap, two systematic reviews were performed which identified three instruments that had sufficient evidence of validity, reliability and feasibility to be considered for the final COS.
Results
At the fourth international HOME meeting, there was broad consensus among all stakeholders that the Patient‐Oriented Eczema Measure (POEM) should be used as the core instrument (87·5% agreed, 9·4% unsure, 3·1% disagreed).
Conclusions
All relevant stakeholders are encouraged to use POEM as the chosen instrument to measure the core domain of symptoms in all future AE clinical trials. Other instruments of interest can be used in addition to POEM.
What's already known about this topic?
There is insufficient high‐quality evidence for many of the treatments of atopic eczema (AE), which is partly due to the heterogeneity in outcomes used in clinical trials.
The Harmonising Outcome Measures for Eczema initiative defined ‘symptoms’ as one of the core outcome domains that should be measured in AE clinical trials.
What does this study add?
Consensus was reached on the Patient‐Oriented Eczema Measure (POEM) as the core instrument to measure symptoms.
This statement should promote awareness among all stakeholders.
What are the clinical implications of this work?
Not only itch and sleeplessness are symptoms important for patients with AE.
All relevant stakeholders are encouraged to use POEM as the chosen instrument to measure the core domain of symptoms in all future AE clinical trials.
Respond to this article
Linked Comment: Naldi. Br J Dermatol 2017; 176:852–853
Plain language summary available online
In cooperation with the Core Outcome Measures in Effectiveness Trials (COMET) initiative, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative aimed ...to develop a guideline on how to select outcome measurement instruments for outcomes (i.e., constructs or domains) included in a "Core Outcome Set" (COS). A COS is an agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population.
Informed by a literature review to identify potentially relevant tasks on outcome measurement instrument selection, a Delphi study was performed among a panel of international experts, representing diverse stakeholders. In three consecutive rounds, panelists were asked to rate the importance of different tasks in the selection of outcome measurement instruments, to justify their choices, and to add other relevant tasks. Consensus was defined as being achieved when 70 % or more of the panelists agreed and when fewer than 15 % of the panelists disagreed.
Of the 481 invited experts, 120 agreed to participate of whom 95 (79 %) completed the first Delphi questionnaire. We reached consensus on four main steps in the selection of outcome measurement instruments for COS: Step 1, conceptual considerations; Step 2, finding existing outcome measurement instruments, by means of a systematic review and/or a literature search; Step 3, quality assessment of outcome measurement instruments, by means of the evaluation of the measurement properties and feasibility aspects of outcome measurement instruments; and Step 4, generic recommendations on the selection of outcome measurement instruments for outcomes included in a COS (consensus ranged from 70 to 99 %).
This study resulted in a consensus-based guideline on the methods for selecting outcome measurement instruments for outcomes included in a COS. This guideline can be used by COS developers in defining how to measure core outcomes.
The COMET Handbook: version 1.0 Williamson, Paula R; Altman, Douglas G; Bagley, Heather ...
Trials,
06/2017, Letnik:
18, Številka:
Suppl 3
Journal Article
Recenzirano
Odprti dostop
The selection of appropriate outcomes is crucial when designing clinical trials in order to compare the effects of different interventions directly. For the findings to influence policy and practice, ...the outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. It is now widely acknowledged that insufficient attention has been paid to the choice of outcomes measured in clinical trials. Researchers are increasingly addressing this issue through the development and use of a core outcome set, an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area.Accumulating work in this area has identified the need for guidance on the development, implementation, evaluation and updating of core outcome sets. This Handbook, developed by the COMET Initiative, brings together current thinking and methodological research regarding those issues. We recommend a four-step process to develop a core outcome set. The aim is to update the contents of the Handbook as further research is identified.
Background
Symptoms have been identified as a core outcome domain for atopic eczema (AE) trials. Various instruments exist to measure symptoms in AE, but they vary in quality and there is a lack of ...standardization between clinical trials. Our objective was to systematically evaluate the quality of the evidence on the measurement properties of AE symptom instruments, thereby informing consensus discussions within the Harmonising Outcome Measures for Eczema (HOME) initiative regarding the most appropriate instruments for the core outcome domain symptoms.
Methods
Using the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) checklist and predefined criteria for good measurement properties on identified development and validation studies of AE symptom instruments, a best evidence synthesis was performed to draw an overall conclusion on quality of the instruments and to provide recommendations.
Results
Eighteen instruments were identified and evaluated. When the quality and results of the studies were considered, only five of these instruments had sufficient validation data to consider them for the core outcome set for the core outcome domain symptoms. These were the paediatric Itch Severity Scale (ISS), Patient‐Oriented Eczema Measure (POEM), Patient‐Oriented SCOring Atopic Dermatitis (PO‐SCORAD), Self‐Administered Eczema Area and Severity Index (SA‐EASI) and adapted SA‐EASI.
Conclusions
ISS (paediatric version), POEM, PO‐SCORAD, SA‐EASI and adapted SA‐EASI are currently the most appropriate instruments and therefore have the potential to be recommended as core symptom instrument in future clinical trials. These findings will be utilized for the development of a core outcome set for AE.
Background
The Harmonising Outcome Measures for Eczema (HOME) initiative has identified quality of life (QoL) as a core outcome domain to be evaluated in every eczema trial. It is unclear which of ...the existing QoL instruments is most appropriate for this domain. Thus, the aim of this review was to systematically assess the measurement properties of existing measurement instruments developed and/or validated for the measurement of QoL in adult eczema.
Methods
We conducted a systematic literature search in PubMed and Embase identifying studies on measurement properties of adult eczema QoL instruments. For all eligible studies, we assessed the adequacy of the measurement properties and the methodological quality with the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A best evidence synthesis summarizing findings from different studies was the basis to assign four degrees of recommendation (A–D).
Results
A total of 15 articles reporting on 17 instruments were included. No instrument fulfilled the criteria for category A. Six instruments were placed in category B, meaning that they have the potential to be recommended depending on the results of further validation studies. Three instruments had poor adequacy in at least one required adequacy criterion and were therefore put in category C. The remaining eight instruments were minimally validated and were thus placed in category D.
Conclusions
Currently, no QoL instrument can be recommended for use in adult eczema. The Quality of Life Index for Atopic Dermatitis (QoLIAD) and the Dermatology Life Quality Index (DLQI) are recommended for further validation research.
COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) is an initiative of an international multidisciplinary team of researchers who aim to improve the selection of ...outcome measurement instruments both in research and in clinical practice by developing tools for selecting the most appropriate available instrument.
In this paper these tools are described, i.e. the COSMIN taxonomy and definition of measurement properties; the COSMIN checklist to evaluate the methodological quality of studies on measurement properties; a search filter for finding studies on measurement properties; a protocol for systematic reviews of outcome measurement instruments; a database of systematic reviews of outcome measurement instruments; and a guideline for selecting outcome measurement instruments for Core Outcome Sets in clinical trials. Currently, we are updating the COSMIN checklist, particularly the standards for content validity studies. Also new standards for studies using Item Response Theory methods will be developed. Additionally, in the future we want to develop standards for studies on the quality of non-patient reported outcome measures, such as clinician-reported outcomes and performance-based outcomes.
In summary, we plea for more standardization in the use of outcome measurement instruments, for conducting high quality systematic reviews on measurement instruments in which the best available outcome measurement instrument is recommended, and for stopping the use of poor outcome measurement instruments.
PDF
