Choosing the most adequate PROM for a study is a non‐trivial process. The aim of this study was to provide a catalogue with analyses of content and construct validity of PROMs relevant to research in ...sports science, including all published local translations. The most commonly used PROMs in sports research were selected from a PubMed search “patient reported outcome measures sports”, identifying 439 articles and 194 different PROMs. Articles describing development of the 61 selected PROMs were assessed for content validity, and all articles regarding construct validity of each PROM and all published translations (in total 622 articles) were analyzed. A catalogue with assessments of the 61 PROMs was produced. The majority were of inferior validity, with few exceptions. The most common reason for this was that the PROM had not been developed by methods that ensure high content validity. Another major reason for inferior validity was that construct validity had not been secured by adequate statistical methods. In conclusion, this catalogue provides a tool for researchers to facilitate choosing the most valid PROM for studies in sports research. Furthermore, it shows for popular PROMs where further validation is needed, and for fields in musculoskeletal medicine where valid PROMs are lacking. It is suggested that a targeted effort is made to develop valid PROMs for major conditions in musculoskeletal research. The current method is easier to practice compared with assessment after COSMIN guidelines.
Results by patient‐reported outcome measures (PROMs) from randomized controlled trials (RCTs) in musculoskeletal research often influence healthcare strategies. We aimed to evaluate to which extent ...these RCTs use adequate PROMs, and how this influences the results and conclusions. We identified RCTs of sports research relevance with PROMs as primary outcomes published in 13 preselected journals between January 1, 2008, and November 1, 2019; all journals regularly publish results from musculoskeletal research. Five journals have a high impact factor (>15), and eight with lower impact factors are widely read journals. It was assessed whether the RCTs had used PROMs with high content validity and whether the most adequate PROMs were used (ie, the most well developed and well validated for the patients enrolled in the study). We registered journal impact factor, year of publication, existence of a registered protocol, and whether the study showed significant difference between interventions. A total of 54 RCTs with 56 primary outcomes comprising 26 different PROMs were identified. For 13 RCTs (24%), a protocol was not published. In only 24 of RCTs (44%), the most appropriate PROM had been used as primary outcome, independent of a registered protocol, ranking of the journal, and year of publication. In seven cases, PROMs were used to evaluate a condition that they had not been developed for. RCTs that used the most adequate PROM showed significantly more often (46%) difference in outcomes in contrast to RCTs that used inadequate PROMs (22%) (P = 0.0483). In conclusion, in the majority of RCTs, the most adequate PROM had not been used. Studies, in which the most adequate PROM had been used as outcome, were significantly more likely to show significant difference between interventions. The extent to which protocols were not available was surprisingly high. Journals should request that adequate PROMs are used in RCTs, and if this is not the case that it is discussed how it might influence the results and conclusions. Likewise, it should be requested that a protocol is published or registered.
Translating patient‐reported outcome measures (PROMs) can alter the meaning of items and undermine the PROM's psychometric properties (quantified as cross‐cultural differential item functioning DIF). ...The aim of this paper was to present the theoretical background for PROM translation, adaptation, and cross‐cultural validation, and assess how PROMs used in sports medicine research have been translated and adapted. We also assessed DIF for the Knee Injury and Osteoarthritis Outcome Score (KOOS) across Danish, Norwegian, and Swedish versions. We conducted a search in PubMed and Scopus to identify the method of translation, adaptation, and validation of PROMs relevant to musculoskeletal research. Additionally, 150 preoperative KOOS questionnaires were obtained from the Scandinavian knee ligament reconstruction registries, and cross‐cultural DIF was evaluated using confirmatory factor analysis and Rasch analysis. There were 392 studies identified, describing the translation of 61 PROMs. Ninety‐four percent were performed with forward‐backward technique. Forty‐nine percent used cognitive interviews to ensure appropriate wording, understandability, and adaptation to the target culture. Only two percent were validated according to modern test theory. No study assessed cross‐cultural DIF. One KOOS subscale showed no cross‐cultural DIF, two had DIF with respect to some (but not all) items, and thus conversion tables could be constructed, and two KOOS subscales could not be pooled. Most PROM translations are of undocumented quality, despite the common conclusion that they are valid and reliable. Scores from three of five KOOS subscales can be pooled across the Danish, Norwegian, and Swedish versions, but two of these must be adjusted for DIF.
What is a PROM and why do we need it? Krogsgaard, Michael R.; Brodersen, John; Christensen, Karl Bang ...
Scandinavian journal of medicine & science in sports,
20/May , Letnik:
31, Številka:
5
Journal Article
Recenzirano
The purpose of this article was to introduce the reader to the nature of patient‐reported outcome measures (PROMs) and pitfalls in their use. PROMs collect subjective information directly from the ...patient regarding specific or general conditions and add to clinical and functional outcomes, and turn unmeasurable subjective qualities into quantitative measures. PROMs are questionnaires consisting of items: questions or statements with predefined response options. The items in an adequate PROM have been developed by involvement of patients with the condition in focus, and the PROM has been validated for these patients using suitable statistical methods. An adequate well‐targeted PROM is more responsive than an inadequate PROM. Unfortunately, many studies use inadequate PROMs as outcomes. The methods used to generate PROMs should be described as thoroughly as those used to develop any other types of measurement instruments, and the choice of PROM should always be explained and thereby justified. If the PROM used is not adequate, the consequences for the interpretation of the results should be discussed. In many cases, an adequate PROM does not exist. If the best available PROM is chosen, there are methods to validate the adequacy of the chosen PROM, which make an interpretation of the study results possible.
Deviations from adequate use and reporting of PROMs may be problematic and misleading. The aim of this study was to investigate the extent of such problems in randomized clinical trials (RCTs). RCTs ...involving sports medicine research that used PROMs as primary outcomes were identified in 13 preselected journals. The articles were reviewed for nine potential problems related to how the PROM was used and how the data had been reported. The potential problems were as follows: aggregating subscale scores; combining patient‐reported scores with physical, clinical, or para‐clinical measures; using a PROM to diagnose or evaluate the individual patient; using a PROM for one leg or arm; selectively excluding domains or items; constructing a PROM for the specific occasion; mixing PROM formats (ie, digital, paper, telephone, e‐mail, in person); ambiguous instructions for how the PROM should be completed; and recall bias. As covariates, we registered journal impact factor, year of publication, and existence of a registered protocol. In 29 (53.7%) of 54 identified RCTs, at least one potential problem was identified, the most common being aggregation of domain scores. This was not different with a published protocol or dependent on journal rankings, except for exclusion of domains, which was most common in high‐ranking journals. Aggregation of domain scores was significantly less common in recently published articles compared with older articles (P = .03). Potential problematic use of PROMs and reporting of PROM data are common in RCTs, also in high‐ranking journals, but less so in more recent articles.
Psychometric validation of PROM instruments Christensen, Karl B.; Comins, Jonathan D.; Krogsgaard, Michael R. ...
Scandinavian journal of medicine & science in sports,
June 2021, 2021-Jun, 2021-06-00, 20210601, Letnik:
31, Številka:
6
Journal Article
Recenzirano
The aim was to provide an overview of the different statistical methods for validation of patient‐reported outcome measures, ranging from simple statistical methods available in all software packages ...to advanced statistical models that require specialized software. A non‐technical summary of classical test theory (CTT) and modern test theory (MTT) is provided. Specifically, confirmatory factor analysis, item response theory, and Rasch analysis is outlined. One CTT and three MTT methods were used to validate the two subscales (Symptoms and Quality of Life) from the Knee Injury and Osteoarthritis Outcome Score (KOOS). For each methodology, two analyses were considered: (i) a unidimensional analysis ignoring the pre‐specified dimensionality, and (ii) a two‐dimensional analysis using the pre‐specified dimensionality. While CTT did not adequately address central issues regarding the validity of the KOOS subscales, the three MTT methods yielded very similar results. In conclusion, MTT methods offer analysis of all relevant properties related to the validity of patient‐reported outcome measures, while this is not the case for CTT. Claims about sufficient validity based on CTT methods are inadequate and should not be trusted.
How to develop a condition‐specific PROM Comins, Jonathan D.; Brodersen, John; Siersma, Volkert ...
Scandinavian journal of medicine & science in sports,
June 2021, 2021-Jun, 2021-06-00, 20210601, Letnik:
31, Številka:
6
Journal Article
Recenzirano
Developing new patient‐reported outcome measures (PROMs) for application in clinical studies can be necessary if an adequate PROM does not exist. For adequate measurement, it is essential that the ...PROM has face validity (ie, is perceived to be relevant by clinicians and researchers) and has high content validity (ie, content relevance and content coverage for the targeted patient group). The steps needed to create PROMs that possess face and content validity for a specific condition are described in this paper. Face validity is achieved by item identification and generation through literature review. Content validity is confirmed through repetitive cognitive interviews of patients from the targeted patient group in order to generate a consensus‐based pilot‐version of the new PROM. This qualitative process ensures that items are appropriately worded, understandable, and minimizes doubts about how items should be answered. A practical example of this process is presented, which shows the development of the Knee Numeric‐Entity Evaluation Score (KNEES‐ACL), a condition‐specific PROM for patients with deficiency of the anterior cruciate ligament (ACL).
To use an inadequate patient reported outcome measure (PROM) or use a PROM in an inappropriate way potentially influences the quality of measurement. The objectives of this study were to define ...potential inadequate uses of PROMs in sports research studies and estimate how often they occur. A consensus group consisting of medical researchers, statisticians, and psychometricians identified and defined potentially irregular applications of PROMs. Occurrence of these in 349 consecutive articles in sports medicine in which PROMs were used as primary outcomes was reviewed. In all, 14 different potential problems were defined, and one or several occurred in 172 of the articles (49%). These were as follows: using a PROM that was developed for a different patient group (100 cases), using two or more PROMs with identical questions (94), aggregation of domain sum scores (82), combinations of subjective and objective measures (27), using a PROM to diagnose or evaluate the individual patient (7), using a PROM for a single limb (3), recall bias (3), exclusion of domains or items (3), construction of a PROM for a specific occasion (2), categorization of the scale (2), and mixing different versions of a PROM (1). Adaption of scale scores (e. g., to percentage) when results are reported (144) carries a risk of miscalculation and distorted impression of results. Data related to uncertainty about completing the PROM and the handling of missing data were not provided in the manuscripts. In conclusion, potential problems in the use and reporting of PROMs are common in sports research, and this can influence the validity of reported results.
Choosing the most appropriate patient‐reported outcome measure (PROM) for a clinical study is essential in order to achieve trustworthy results. This choice will depend on (a) the objective of the ...study and hence the research question; (b) the choice of a theoretical framework, such as the World Health Organization's International Classification of Functioning, Disability, and Health (ICF); (c) whether there currently is a PROM that possesses high content validity and high construct validity for the specific patient group and objective, and if not; (d) the decision on whether to use a suboptimal PROM or develop and validate a new PROM. This paper presents the steps that should be followed in order to assess the relevance of PROMs and suggests ways to enhance the choice depending on the goal of the study.
PDF
