UNI-MB - logo
UMNIK - logo
 
E-resources
Full text
Peer reviewed Open access
  • Abstract 275: Hypothesizing...
    Vaddi, Venya; Hou, Jesse W.; Rao, Neal M.; Saver, Jeffrey L.

    Stroke: vascular and interventional neurology, 11/2023, Volume: 3, Issue: S2
    Journal Article

    Abstract only Introduction A “reproducibility crisis” has afflicted the recent medical literature, affecting 75‐89% of major reported findings. An important cause of the inability to replicate results is hypothesizing after the results are known (HARKing): presenting a post hoc hypothesis according with findings as if it were a confirmed a priori hypothesis. Meeting abstract instructions may affect the frequency of HARKing. Methods We tested the hypothesis that, compared with abstract instructions NOT indicating a "hypothesis" should be stated, instructions indicating THAT a "hypothesis" should be stated would increase the proportion of abstracts judged by field content experts to exhibit HARKing. Hypothesis pre‐registered at AsPredicted.org (#5019). We analyzed randomly sampled abstracts from 4 International Stroke Conference (ISC) meeting years with highly forceful (2015), moderately forceful (2017), and non‐forceful (2013 and 2022) requirements for statement of a “hypothesis” in abstract preparation instructions, regardless of whether a hypothesis statement was appropriate (confirmatory science) or inappropriate (exploratory science). Results The 163 abstracts (39 to 43 in each meeting year) equally sampled the 14‐17 ISC topic categories. Explicit “hypothesis” statements were more common in the highly and moderately forceful instruction years (30% and 27%) than in the non‐forceful instruction years (9% and 13%) (p = 0.006). Among the total 32 hypothesis statements, 81% were deemed falsifiable, and 19% non‐falsifiable. None (0%) of hypotheses were stated as pre‐registered prior to study conduct. Overall, 59% (19/32) of the hypothesis statements were adjudicated by two independent content experts as likely to be hypothesizing after the results were known (HARKing). HARKing was more common in the highly and moderately forceful instruction years (20% and 15%) than the non‐forceful instruction years (9% and 3%) (p = 0.03). Conclusion Abstract preparation instructions requiring the use of a hypothesis statement promoted hypothesizing after the results are known, with HARKing present in 1 out of every 5 meeting abstracts. To improve reproducibility of neurovascular research studies, appropriate meeting abstract preparation instructions are desirable, recommending hypothesis statements only for the small proportion of stroke studies that are of confirmatory, rather than exploratory, design.