Urologic oncology randomized controlled trials are frequently fragile - A review of the urology literature

Background: Randomized controlled trials (RCT) in urologic oncology are the basis of patient management. Considerable debate exists on the limitation of statistical reporting of randomized controlled trials. Fragility index (FI) is a measure of the number of events upon which the trials statistical results depend on. The FI is defined as the minimum number of patients whose status would have to change from a 'non-event' to 'event', in order to turn a statistically significant result to a non-significant result and vice versa. Our aim is to examine the FI of RCT's in urologic oncology published in the urology literature. Material and Methods: We exhaustively searched MEDLINE and EMBASE from January 1, 2016 to December 31 2019 for RCT's in urology journals. Only studies reporting dichotomous outcomes were included and FI was calculated for each outcome. The distributions of FI across different journals and types of outcome (primary/ secondary, significant/ non-significant) were assessed. We examined the correlation of FI with sample size and P-value. Results: We identified 216 RCT's, 79 were eligible for analysis. Median FI was 3.0 (2.5, 6.0). One hundred and forty-six (89.6%) outcomes from 72 (94.7%) RCT's had a FI lower than 10 and 6 (3.7%) outcomes from 6 (7.9%) RCT's had a FI = 0. There was no statistically significant difference in FI between different types of outcomes. There was a modest correlation between the FI and the sample size (r_s 0.50, P < 0.01) and a weak correlation between the FI the P value (r_s 0.15 (P < 0.04)). Conclusions: Randomized controlled trials in urology journals that study dichotomous outcomes often report fragile results that should be regarded with caution. Reporting the FI alongside P values may enhance the interpretation and implementation of urologic oncology RCT's.