Comparisons of nomograms and urologists' predictions in prostate cancer

When applying nomograms to a clinical setting it is essential to know how their predictions compare with clinicians'. Comparisons exist outside of the prostate cancer literature. We reviewed these comparisons and conducted 2 experiments comparing predictions of clinicians with prostate cancer nomograms. By using Medline, we searched studies from January 1966 to July 1999 that compared human predictions with nomogram predictions. Next, we conducted 2 experiments: (1) 17 urologists were presented with 10 case vignettes and asked to predict the 5-year recurrence-free probabilities for each patient; (2) case presentations of 63 prostate cancer patients (including full clinical histories with complete diagnostic data and surgical findings) were made to a group of 25 clinicians who were asked to predict organ-confined disease. We found 22 published studies comparing human experts with nomograms, greater than half (13 of 22) showed the nomogram performing above the level of the human expert. Our first experiment showed urologist modification of 165 nomogram predictions led to a decrease in prediction accuracy (c-index decreased from .67 to .55, P < .05). In our second experiment, clinician predictions of organ-confined disease were comparable to the nomogram (area under the receiver operating characteristic curve [AUC] 0.78 and 0.79, respectively). A mixed-model suggests the nomogram did not augment clinician prediction accuracy (doctor excess error 1.4%, P = .75, 95% confidence interval [Cl]: -10.9% to 8.2%). Our data suggest that nomograms do not seem to diminish predictive accuracy and they may be of significant benefit in certain clinical decision making settings.