Two rules for early prediction of bacteremia: Testing in a university and a community hospital

BACKGROUND: Two rules (model 1 and model 2) were previously derived and prospectively validated at the same institution to predict the likelihood of bacteremia. The objective of the present study was to test and compare the performance of the rules in patients admitted to two sites of inpatient care: a university hospital and a community hospital. METHODS: Clinical and laboratory data (including the variables contained in the two models) were collected within 24 hours in all patients admitted to the Department of Medicine of the Beilinson Medical Center, a university hospital in central Israel, and Emek Hospital, a community hospital in northern Israel, because of an acute infectious disease. The scores of the models were compared with the results of blood cultures. RESULTS: The percentage of bacteremia was 15% in the university and 10.5% in the community hospital. The area under the receiver-operating characteristic curve was 0.56 ± 0.04 SE for model 1, and 0.67 ± 0.04 SE for model 2 in the university hospital; and 0.59 ± 0.05 SE versus 0.63 ± 0.04 SE, respectively, in the community hospital. At the best calibration, model 1 defined low-risk groups of 205 patients in the university hospital, and 66 patients in the community hospital, with prevalences of bacteremia of 13% and 15%. The high-risk groups defined by model 1 had prevalences of 30% and 32%. Model 2 defined low-risk groups with prevalences of bacteremia of 7% (8 of 114) and 8% (6 of 76); and high-risk groups with percentages of 29% and 38%. CONCLUSIONS: The overall accuracy of the two models deteriorated significantly. Both models defined groups at high risk of bacteremia, but the percentages of bacteremia and of death in the low-risk groups do not encourage withholding blood cultures in these patients. The failure of the two models points toward the need for external validation, and for monitoring performance of prediction models over time.