Using video recorded neonatal intensive care unit scenarios to test agreement between clinicians' assessments

Clinicians in Neonatal Intensive Care Units are required to perform assessments under extreme pressure to determine the illness severity of neonates. Currently there is no reference that indicates the clinical condition of a neonate according to its vital signs. Agreement between clinicians' assessments can provide initial indication for the clinicians' ability to systematically perform assessments. Agreement was tested between 16 clinicians who viewed 31 recorded cases and scored the illness severity of a neonate mannequin in each case. The agreement level was fair (0.28), but high correlation between the assessments (0.8<r<0.94) suggests that the clinicians were able to systematically score the cases, and they had similar interpretation about the relative illness severity between cases. Future studies are required to determine the sources for the relative low agreement level and to identify means for improvement. Establishment of a valid reference early warning score is essential to guide the assessment.