A mixture of views network with applications to multi-view medical imaging

This paper examines data fusion methods for multi-view data classification. We present a decision concept that explicitly takes into account the input multi-view structure, where for each case there is a different subset of relevant views. This data fusion concept, which we dub Mixture of Views, is implemented by a special purpose neural network architecture. The single view decisions are combined by a data-driven decision, into a global decision according to the relevance of each view in a given case. The method was applied to two challenging computer-aided diagnosis (CADx) tasks: the task of classifying breast microcalcifications as benign or malignant based on craniocaudal (CC) and mediolateral oblique (MLO) mammography views and segmenting Multiple Sclerosis (MS) white matter lesions. The experimental results show that our method outperforms previously suggested fusion methods.