X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words

Uri Avni*, Hayit Greenspan, Eli Konen, Michal Sharon, Jacob Goldberger

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

170 Scopus citations


In this study we present an efficient image categorization and retrieval system applied to medical image databases, in particular large radiograph archives. The methodology is based on local patch representation of the image content, using a bag of visual words approach. We explore the effects of various parameters on system performance, and show best results using dense sampling of simple features with spatial content, and a nonlinear kernel-based support vector machine (SVM) classifier. In a recent international competition the system was ranked first in discriminating orientation and body regions in X-ray images. In addition to organ-level discrimination, we show an application to pathology-level categorization of chest X-ray data, the most popular examination in radiology. The system discriminates between healthy and pathological cases, and is also shown to successfully identify specific pathologies in a set of chest radiographs taken from a routine hospital examination. This is a first step towards similarity-based categorization, which has a major clinical implications for computer-assisted diagnostics.

Original languageEnglish
Article number5643927
Pages (from-to)733-746
Number of pages14
JournalIEEE Transactions on Medical Imaging
Issue number3
StatePublished - Mar 2011


  • Chest radiography
  • X-ray
  • computer-aided diagnosis (CAD)
  • disease labeling
  • image categorization
  • image patches
  • image retrieval
  • visual words


Dive into the research topics of 'X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words'. Together they form a unique fingerprint.

Cite this