Infinite images: Creating and exploring a large photorealistic virtual space

Biliana Kaneva, Josef Sivic, Antonio Torralba, Shai Avidan, William T. Freeman

Research output: Contribution to journalArticlepeer-review

Abstract

We present a system for generating infinite images from large collections of photos by means of transformed image retrieval. Given a query image, we first transform it to simulate how it would look if the camera moved sideways and then perform image retrieval based on the transformed image. We then blend the query and retrieved images to create a larger panorama. Repeating this process will produce an infinite image. The transformed image retrieval model is not limited to simple 2-D left/right image translation, however, and we show how to approximate other camera motions like rotation and forward motion/zoom-in using simple 2-D image transforms. We represent images in the database as a graph where each node is an image and different types of edges correspond to different types of geometric transformations simulating different camera motions. Generating infinite images is thus reduced to following paths in the image graph. Given this data structure we can also generate a panorama that connects two query images, simply by finding the shortest path between the two in the image graph. We call this option the image taxi. Our approach does not assume photographs are of a single real 3-D location, nor that they were taken at the same time. Instead, we organize the photos in themes, such as city streets or skylines and synthesize new virtual scenes by combining images from distinct but visually similar locations. There are a number of potential applications to this technology. It can be used to generate long panoramas as well as content aware transitions between reference images or video shots. Finally, the image graph allows users to interactively explore large photo collections for ideation, games, social interaction, and artistic purposes.

Original languageEnglish
Article number5473074
Pages (from-to)1391-1407
Number of pages17
JournalProceedings of the IEEE
Volume98
Issue number8
DOIs
StatePublished - Aug 2010
Externally publishedYes

Keywords

  • Image mosaicing
  • Internet images
  • Large datasets
  • Scene recognition
  • Transformed image retrieval
  • Virtual scene synthesis

Fingerprint

Dive into the research topics of 'Infinite images: Creating and exploring a large photorealistic virtual space'. Together they form a unique fingerprint.

Cite this