Utilizing the IOMMU Scalably

Omer Peleg, Adam Morrison, Benjamin Serebrin, Dan Tsafrir

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


IOMMUs provided by modern hardware allow the OS to enforce memory protection controls on the DMA operations of its I/O devices. An IOMMU translation management design must scalably handle frequent concurrent updates of IOMMU translations made by multiple cores, which occur in high throughput I/O workloads such as multi-Gb/s networking. Today, however, OSes experience performance meltdowns when using the IOMMU in such workloads. This paper explores scalable IOMMU management designs and addresses the two main bottlenecks we find in current OSes: (1) assignment of I/O virtual addresses (IOVAs), and (2) management of the IOMMU's TLB. We propose three approaches for scalable IOVA assignment: (1) dynamic identity mappings, which eschew IOVA allocation altogether, (2) allocating IOVAs using the kernel's kmalloc, and (3) per-core caching of IOVAs allocated by a globally-locked IOVA allocator. We further describe a scalable IOMMU TLB management scheme that is compatible with all these approaches. Evaluation of our designs under Linux shows that (1) they achieve 88.5%-100% of the performance obtained without an IOMMU, (2) they achieve similar latency to that obtained without an IOMMU, (3) scalable IOVA allocation and dynamic identity mappings perform comparably, and (4) kmalloc provides a simple solution with high performance, but can suffer from unbounded page table blowup.

Original languageEnglish
Title of host publicationProceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015
PublisherUSENIX Association
Number of pages14
ISBN (Electronic)9781931971225
StatePublished - 2015
Externally publishedYes
Event2015 USENIX Annual Technical Conference, USENIX ATC 2015 - Santa Clara, United States
Duration: 8 Jul 201510 Jul 2015

Publication series

NameProceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015


Conference2015 USENIX Annual Technical Conference, USENIX ATC 2015
Country/TerritoryUnited States
CitySanta Clara


Dive into the research topics of 'Utilizing the IOMMU Scalably'. Together they form a unique fingerprint.

Cite this