IMAP Server Benchmarking

"Which IMAP server is the fastest?" is a difficult question to answer, because the answer completely depends on what IMAP clients are used. There are basically two types of IMAP clients:

  1. Caching IMAP clients which fetch new messages only once from the server. After that the only metadata the client is interested of is the message's flags. Most real IMAP clients belong to this category (e.g. Outlook, Thunderbird, Apple Mail).
  2. Non-caching IMAP clients keep fetching the same messages over and over again. Webmails typically belong to this category.

Some IMAP servers cache commonly used metadata for non-caching clients to avoid reading and parsing the message multiple times. This has however some problems:

If clients can be placed into two categories, servers can be placed into three:

  1. Non-caching (or very little / non-permanently caching) servers parse the message whenever a client requests for metadata (e.g. UW-IMAP, Courier).
  2. Statically caching servers keep a predefined list of metadata permanently cached (e.g. Cyrus).
  3. Dynamically caching servers change the list of permanently cached metadata based on what clients actually use (e.g. Dovecot).

We can describe the client <-> server performance as a table (assumes theoretically optimal implementations):

Non-caching server

Statically caching server

Dynamically caching server

Caching client

Optimal performance.

Wastes disk I/O and disk space unneededly.

Near-optimal performance.

Non-caching client

Worst performance.

Optimal performance if cached fields match exactly the wanted fields. Near-optimal if all fields are cached. Bad performance if some fields are missing.

Optimal performance.

So when you read about IMAP benchmarks, make sure you know what caching models the benchmark tests for.

Other issues

Software

Benchmarking (last edited 2013-01-18 06:53:25 by TimoSirainen)