Differences between revisions 3 and 99 (spanning 96 versions)
Revision 3 as of 2007-12-01 13:58:39
Size: 5064
Editor: TimoSirainen
Comment:
Revision 99 as of 2013-09-02 02:01:20
Size: 833
Editor: TimoSirainen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= IMAP Tester = = IMAP Server Tester =
Line 3: Line 3:
imaptest was originally written for stress testing [http://www.dovecot.org/ Dovecot] IMAP server. Its primary testing method is still stress testing, but now it also parses the incoming IMAP replies and complains if anything looks broken. ImapTest is a generic IMAP server compliancy tester that works with all IMAP servers. It supports:
Line 5: Line 5:
== Compiling ==  * [[ImapTest/TestFeatures|Stress testing with state tracking]]. ImapTest sends random commands to the server and verifies that server's output looks correct.
 * [[ImapTest/ScriptedTests|Scripted testing]] where it runs a list of predefined scripted tests and verifies that server returns expected output.
 * [[Benchmarking]]
Line 7: Line 9:
 1. Download and compile Dovecot v1.1 sources. imaptest uses its library functions.
  1. Get the latest beta from http://dovecot.org/releases/1.1/beta/
  1. Compiling goes the usual way: {{{./configure && make}}} (there's no need for {{{make install}}})
 1. Download http://www.dovecot.org/tools/imaptest.c to Dovecot sources' root directory (i.e. where the {{{configure}}} script exists)
 1. imaptest's header comment shows the compile command for gcc, use it or something similar.
== Table of Contents ==
Line 13: Line 11:
You may want to modify the default configuration from #defines at the top of the file. This isn't required, but if you run imaptest often this way you don't have to give the same parameters every time.

== Running ==

The most important parameters (and their defaults) are:
 * host=127.0.0.1: Host name/IP address where to connect.
 * port=143: Port what to use.
 * user=tss: Username template. You can use multiple random users and domains by giving %d in the template. So for example {{{user%d}}} returns user1..user100 and {{{user%d@domain%d.org}}} returns user1..100@domain1..100.org. The 1..100 range is hardcoded to the binary. You can change them from {{{USER_RAND}}} and {{{DOMAIN_RAND}}} defines.
 * password=pass: Password to use for all users. There's currently no way to use different passwords for different users.
 * mbox=/home/tss/mail/dovecot-crlf: Path to mbox file where to append messages from. See below for how this is used.

Other useful parameters include:
 * clients=10: Number of simultaneous client connections to use.
 * secs=n: Run imaptest n seconds and then exit.
 * box=INBOX: Mailbox to use for testing. INBOX is the default.
 * copybox=n: When testing COPY command, this specifies the destination mailbox.
 * seed=n: Seed to use for random generator. Setting this to some specific value makes repeated benchmarks a bit more reliable, because the used commands should be the same then.
 * disconnect_quit: If a client gets disconnected, quit. Use logout=0 parameter with this one. This is useful when debugging some problem in the server.
 * no_pipelining: Don't send multiple commands at once to server.

Selecting what to test:
 * <state>=<probability>[,<probability2>]: State probabilities to use. See below.
 * checkpoint=n: Run a checkpoint every n seconds. See below.
 * random: Switch randomly between states.
 * no_tracking: Don't track and complain about IMAP state. Makes it use a bit less CPU.

== Append mbox ==

When saving messages, imaptest needs to get the messages from somewhere. mbox=path parameter specifies path to a file in mbox format that's used. Messages are sequentially appended from there. Once imaptest reaches the last message, it wraps back to appending the first message.

Currently imaptest's state tracking expects that Message-IDs are unique within the mbox, otherwise it gives bogus errors. If you really want to avoid changing the Message-IDs, use no_tracking setting to disable state tracking.

You can get a test mbox file from for example http://dovecot.org/archives/dovecot.mbox. Note that it contains non-unique Message-IDs so you'll have to write some script to change those.

== States ==

|| Name || Short name || Description ||
|| AUTHENTICATE || Auth ||
|| LOGIN || Logi ||
|| LIST || List ||
|| LCREATE || LCre ||
|| LDELETE || LDel ||
|| STATUS || Stat ||
|| SELECT || Sele ||
|| FETCH || Fetc ||
|| FETCH2 || Fet2 ||
|| SEARCH || Sear ||
|| SORT || Sort ||
|| THREAD || Thre ||
|| COPY || Copy ||
|| STORE || Stor ||
|| DELETE || Dele ||
|| EXPUNGE || Expu ||
|| APPEND || Appe ||
|| NOOP || Noop ||
|| CHECK || Chec ||
|| LOGOUT || Logo ||
|| DISCONNECT || Disc ||
|| DELAY || Dela ||
|| CHECKPOINT! || ChkP ||

== State tracking ==

imaptest should catch the following errors:

 * Referring to sequences that haven't been announced with EXISTS.
 * Sequence <-> UID map changes unexpectedly
 * Message's metadata changes unexpectedly. Messages are uniquely identified by their Message-ID: header.
  * BODY, BODYSTRUCTURE, ENVELOPE and RFC822.SIZE are currently tracked.
  * IMAP servers handle FETCHes for expunged messages differently. imaptest tries to catch these.
 * FETCH FLAGS listing keywords that haven't been announced with untagged FLAGS.
 * Untagged FLAGS reply dropping a keyword that's still in use.

Checkpointing works by letting all the pending commands finish. Then CHECK command is sent to all sessions. Once they're done, imaptest verifies that all clients' mailbox state looks exactly the same:

 * Number of messages is the same
 * Sequence <-> UID map is the same
 * Flags and keywords are the same
 * \Recent flag for a message exists only session (all mailboxes are SELECTed currently)

== Common tests ==
 * [[ImapTest/Installation|Installation]]
 * [[ImapTest/Running|Running]]
  * [[ImapTest/Examples|Examples]]
 * [[ImapTest/States|States]] that can be used for stress testing
 * [[ImapTest/ScriptedTests|Scripted Testing Configuration]]
 * [[ImapTest/TestFeatures|Supported tests]]
 * [[ImapTest/ServerStatus|IMAP Server Compliancy Status]]

IMAP Server Tester

ImapTest is a generic IMAP server compliancy tester that works with all IMAP servers. It supports:

Table of Contents

None: ImapTest (last edited 2013-09-02 02:01:42 by TimoSirainen)