3452
Comment:
|
6277
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
= IMAP Server Compliance = | = IMAP Server Compliancy Status = * Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical. * It would actually be better to be consistent across all IMAP commands, but especially when NOOP or IDLE is used. Some servers have inconsistent IMAP session states because the client connections end up in e.g. different servers that don't synchronize immediately. This can cause problems with clients that rely on the different sessions seeing the same state. * \Recent: Exactly one session sees a new message as \Recent - no more and no less. * imaptest prints an error if it notices multiple sessions having the same \Recent flag ("Message seq=.. UID=.. has \Recent flag in multiple sessions"). This happens automatically when running imaptest with multiple clients for the same user, e.g. `imaptest user=testuser clients=10` * But missing \Recent flags isn't detected with any tests * Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other * Test by running stress test with own_flags parameter and multiple clients for the same user, e.g. `imaptest user=testuser clients=10 own_flags` * Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents? * Yes: Yes, everything can be fetched * Cached: Only cached data can be fetched. Message header or body can't be. * No: Nothing but flags can be fetched. * This needs to be tested manually to see how it behaves - there is no ready imaptest test for this. * Expunge store: If message is expunged in one session, can another session update its flags? * Yes: Yes, and a 3rd session also sees the change * Session: Yes, but a 3rd session won't see the change * Delayed: Yes, but a 3rd session won't see '''any''' changes until EXPUNGEs are reported * Ignored: STORE command replies OK, but no change is made * No: STORE command replies NO * This needs to be tested manually to see how it behaves - there is no ready imaptest test for this. * Failures (failures/total): Number of failures using [[ImapTest/ScriptedTests|scripted tests]]. These numbers may not be exact all the time, because the tests are still changing. * Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands. * Base fails: Number of individual base IMAP4rev1 protocol commands that failed. * Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers. * Test using e.g. `imaptest user=testuser test=tests/` Fully compliant servers: ||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' || ||[[http://www.dovecot.org/|Dovecot]] ||Yes ||Yes ||Yes ||Yes / Cached (depends on storage) ||Yes ||0/40 ||0/403 ||0/100 || ||[[https://github.com/jonabbey/panda-imap/|Panda IMAP]] 2008, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||0/34 ||0/328 ||0/97 || ||[[http://netwinsite.com/surgemail/|SurgeMail]] 5.0h3 ||Yes ||Yes ||Yes ||Yes ||Yes ||0/35 ||0/342 ||0/26 || |
Line 3: | Line 32: |
* Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical. * \Recent: Exactly one session sees a new message as \Recent - no more and no less. * Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other (own_flags test) * Search: SEARCH command correctness. * Expunges: How messages expunged by another session are handled. * A: Expunging a message from one session keeps it available for other sessions until they can be notified that it's expunged. Flag updates to expunged messages may not be visible to other sessions though. * B: Some cached fields may be returned for expunged messages, but message bodies can't be retrieved anymore. Flags can still be updated within the session (but updates may not be visible to other sessions). * C: Even message flags can't be updated for expunged messages. * Failures: Number of failures using [:ImapTest/Tests:command/reply tests] (failures / number of tests). These numbers may not be exact all the time, because the tests are still changing. |
|
Line 13: | Line 33: |
Fully compliant servers: | |
Line 15: | Line 34: |
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' || || [http://www.dovecot.org/ Dovecot] 1.1.rc2+ || Yes || Yes || Yes || OK || B || 0/21 || || [http://www.washington.edu/imap/ UW-IMAP] 2007a.DEV, mix format || Yes || Yes || Yes || OK || A || 0/21 || |
|
Line 20: | Line 36: |
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' || ||[[http://www.washington.edu/imap/|UW-IMAP]] 2007b, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||2/34 ||0/328 ||6/53 || ||[[http://www.isode.com/products/m-box.html|Isode M-Box]] 14.3a0 ||No ||Unreliable ||Yes ||Cached ||Ignored ||4/40 ||1/408 ||8/112 || ||[[http://www.communigate.com/community/|CommuniGate Pro]] 5.2.1 ||Yes ||Yes ||Yes ||Cached (some) ||No ||8/34 ||8/328 ||0/0 || ||[[https://www.cyrusimap.org/|Cyrus]] 3.0.10 ||No ||Unreliable ||Bugs ||Cached ||Delayed ||0/35 ||0/366 ||0/100 || ||[[http://www.sun.com/software/products/messaging_srvr/index.xml|Sun Java Messenging Server]] 6.3-0.15 ||No ||Unreliable ||Bugs ||Yes ||Delayed ||9/34 ||17/328 ||9/21 || ||[[http://www.archiveopteryx.org/|Archiveopteryx]] 3.0.3 ||No ||Unreliable ||No ||No ||No ||13/38 ||25/346 ||6/26 || ||[[http://www.courier-mta.org/imap/|Courier]] 4.3.1 ||Yes ||Unreliable ||Yes ||No ||No ||18/34 ||33/328 ||20/53 || ||[[http://www.gmail.com/|GMail]] 2012-08-12 ||No ||Not implemented ||Bugs ||Cached (some) ||Ignored ||12/49 ||66/376 ||0/0 || ||[[http://www.zarafa.com/|Zarafa Collaboration Platform]] 7.1.10 ||No ||Unreliable ||Yes ||No ||No ||6/35 ||14/340 ||0/0 || ||[[http://www.zimbra.com/|Zimbra]] 5.0.5 ||Yes ||Yes ||Yes ||Cached ||No ||8/34 ||50/328 ||2/33 || ||[[http://www.microsoft.com/exchange/default.mspx|MS Exchange]] ||No ||Unreliable ||? ||No ||No ||16/40 ||52/287 ||0/0 || ||[[http://www.citadel.org/|Citadel]] 7.36 ||No ||Unreliable ||Yes ||Yes ||Session (flag changes are never seen by other sessions) ||19/34 ||98/328 ||0/0 || ||[[http://www.hmailserver.com/|hMailServer]] 5.3.3 b1879 ||? ||? ||? ||? ||? ||||||<style="text-align: center;">Fails hardcoded OK response format test. Other tests don't work, if OK response test fails. || |
|
Line 21: | Line 51: |
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' || || [http://www.archiveopteryx.org/ Archiveopteryx] 2.06 || No |||| ? (connections hang too easily) || Bugs || C || || [http://www.courier-mta.org/imap/ Courier] 4.3.0 || Yes || Unreliable || Yes || SMALLER/LARGER wrong || C || 10/21 || || [http://cyrusimap.web.cmu.edu/ Cyrus] 2.3.11 || No || Unreliable || Bugs || Int.dates wrong || B? Buggy || || [http://www.dovecot.org/ Dovecot] 1.0 || Yes || Unreliable || Bugs || Int.dates wrong || B || || [http://www.isode.com/products/m-box.html Isode M-Box] 14.2a0 || No || Unreliable || Yes || () bugs || C || || [http://www.sun.com/software/products/messaging_srvr/index.xml Sun Java Messenging Server] 6.3-0.15 || No || Unreliable || Bugs || Bugs || A || 6/20 || || [http://www.washington.edu/imap/ UW-IMAP] 2007, mix format || Yes || Yes || Bugs || OK || A || || [http://www.zimbra.com/ Zimbra] 5.0.1 || Yes || Yes || Yes || OK || C || 8/20 || |
|
Line 32: | Line 55: |
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' || || [http://www.citadel.org/ Citadel] 7.32 || No || ? || ? || Bugs || EXPUNGEs sent too often || || [http://www.communigate.com/community/ CommuniGate Pro] 5.2.0 || ? || ? || ? || () bugs || C - EXPUNGEs lost sometimes || 10/19 || || [http://www.ibm.com/software/lotus/products/notes/ IBM Domino] 8.0 || No || Unreliable || ? || SMALLER/LARGER not supported || C - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors || 16/20 || || [http://www.kerio.eu/kms_home.html Kerio Mail Server] 6.5.0 || ? || ? || ? || Bugs || EXPUNGEs not sent || 16/20 || |
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failures''' || ||[[http://www.dbmail.org/|dbmail]] 3.0.2 ||? ||Unreliable ||? ||Yes ||Yes || 9/361 - UID/sequence mapping becomes wrong || ||[[http://www.ibm.com/software/lotus/products/notes/|IBM Domino]] 8.0 ||No ||Unreliable ||? ||? ||No ||16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors || ||[[http://www.kerio.eu/kms_home.html|Kerio Mail Server]] 6.5.1 ||? ||? ||? ||? ||? ||18/34 - EXPUNGEs are sent wrong || ||[[http://www.axigen.com/|Axigen]] 7.1.2 ||? ||Unreliable ||Yes ||Broken ||Broken ||19/312 - FETCH/STORE sends EXPUNGEs immediately || |
IMAP Server Compliancy Status
- Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical.
- It would actually be better to be consistent across all IMAP commands, but especially when NOOP or IDLE is used. Some servers have inconsistent IMAP session states because the client connections end up in e.g. different servers that don't synchronize immediately. This can cause problems with clients that rely on the different sessions seeing the same state.
- \Recent: Exactly one session sees a new message as \Recent - no more and no less.
imaptest prints an error if it notices multiple sessions having the same \Recent flag ("Message seq=.. UID=.. has \Recent flag in multiple sessions"). This happens automatically when running imaptest with multiple clients for the same user, e.g. imaptest user=testuser clients=10
- But missing \Recent flags isn't detected with any tests
- Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other
Test by running stress test with own_flags parameter and multiple clients for the same user, e.g. imaptest user=testuser clients=10 own_flags
- Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents?
- Yes: Yes, everything can be fetched
- Cached: Only cached data can be fetched. Message header or body can't be.
- No: Nothing but flags can be fetched.
- This needs to be tested manually to see how it behaves - there is no ready imaptest test for this.
- Expunge store: If message is expunged in one session, can another session update its flags?
- Yes: Yes, and a 3rd session also sees the change
- Session: Yes, but a 3rd session won't see the change
Delayed: Yes, but a 3rd session won't see any changes until EXPUNGEs are reported
- Ignored: STORE command replies OK, but no change is made
- No: STORE command replies NO
- This needs to be tested manually to see how it behaves - there is no ready imaptest test for this.
Failures (failures/total): Number of failures using scripted tests. These numbers may not be exact all the time, because the tests are still changing.
- Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands.
- Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
- Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers.
Test using e.g. imaptest user=testuser test=tests/
Fully compliant servers:
Server |
Checkpoint |
\Recent |
Atomic flags |
Expunge fetch |
Expunge store |
Failure groups |
Base fails |
Ext fails |
Yes |
Yes |
Yes |
Yes / Cached (depends on storage) |
Yes |
0/40 |
0/403 |
0/100 |
|
Panda IMAP 2008, mix format |
Yes |
Yes |
Yes |
Yes |
Session |
0/34 |
0/328 |
0/97 |
SurgeMail 5.0h3 |
Yes |
Yes |
Yes |
Yes |
Yes |
0/35 |
0/342 |
0/26 |
Non-compliant servers:
Server |
Checkpoint |
\Recent |
Atomic flags |
Expunge fetch |
Expunge store |
Failure groups |
Base fails |
Ext fails |
UW-IMAP 2007b, mix format |
Yes |
Yes |
Yes |
Yes |
Session |
2/34 |
0/328 |
6/53 |
Isode M-Box 14.3a0 |
No |
Unreliable |
Yes |
Cached |
Ignored |
4/40 |
1/408 |
8/112 |
CommuniGate Pro 5.2.1 |
Yes |
Yes |
Yes |
Cached (some) |
No |
8/34 |
8/328 |
0/0 |
Cyrus 3.0.10 |
No |
Unreliable |
Bugs |
Cached |
Delayed |
0/35 |
0/366 |
0/100 |
Sun Java Messenging Server 6.3-0.15 |
No |
Unreliable |
Bugs |
Yes |
Delayed |
9/34 |
17/328 |
9/21 |
Archiveopteryx 3.0.3 |
No |
Unreliable |
No |
No |
No |
13/38 |
25/346 |
6/26 |
Courier 4.3.1 |
Yes |
Unreliable |
Yes |
No |
No |
18/34 |
33/328 |
20/53 |
GMail 2012-08-12 |
No |
Not implemented |
Bugs |
Cached (some) |
Ignored |
12/49 |
66/376 |
0/0 |
No |
Unreliable |
Yes |
No |
No |
6/35 |
14/340 |
0/0 |
|
Zimbra 5.0.5 |
Yes |
Yes |
Yes |
Cached |
No |
8/34 |
50/328 |
2/33 |
No |
Unreliable |
? |
No |
No |
16/40 |
52/287 |
0/0 |
|
Citadel 7.36 |
No |
Unreliable |
Yes |
Yes |
Session (flag changes are never seen by other sessions) |
19/34 |
98/328 |
0/0 |
hMailServer 5.3.3 b1879 |
? |
? |
? |
? |
? |
Fails hardcoded OK response format test. Other tests don't work, if OK response test fails. |
Major problems with multiple connections (makes further testing difficult and MAY CAUSE ACCIDENTAL MAIL LOSS!):
Server |
Checkpoint |
\Recent |
Atomic flags |
Expunge fetch |
Expunge store |
Failures |
dbmail 3.0.2 |
? |
Unreliable |
? |
Yes |
Yes |
9/361 - UID/sequence mapping becomes wrong |
IBM Domino 8.0 |
No |
Unreliable |
? |
? |
No |
16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors |
Kerio Mail Server 6.5.1 |
? |
? |
? |
? |
? |
18/34 - EXPUNGEs are sent wrong |
Axigen 7.1.2 |
? |
Unreliable |
Yes |
Broken |
Broken |
19/312 - FETCH/STORE sends EXPUNGEs immediately |