Differences between revisions 107 and 108
Revision 107 as of 2023-02-23 19:59:16
Size: 6978
Editor: TimoSirainen
Comment:
Revision 108 as of 2023-11-07 21:14:12
Size: 98
Editor: TimoSirainen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
 * Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical.
  * It would actually be better to be consistent across all IMAP commands, but especially when NOOP or IDLE is used. Some servers have inconsistent IMAP session states because the client connections end up in e.g. different servers that don't synchronize immediately. This can cause problems with clients that rely on the different sessions seeing the same state.
 * \Recent: Exactly one session sees a new message as \Recent - no more and no less.
  * imaptest prints an error if it notices multiple sessions having the same \Recent flag ("Message seq=.. UID=.. has \Recent flag in multiple sessions"). This happens automatically when running imaptest with multiple clients for the same user, e.g. `imaptest user=testuser clients=10`
  * But missing \Recent flags isn't detected with any tests
 * Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other
  * Test by running stress test with own_flags parameter and multiple clients for the same user, e.g. `imaptest user=testuser clients=10 own_flags` - If no errors are printed then it's likely ok.
 * Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents?
  * Yes: Yes, everything can be fetched
  * Cached: Only cached data can be fetched. Message header or body can't be.
  * No: Nothing but flags can be fetched.
  * This needs to be tested manually to see how it behaves - there is no ready imaptest test for this. Manual testing with IMAP:
   * Open two IMAP sessions and `SELECT INBOX` in both
   * Session 1: `STORE 1 +FLAGS \Deleted`, `EXPUNGE`
   * Session 2: `FETCH 1 BODYSTRUCTURE` -> Success = Cached
   * Session 2: `FETCH 1 BODY.PEEK[]` -> Success = Yes
 * Expunge store: If message is expunged in one session, can another session update its flags?
  * Yes: Yes, and a 3rd session also sees the change
  * Session: Yes, but a 3rd session won't see the change
  * Delayed: Yes, but a 3rd session won't see '''any''' changes until EXPUNGEs are reported
  * Ignored: STORE command replies OK, but no change is made
  * No: STORE command replies NO
  * This needs to be tested manually to see how it behaves - there is no ready imaptest test for this. Manual testing with IMAP:
   * Open three IMAP sessions and `SELECT INBOX` in all
   * Session 1: `STORE 1 +FLAGS \Deleted`, `EXPUNGE`
   * Session 2: `STORE 1,2 +FLAGS \Seen`
   * Session 3: `FETCH 1,2 FLAGS`
    * If neither mail shows \Seen flag, run in Session 3 `NOOP` and `FETCH 1 FLAGS`. This should result in EXPUNGE notification and mail 2 showing \Seen flag. This result is `Delayed`.
 * Failures (failures/total): Number of failures using [[ImapTest/ScriptedTests|scripted tests]]. These numbers may not be exact all the time, because the tests are still changing.
  * Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands.
  * Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
  * Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers.
  * Test using e.g. `imaptest user=testuser test=tests/`
Fully compliant servers:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.dovecot.org/|Dovecot]] ||Yes ||Yes ||Yes ||Yes / Cached (depends on storage) ||Yes ||0/40 ||0/403 ||0/100 ||
||[[https://github.com/jonabbey/panda-imap/|Panda IMAP]] 2008, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||0/34 ||0/328 ||0/97 ||
||[[http://netwinsite.com/surgemail/|SurgeMail]] 5.0h3 ||Yes ||Yes ||Yes ||Yes ||Yes ||0/35 ||0/342 ||0/26 ||
Line 41: Line 3:



Non-compliant servers:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.washington.edu/imap/|UW-IMAP]] 2007b, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||2/34 ||0/328 ||6/53 ||
||[[http://www.isode.com/products/m-box.html|Isode M-Box]] 14.3a0 ||No ||Unreliable ||Yes ||Cached ||Ignored ||4/40 ||1/408 ||8/112 ||
||[[http://www.communigate.com/community/|CommuniGate Pro]] 5.2.1 ||Yes ||Yes ||Yes ||Cached (some) ||No ||8/34 ||8/328 ||0/0 ||
||[[https://www.cyrusimap.org/|Cyrus]] 3.0.10 ||No ||Unreliable ||Bugs ||Cached ||Delayed ||0/35 ||0/366 ||0/100 ||
||[[http://www.sun.com/software/products/messaging_srvr/index.xml|Sun Java Messenging Server]] 6.3-0.15 ||No ||Unreliable ||Bugs ||Yes ||Delayed ||9/34 ||17/328 ||9/21 ||
||[[http://www.archiveopteryx.org/|Archiveopteryx]] 3.0.3 ||No ||Unreliable ||No ||No ||No ||13/38 ||25/346 ||6/26 ||
||[[http://www.courier-mta.org/imap/|Courier]] 4.3.1 ||Yes ||Unreliable ||Yes ||No ||No ||18/34 ||33/328 ||20/53 ||
||[[http://www.gmail.com/|GMail]] 2012-08-12 ||No ||Not implemented ||Bugs ||Cached (some) ||Ignored ||12/49 ||66/376 ||0/0 ||
||[[http://www.zarafa.com/|Zarafa Collaboration Platform]] 7.1.10 ||No ||Unreliable ||Yes ||No ||No ||6/35 ||14/340 ||0/0 ||
||[[http://www.zimbra.com/|Zimbra]] 5.0.5 ||Yes ||Yes ||Yes ||Cached ||No ||8/34 ||50/328 ||2/33 ||
||[[http://www.microsoft.com/exchange/default.mspx|MS Exchange]] ||No ||Unreliable ||? ||No ||No ||16/40 ||52/287 ||0/0 ||
||[[http://www.citadel.org/|Citadel]] 7.36 ||No ||Unreliable ||Yes ||Yes ||Session (flag changes are never seen by other sessions) ||19/34 ||98/328 ||0/0 ||
||[[http://www.hmailserver.com/|hMailServer]] 5.3.3 b1879 ||? ||? ||? ||? ||? ||||||<style="text-align: center;">Fails hardcoded OK response format test. Other tests don't work, if OK response test fails. ||




Major problems with multiple connections (makes further testing difficult and MAY CAUSE ACCIDENTAL MAIL LOSS!):
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failures''' ||
||[[http://www.dbmail.org/|dbmail]] 3.0.2 ||? ||Unreliable ||? ||Yes ||Yes || 9/361 - UID/sequence mapping becomes wrong ||
||[[http://www.ibm.com/software/lotus/products/notes/|IBM Domino]] 8.0 ||No ||Unreliable ||? ||? ||No ||16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors ||
||[[http://www.kerio.eu/kms_home.html|Kerio Mail Server]] 6.5.1 ||? ||? ||? ||? ||? ||18/34 - EXPUNGEs are sent wrong ||
||[[http://www.axigen.com/|Axigen]] 7.1.2 ||? ||Unreliable ||Yes ||Broken ||Broken ||19/312 - FETCH/STORE sends EXPUNGEs immediately ||
Moved to https://dovecot.github.io/imaptest/compliancy.html

IMAP Server Compliancy Status

Moved to https://dovecot.github.io/imaptest/compliancy.html

None: ImapTest/ServerStatus (last edited 2023-11-07 21:14:12 by TimoSirainen)