This documentation is for Dovecot v1.x, see wiki2 for v2.x documentation.
Differences between revisions 2 and 4 (spanning 2 versions)
Revision 2 as of 2006-03-30 05:19:01
Size: 802
Comment: fix typo
Revision 4 as of 2007-02-16 21:04:53
Size: 2522
Editor: TimoSirainen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== Reading problems == = mbox problems =

== External modifications ==

In general Dovecot doesn't mind if you modify the mbox file externally. It's fine if external software expunges messages or appends new ones. However moving around existing messages, inserting messages in the middle of the file or modifying existing messages isn't allowed.

Especially modifying existing messages (eg. removing attachments) may cause all kinds of problems. If you do that, at the minimum go and delete {{{dovecot.index.cache}}} file from the mailbox, otherwise weird things may happen. However IMAP protocol guarantees that messages don't change at all, and deleting Dovecot's cache file doesn't clear clients' local caches, so it still may not work right.

If you insert messages, or if you "undelete" messages (eg. replace mbox from a backup), you may see errors in Dovecot's logs:
Line 4: Line 12:
Error indexing mbox file /var/mail/user: LF not found where expected mbox sync: UID inserted in the middle of mailbox /home/tss/mail/inbox (817 > 787, seq=18, idx_msgs=32)
Line 7: Line 15:
This happens with Dovecot v0.99.x when there are multiple From-lines in row in the mbox, ie: This is normal. Dovecot just assigned new UIDs for the messages.

== Crashes ==

Dovecot's mbox code is a bit fragile because of the way it works. However instead of just corrupting the mbox file, it usually assert-crashes whenever it notices an inconsistency. You may see crashes such as:
Line 10: Line 22:
From some@one Wed Jan 5 09:03:48 2000
From another@one Wed Jan 5 09:09:25 2000
file mbox-sync-rewrite.c: line 404 (mbox_sync_read_and_move): assertion failed: (need_space == (uoff_t)-mails[idx].space)
Line 14: Line 25:
This shouldn't normally happen, and it's not good in any case to have them. Deleting one of them fixes the problem. Dovecot 1.0 releases don't have this problem anymore. This is a bit difficult problem to fix. Usually this crash has been related to Dovecot rewriting some headers that were broken. If you see these crashes, it would really help if you were able to reproduce the crash.
Line 16: Line 27:
== Corruption problems == If you have such a mailbox which crashes every time when it's tried to be opened, please put the mbox through [http://dovecot.org/tools/mbox-anonymize.pl mbox anonymizer] and send it, the mailbox's {{{dovecot.index}}} and {{{dovecot.index.log}}} files to tss@iki.fi. None of those files contain any actual message contents so it's be safe to send them.
Line 18: Line 29:
These are usually related to incompatible locking settings between all programs accessing the mboxes. Make sure they use the exact same locking methods. == Avoiding crashes ==
Line 20: Line 31:
=== Empty line at the beginning of mbox === Since the crashes usually have been related to broken headers, you should be able to avoid them by filtering out all the Dovecot's internal metadata headers. This is a good idea to do in any case. If you use [:LDA:Dovecot LDA] it does this filtering automatically. Otherwise you could do this in your SMTP server. The headers that you should filter out are:
Line 22: Line 33:
This is an unsolved rarely occurring bug in 0.99.x Dovecot versions. Use 1.0 releases instead.  * Content-Length
 * Status
 * X-IMAP
 * X-IMAPbase
 * X-Keywords
 * X-Status
 * X-UID
 * X-UIDL (if you're using {{{pop3_reuse_xuidl=yes}}})

mbox problems

External modifications

In general Dovecot doesn't mind if you modify the mbox file externally. It's fine if external software expunges messages or appends new ones. However moving around existing messages, inserting messages in the middle of the file or modifying existing messages isn't allowed.

Especially modifying existing messages (eg. removing attachments) may cause all kinds of problems. If you do that, at the minimum go and delete dovecot.index.cache file from the mailbox, otherwise weird things may happen. However IMAP protocol guarantees that messages don't change at all, and deleting Dovecot's cache file doesn't clear clients' local caches, so it still may not work right.

If you insert messages, or if you "undelete" messages (eg. replace mbox from a backup), you may see errors in Dovecot's logs:

mbox sync: UID inserted in the middle of mailbox /home/tss/mail/inbox (817 > 787, seq=18, idx_msgs=32)

This is normal. Dovecot just assigned new UIDs for the messages.

Crashes

Dovecot's mbox code is a bit fragile because of the way it works. However instead of just corrupting the mbox file, it usually assert-crashes whenever it notices an inconsistency. You may see crashes such as:

file mbox-sync-rewrite.c: line 404 (mbox_sync_read_and_move): assertion failed: (need_space == (uoff_t)-mails[idx].space)

This is a bit difficult problem to fix. Usually this crash has been related to Dovecot rewriting some headers that were broken. If you see these crashes, it would really help if you were able to reproduce the crash.

If you have such a mailbox which crashes every time when it's tried to be opened, please put the mbox through [http://dovecot.org/tools/mbox-anonymize.pl mbox anonymizer] and send it, the mailbox's dovecot.index and dovecot.index.log files to tss@iki.fi. None of those files contain any actual message contents so it's be safe to send them.

Avoiding crashes

Since the crashes usually have been related to broken headers, you should be able to avoid them by filtering out all the Dovecot's internal metadata headers. This is a good idea to do in any case. If you use [:LDA:Dovecot LDA] it does this filtering automatically. Otherwise you could do this in your SMTP server. The headers that you should filter out are:

  • Content-Length
  • Status
  • X-IMAP
  • X-IMAPbase
  • X-Keywords
  • X-Status
  • X-UID
  • X-UIDL (if you're using pop3_reuse_xuidl=yes)

None: MboxProblems (last edited 2009-03-15 22:35:12 by localhost)