This documentation is for Dovecot v1.x, see wiki2 for v2.x documentation.

Zlib plugin

Zlib plugin can be used to read compressed mbox and maildir files. Zlib plugin supports compression using zlib/gzip and bzlib/bzip2 (v1.1.2+).

You can enable zlib plugin by just loading it, there is no other configuration:

protocol imap {
  ..
  mail_plugins = zlib
}
protocol pop3 {
  ..
  mail_plugins = zlib
}

mbox

Compressed mbox files can be accessed only as read-only. The compression is detected based on the file name, so your compressed mboxes should end with .gz or .bz2 extension.

Maildir

When this plugin is loaded Dovecot can read both compressed and uncompressed files from Maildir. If you've enabled both gzip and bzip2 support you can have files compressed with either one of them in the Maildir. The compression is detected by reading the first few bytes from the file and figuring out if it's a valid gzip or bzip2 header. The file name doesn't matter. This means that an IMAP client can also try to exploit security holes in zlib/bzlib by writing specially crafted mails using IMAP's APPEND command. v1.2.5+ fixes this by not allowing clients to save mails that are detected as compressed.

All mails must have ,S=<size> in their filename where <size> contains the original uncompressed mail size, otherwise there will be problems with quota calculation as well as other potential random failures. Note that if the filename doesn't contain the ,S=<size> before compression, adding it afterwards changes the base filename and thus the message UID. The safest thing to do is simply to not compress such files.

You should also preserve the file's mtime so INTERNALDATE doesn't change.

Compression

You'll probably want to use some cronjob to compress old mails. However note that to avoid seeing duplicate mails in rare race conditions you'll have to use the included maildirlock utility (v1.1.2+). The idea is to:

  1. Find the mails you want to compress in a single maildir.
    • Skip files that don't have ,S=<size> in the filename.

  2. Compress the mails to tmp/

    • Update the compressed files' mtimes to be the same as they were in the original files (e.g. touch command)
  3. Run maildirlock <path> <timeout>. It writes PID to stdout, save it.

    • <path> is path to the directory containing Maildir's dovecot-uidlist (the control directory, if it's separate)

    • <timeout> specifies how long to wait for the lock before failing.

  4. If maildirlock grabbed the lock successfully (exit code 0) you can continue.
  5. For each mail you compressed:
    1. Verify that it still exists where you last saw it.
    2. If it doesn't exist, delete the compressed file. Its flags may have been changed or it may have been expunged. This happens rarely, so just let the next run handle it.
    3. If the file does exist, rename() (mv) the compressed file over the original file.

      • Dovecot can now read the file, but to avoid compressing it again on the next run, you'll probably want to rename it again to include e.g. a "Z" flag in the file name to mark that it was compressed (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ). Remember that the Maildir specifications require that the flags are sorted by their ASCII value, although Dovecot itself doesn't care about that.

  6. Unlock the maildir by sending a TERM signal to the maildirlock process (killing the PID it wrote to stdout).

Compression while saving (v2.0+)

Since v2.0+ Dovecot supports compression while saving mails (via LDA or IMAP APPEND command). There are two settings related to this:

plugin {
  zlib_save_level = 6 # 1..9
  zlib_save = gz # or bz2, If this config entry missing, compression is disabled. 
}

Don't forget to also enable zlib plugin (either at root level, or in lda and lmtp protocol sections):

mail_plugins = zlib

Per mailbox compression level via SQL

# When mailboxes.compression and mailboxes.compressionSave return Null, compression is disabled.
select mailboxes.home AS home, mailboxes.compression AS zlib_save_level, mailboxes.compressionSave as zlib_save from mailboxes where mailbox.address='%Lu'

None: Plugins/Zlib (last edited 2010-10-19 01:59:03 by staff-nat)