• TimeMovedBackwards

Time moved backwards error

Dovecot isn't very forgiving if your system's time moves backwards. There are usually two possibilities why it's moving backwards:

1. You're running ntpdate periodically. This isn't a good idea.

2. You're using some kind of a virtual server and you haven't configured it right (or it's buggy).

Moving time backwards might cause various problems (see below), so Dovecot versions older than v2.0 don't even try to handle the situation.

Time synchronization

There are two choices for synchronizing your clock:

1. Use ntpd. It periodically checks the current time from NTP server and slows down or speeds up the clock if necessary. Unlike ntpdate, it doesn't just move the time forwards or backwards (unless the difference is large).

   • If the time difference is too large for ntpd and it "steps", then use "-x" as a command line option for ntpd or use "tinker step 0" in /etc/ntp.conf.

     • This shows up in logs as: ntpd[17697]: time reset -2.075483 s

2. If ntpd doesn't work well (e.g. a bad network connection), you can use clockspeed or chrony as well.

If all else fails, you can just go and remove the error checking code from src/lib/ioloop.c. It's unlikely that anything will break badly, but you might get some errors logged once in a while.

In some systems ntpd/ntpdate is run at boot, but only after Dovecot has started. That can cause Dovecot to die immediately. If you have this problem, fix your init scripts to run ntpd/ntpdate first, before starting Dovecot. Also, seriously consider running ntp-wait before starting Dovecot.

Bugs/Issues

• With Xen you should run ntpd only in dom0. Other domains should synchronize time automatically (see this Xen FAQ and this thread).

• Time moved backwards by 4398 seconds? Buggy kernel/hardware.

What about Daylight Saving/Summer time?

On Unix-like systems, time is stored internally as the number of seconds since January 1, 1970, 00:00:00 UTC (see Unix_time on Wikipedia); concepts such as time zones and daylight saving time are applied in user space by the C library, and will normally not have an impact on Dovecot's behavior.

Dovecot shouldn't just die!

Dovecot v2.0 finally tries to handle this a bit more gracefully. Its behavior when time moves backwards is:

• Existing imap and pop3 processes either sleep or die, just like with older versions
• Master process stops creating new processes until either the original time is reached, or after a maximum wait of 3 minutes.
• Other processes log a warning, but do nothing else.
• Timeouts are updated so that the timeout is executed approximately at the original intended time.

Dovecot v2.0 also notices when time unexpectedly jumps forwards. In that situation it logs a warning and also updates timeouts.

The reason why imap/pop3 processes get killed and new ones can't be created for a while is to avoid problems related to timestamps. Some issues are:

• Uniqueness of Maildir filenames and dbox global unique identifiers relies on a growing timestamp
• Dotlock files' staleness is detected by looking at its mtime.
• Timestamps are stored internally all around in memory (as well as in index files) and compared to current time. Those checks may or may not be buggy if current time shrinks.

While killing mail processes doesn't fully solve any of those issues, they're at least less likely to happen then.

Create CRON restart task (Dovecot less than version 2.0)

• Create a task in roots crontab to execute /etc/init.d/dovecot every minute for the first 10 minutes after the daily timesync
• Every minute does sound very 'shotgun approach' however it ensures that however long it takes for ntpd to complete its task that dovecot will be restarted
• Dovecot will continue to kill itself until the delta time has been exceeded at which point it will run until the next ntpd cycle.

• Note if your daily timesync is > 10 minutes... you need a new server - forget worrying about dovecot

• This is a bandaid to avoid the result of clients not being able to download their email everyday due to a rather expeditious system clock.

Cron script to gracefully check Dovecot and restart if necessary

This script has been tested on a Centos 5.3 system. It should work on any GNU/Linux system. Make sure that lsof command is installed. If you are not running POP3 then changing the port to 143 for IMAP should work fine.

#!/bin/sh

HOST='localhost'
#PORT=110
PORT=143
#HP=@$HOST:$PORT
HP=:$PORT
echo 'Checking to see if Dovecot is up...'
if ( /usr/sbin/lsof -Pni $HP | grep "$PORT (LISTEN)" 2>&1 >/dev/null ); then
  echo 'Dovecot is up';
else
  echo 'Dovecot is down, restarting...';
  /etc/init.d/dovecot restart
  logger -p mail.info dovecot_check_restart.sh restarting Dovecot
fi

I use the root crontab line:

* * * * * /usr/local/sbin/dovecot_check_restart.sh 2>&1 > /dev/null

to run this script.

Or why not use netstat

#!/bin/sh

netstat -an|grep -ce ':993.*LISTEN' >/dev/null 2>&1

if [ $? = 0 ]
then
  echo 'Dovecot is up';
else
  echo 'Dovecot is down, restarting...';
  /etc/init.d/dovecot restart
  logger -p mail.info dovecot_keepalive: Dovecot is down, restarting...
fi

Even simpler with "nc":

#!/bin/sh

HOST='localhost'
PORT=143

nc -z $HOST $PORT 
if [ $? -ne 0 ]; then
  service dovecot restart 2>&1 > /dev/null
  logger -p mail.info dovecot_check_restart.sh restarting Dovecot
fi