Somehow I have missed configuring logrotate to do its magic on the Dovecot log files, lets fix that.
Create /etc/logrotate.d/dovecot
and put this inside:
/var/log/mail/dovecot*.log {
missingok
monthly
notifempty
delaycompress
sharedscripts
postrotate
doveadm log reopen
endscript
}
To force the rotation, use this command:
$ logrotate --force /etc/logrotate.conf