Removing Duplicate Messages Using Procmail

Since we use the Postfix mail transport agent, it handles alias expansion in a non-centralized fashion. Due to how it handles aliases it never has a full view of all the recipients to prune duplicate entries. A solution to fixing this problem is to use procmail.

Procmail stores a configuration file in ~/.procmailrc. Procmail can be used to filter email. In this case, create a ~/.procmailrc and add the text below:

MAILDIR=$HOME/mail

:0 Wh: msgid.lock
| formail -D 8192 .msgid.cache

Then, type the commands:

mkdir ~/mail                # if ~/mail does not exist
chmod 700 ~/mail            # highly recommended 
touch ~/mail/.msgid.cache

This causes procmail to use the formail program to keep track of the last 8192 'Message-Id:' headers and make sure messages that have been seen do not end up in the INBOX or processed by the rest of the rules in the procmailrc file.