Sign in Subscribe

Jali: Postfix mail retention system

Jali: Postfix mail retention system
Photo by Nana Smirnova / Unsplash

Mail servers have become a rare skill these days. Most of us do not think its worthwhile to take the effort to setup a mail. I have a soft spot for it(started my career with Qmail in 2021!). One of the reasons i enjoy a mailing system is the reliability. The damn thing just works!


The pain of hosting your own is spam and deliverabilty. Still, I find it worth the effort. Once I got to speed with mail servers, archival became the enemy. Why? First of all, there are no good free software alternatives for it. Mailpiler community version exists but, the database keeps growing and purging mails just doesn't work. I have not tested enterprise version and using community to backup my personal mails is an overkill.

This is when i asked myself a question,

"How can i call myself a hacker if i don't even try to build a solution? I'm creating one".

Unfortunately, this motivation did not last long. I had to build a similar system at work and a complete solution with UI would consume too much time.

"What will i do? Well, how does archival work?"

"Mails forwarded from the server gets stored in archical database. I need an equivalent solution but without hassle... wait a minute... DAMN!! I can use Postfix!".

This go me high and ecstatic. Back in college, i had always enjoyed being a mood killer. Never thought it would come back to bite my ass,

"What about the UI?". Damn you! "Once i complete the SMTP part, I'll work on it! Scripts or something, IDK!!" (Hope I'm forced)

Inner peace.

Enough digressions, let's double click on the retention system. This system is configured such that it will archive mails for a server with multiple domain configured. Solution for servers with single domain is much simpler.

There are 2 components involved

  1. Mail server: Rewrite addresses and forward all mails from server to archival
  2. Jali: Receive from mail server, rewrite addresses and store in archive mailbox

First, we create lookup tables, copy of virtual_alias_table will do, to rewrite addresses to sender+emailid & rcpt+emailid format. LDAP user backend is used in this example. The default delimiter value in Postfix is +.

Different keywords are used for sent and received mails so that they can be distinguished from one another.

archive_recipients.cf

 bind = yes
 bind_dn = cn=admin,dc=example,dc=com 
 bind_pw = admin_password 
 server_host = ldap_ip
 search_base = dc=example,dc=com 
 query_filter = (&(objectClass=qmailUser)(mail=%s))
 result_attribute = mail 
 result_format = rcpt+%s

archive_senders.cf

 bind = yes
 bind_dn = cn=admin,dc=example,dc=com 
 bind_pw = admin_password 
 server_host = ldap_ip
 search_base = dc=example,dc=com 
 query_filter = (&(objectClass=qmailUser)(mail=%s))
 result_attribute = mail 
 result_format = sender+%s

These lookup tables are set to sender_bcc_maps and recipient_bcc_maps for rewriting addresses for all sender and recipient BCC's as the parameter suggests. These parameters are configure in main.cf with necessary lookup tables.

main.cf

 recipient_bcc_maps = ldap:/etc/postfix/ldap/archive_users.cf
 sender_bcc_maps = ldap:/etc/postfix/ldap/archive_senders.cf`

We can test tables with postmap command

$ postmap -q test@example.com ldap:/etc/postfix/ldap/archive_users.cf
rcpt+test@example.com

Next, the rewrote addresses must be forwarded to archive server. Postfix transport is configured to forward mails to archival system.

rcpt@example.com smtp:[jali_ip]:25
sender@example.com smtp:[jali_ip]:2525

When multiple domains are allowed in the server, let's say you are accepting mails for example.com and example2.com , sent and received mails forwarded to port 25 store both mails in INBOX. Sent mails with id sender+emailid will not be part of Received flag in mail headers making sieve unable to filter. In order to sort this issue, two receiving ports, 25 and 2525, are configured in Jali to accept received and sent mails. Well, it might not be the fanciest and coolest thing, but it works.

Let me explain how Jali works. It gets mail in sender+emailid,rcpt+emailid format, rewrites it to emailid and then delivers to users mailbox. We can use the same user, mailbox... lookup tables configured in our existing mail server.

Port 2525 can be configured by editing master.cf file in postfix

# Port 25
smtp      inet  n       -       y       -       -       smtpd
  -o smtpd_tls_security_level=may
  -o smtpd_tls_auth_only=no
# Port 2525
2525 inet  n       -       y       -       -       smtpd
  -o smtpd_tls_security_level=may
  -o smtpd_tls_auth_only=no

Once the ports are enabled we can configure address rewriting. Following conditions must be met for mailbox delivery

  • We need to rewrite the envelope recipient
  • Only rewrote addresses can be passed to virtual maps for delivery

Hence, Canonical Address Mapping is used. Canonical mapping can be used to rewrite To/From addresses in SMTP Envelope as well as Message Headers. Postfix has separate configuration option for senders (sender_canonical_maps) and recipients (recipient_canonical_maps) as well as both (canonical_maps). We will only configure for recipients.

Example configuration

canonical_classes = envelope_recipient
recipient_canonical_maps = pcre:/etc/postfix/mails.pcre
virtual_alias_maps = proxy:ldap:/etc/postfix/ldap/aliases.cf,proxy:ldap:/etc/postfix/ldap/groups.cf
virtual_mailbox_domains = proxy:ldap:/etc/postfix/ldap/domains.cf
virtual_mailbox_maps = proxy:ldap:/etc/postfix/ldap/recipients.cf
smtpd_sender_login_maps = proxy:ldap:/etc/postfix/ldap/senders.cf
local_recipient_maps = $virtual_mailbox_maps

Create a PCRE table named mails.pcre for recipient_canonical_maps 

/^(rcpt\+)(.*)@example\.com$/  ${2}@example.com
/^(sender\+)(.*)@example\.com$/  ${2}@example.com

This will ensure that forwarded mails are converted to required format.

Mails sent from our server should be stored in users .Sent folder. In order to achieve this, we will create a global sieve script.

Add following lines to 90-sieve.conf in dovecot to enable sieve and global scripts

plugin {
  sieve_default = file:/home/sieve;active=/home/sieve/.dovecot.sieve
  sieve_before= /home/sieve/.dovecot.sieve
  sieve_global = /home/sieve/global
}

Next, we configure the active script ie /home/sieve/.dovecot.sieve. Sieve will only use scripts included in this file.

require ["include"];
include :global "sent_mails";

Finally, we create a sent_mail.sieve file in global folder. Following script will move mail into Sent folder. Sieve will autocreate if folder does not exists

require ["fileinto", "mailbox", "regex", "envelope", "index", "subaddress", "imap4flags"];

if header :matches ["Received"] "*<sender+*>*" {
    fileinto :create "Sent";
    stop;
}

The setup is complete and mails will be stored in following directories

Incoming => $USER_DIRECTORY/Maildir/new
Sent => $USER_DIRECTORY/Maildir/.Sent/new

Jali logs

2025-02-14T00:00:57.144790+05:30 Mail-archival dovecot: lmtp(test@example.com)<606297><gMWNBKVAr2dZQAkAxOySnA>: sieve: msgid=<1739538597.1457@example.com>: fileinto action: stored mail into mailbox 'Sent'
2025-02-14T00:00:57.147442+05:30 Mail-archival postfix/lmtp[606508]: 0A2E647386: to=<test@example.com>, orig_to=<sender+test@example.com>, relay=0.0.0.0[0.0.0.0]:24, delay=0.11, delays=0.02/0.02/0/0.07, dsn=2.0.0, status=sent (250 2.0.0 <test@example.com> gMWNBKVAr2dZQAkAxOySnA Saved)

2025-02-14T00:00:59.738140+05:30 Mail-archival dovecot: lmtp(test@example.com)<606376><GLv/J6RAr2eoQAkAxOySnA>: sieve: msgid=<1739538597.1457@example.com>: stored mail into mailbox 'INBOX'
2025-02-14T00:00:59.738705+05:30 Mail-archival postfix/lmtp[606349]: A123F4738E: to=<test@example.com>, orig_to=<rcpt+test@example.com>, relay=0.0.0.0[0.0.0.0]:24, delay=0.09, delays=0.02/0/0/0.07, dsn=2.0.0, status=sent (250 2.0.0 <test@example.com> GLv/J6RAr2eoQAkAxOySnA Saved)