Setting up a mail server using Postfix on Ubuntu 10.04

Introduction.

Background.

I had already build a VoIP system using CentOS (another Unix-like OS) as a base with trixbox CE providing the VoIP telephony. It was a fun project and for all the same reasons I wanted to do something similar for our messaging. I naturally (or some may argue un-naturally) started thinking about Microsoft Exchange, but was put off by the cost and all the additional features that I might not use! Then I remembered reading something about Postfix and thought “I can setup a VoIP system, surely it can’t be too hard to setup a mail server on a *nix box?” It also seemed like I nice way to create a rounded “communication” solution for our organisation encompassing telephony and email (my thoughts immediately turned to shared calendaring too).

With the availability of devices like the iPhone and iPad (yes, I’m a fan-boy!), it meant that I could create a mail platform that could be accessed “anywhere”, “anytime” on “any device” (and by any device read: iPad or iPhone ?). This was a plus for management, who wouldn’t need to be restricted to carrying around a laptop and 3G modem in order to access emails!

So I started searching on the internet for a cookbook that would outline the steps and one of the first I came across (and ultimately stayed with) was Ivar Abrahamsen’s step-by-step guide at flurdy.com. It’s a fantastic guide that outlines things very clearly with details on “how” and “why” each step is taken.

This HOWTO is effectively a duplicate (and to some degree a subset) of flurdy’s guide, with some of my own changes/omissions based on my own situation and experiences during the install. It was also to some degree intended for my own reference in case I needed to recover my server one day. All credit goes to Ivar Abrahamsen without whom I’d never have got my server off the ground. I didn’t plan on using my server in the cloud so I removed the references to ec2.

So why would you build your own mail server? There are a number of advantages to having your own mail server, and a cursory check of the MX records of a number of organisations I deal with revealed that almost everyone was doing it! Reason 1: I didn’t want to be left out!! More selfishly, it was a way to gain total control over the server, and with my “portable mail” plan relying on IMAP it meant I couldn’t have an ISP reigning me back on disk space!!

I did have concerns about being a target for spammers and hackers, so it was a relief to see the sections in flurdy’s guide that dealt with spam and antivirus as well as the encryption of the traffic – another plus if the aforementioned management decided to access their mail, using webmail from a internet café while on holiday!

Base OS Distribution

This HOWTO is based on Ubuntu Server 10.04 LTS (Lucid Lynx) 64-bit and its Debian base which uses apt-get (aptitude) to download and install its packages.

Why 10.04 and what is LTS? Lucid Lynx (10.04) is an LTS (Long Term Support) release. It (the server version) will be supported with security updates until April 2015. The long life cycle will hopefully mean a more stable platform, enhanced support and more time before we have to consider upgrading.

Hardware configuration

You know what they say about assumptions… Unfortunately I didn’t do enough research when picking my server hardware, and just went for the highest spec machine I could afford. Little did I know there would be issues around driver support for the RAID controller and GRUB’s inability to recognise GPT partitions for disks larger than 2TB! There are workarounds, but I had other options…

Here’s the specifications of the server I used for this exercise:

HP Proliant ML110 G6 with Intel G6950 (Dual-Core 2.8Ghz processor)
4GB of RAM
2 x 250GB SATA drives
2 x 2TB SATA drives

Never one to let road block get in the way of accomplishing a goal, I did the following steps in order to prepare the server for (and install) the OS:

First I disabled the onboard RAID and configured the four drives as individuals making sure the 250GB drives were first in the SATA chain.
Next, I followed the advanced Ubuntu setup here to configure software RAID using “mdadm”. There’s quite a good “cheat-sheet” here if you’re interested, otherwise a handy command is:
```
sudo cat /proc/mdstat
```
to see the various components of the array and
```
sudo mdadm --detail /dev/mdx
```
where x is 0,1,2…
When selecting the mount point for the array utilising the partitions on the 250GB drives I selected “/” and “/swap”.
When selecting the mount point for the array utilising the partitions on the 2TG drives I selected “/var” as this is the root for the location of the mail directory storage.
Installation was basically a “follow the prompts” procedure, other than the steps taken when partitioning the drives. I interrupted the network detection to define a fixed IP address, and when prompted didn’t install any additional software.
First boot after install was clean and good! No problems, with a little processing overhead as the RAID sync’d.

Software components

OS: Ubuntu Linux

Ah the age old distro argument… Why Ubuntu? It’s free, simple and slick with solid package management. As Ubuntu is derived from Debian the installations used here will be apt-get based..

MTA: Postfix

Simple, free, proven and slick. I agree with flurdy on this one – I’m a sucker for anything that works easily and like it was designed to. Postfix is all that! It’s powerful, well established, but not too bloated, and is security conscious from the start.

POP/IMAP: Courier IMAP

Simple, free and support for IMAPv4.

Database: MySQL

I haven’t had much exposure to database software other than the development I’ve done using MS SQL 2008 Express. MySQL, however, is well supported for the sort of lookups required in a mail server environment and these packages support it “out-of-the-box”.

Content Check: Amavisd-new

Easy plug-in content interface solution for spam, virus checking etc.

Anti-SPAM: SpamAssassin & Postgrey

SpamAssassin – A powerful and renowned SPAM fighting tool.
Postgrey – an excellent little script to stop 99% of all SPAM. On first contact for specific “from-to” combinations, it tells the sender server to try again in a little while, which (of course) most spammers can’t afford to do. When the proper servers try again after a few minutes delay, it lets the mail through.

Anti-Virus: ClamAV

A FREE virus scanner this is trusted and proven, which includes an update daemon.

Authentication: Cyrus SASL

Secure and trusted cryptography technology for authentication of SMTP traffic.

Encryption: TLS

Secure and trusted cryptography technology for encryption of SMTP traffic. Not to be confused with client encryption technology like GnuPG and S/MIME. Formerly referenced as SSL.

Webmail: Roundcube

For me interface is everything. It’s all about the “look-and-feel” baby! Roundcube is an “ajaxified” and prettier webmail client than SquirrelMail but not quite as solid… yet!

Software configuration

Repositories

For this install you need the main and universe repositories, but I also throw in the other “safe” ones: restricted and multiverse (and partner, when available) if they are not already defined.

 sudo vi /etc/apt/sources.list

Packages

There are a number of packages that constitute this installation. We will install and configure them individually, but before starting first check your package sources are correctly pointing to the main, multiverse, restricted and universe repositories of your current Ubuntu version as detailed above.

Once complete, install any updates for the base install.

sudo aptitude update
sudo aptitude safe-upgrade

MySQL

The first core component to install is MySQL.

sudo aptitude install mysql-client mysql-server

This will prompt you for a MySQL root password. Choose something secure and easy to remember (Current recommendations are for 12 alpha-numeric characters using case mixes). In this example we’ll set it to rootPASSWORD.

Postfix

Next install Postfix, assuming it wasn’t selected at the end of the Ubuntu install and similar steps taken to configure it.

sudo aptitude install postfix postfix-mysql

During the install it will prompt you to choose the “type” of email server. Select “internet site” as it is assumed this install will connect to the internet to send and receive email. The install will also suggest/request a server name. Alter as required using your FQDN or DynDNS equivalent.

openSSH Server

Because it’s easier to cut-and-paste using SSH telnet on your own PC, than keying it all in on the server console.

sudo aptitude install openssh-server

SASL

sudo aptitude install libsasl2-modules libsasl2-modules-sql libgsasl7 libauthen-sasl-cyrus-perl sasl2-bin libpam-mysql

ClamAV

sudo aptitude install clamav-base libclamav6 clamav-daemon clamav-freshclam

Amavis, SpamAssassin, postgrey

sudo aptitude install amavisd-new
sudo aptitude install spamassassin spamc
sudo aptitude install postgrey

Roundcube

sudo aptitude install roundcube roundcube-mysql php-pear php5-cli

phpMyAdmin

sudo aptitude install phpmyadmin

Select “Yes” to set it up and enter the root MySQL password defined above. Create a phpmyadmin MySQL user password, and repeat to confirm. When asked, accept “apache2” as the web server.

ShoreWall

sudo aptitude install shorewall-common shorewall-perl

Courier

sudo aptitude install courier-base courier-authdaemon courier-authlib-mysql courier-imap courier-imap-ssl courier-ssl

The install will prompt you about web directories. You can say “No” to this. It will also warn you about the certificate location. Ignore this.

Extras

I also install a few other packages to add functionality but have nothing to do with the mail server setup.

sudo aptitude install vim mutt lynx

vim – although I use Vi for editing files, VIM comes with more features if required.
mutt – a simple email client with support for MIME, IMAP, POP3, message threading etc…
lynx – is a text-based web browser for use on cursor-addressable character cell terminals and is very configurable (Yeah, I don’t know why either…)

Package status

At any stage to find out which packages you may have installed, you can use the following commands (substituting “postfix” with the package you’re checking):

sudo dpkg --list | grep postfix sudo aptitude search postfix

Stage One: A simple mail server

Now let’s configure a simple mail server using some of the packages installed previously. This HOWTO makes the assumption that you’re either NOT logged on as the root or root isn’t (or hasn’t) been enabled – hence the use of the “sudo” command.

Firewall

Shorewall

UFW is bundled with the 10.04 Ubuntu distribution, but I still prefer Shorewall for servers I setup. Basically at first you want to only allow SSH. Then SMTP & IMAP from your IP only. When you are confident that the mail server is secure, you can open SMTP and IMAP to the world.

Later on in the install you may wish to open web access to the webmail and admin GUI. Once again, this can be restricted to specific IPs.

SSH only

By default Shorewall in Ubuntu has an empty set up. You can find the default values for Shorewall in /usr/share/doc/shorwall/default-config, and examples of configurations in /usr/share/doc/shorwall/examples. To start, we will create the basic set up.

First configure which network adapters are accessing the net.

sudo cp /usr/share/doc/shorewall/default-config/interfaces /etc/shorewall/
sudo vi /etc/shorewall/interfaces net eth0 detect dhcp,tcpflags,logmartians,nosmurfs

Then we will configure network zones

sudo cp /usr/share/doc/shorewall/default-config/zones /etc/shorewall/
sudo vi /etc/shorewall/zones

Add the firewall if not there and the internet as a zone.

fw firewall
# loc ipv4
net ipv4

Then (if needed) specify hosts in the Shorewall hosts file. i.e. If you want to specify what your home IP is etc.

sudo cp /usr/share/doc/shorewall/default-config/hosts /etc/shorewall/
sudo vi /etc/shorewall/hosts # loc eth0:192.168.0.0/24

Then set what the default policy is for firewall access.

sudo cp /usr/share/doc/shorewall/default-config/policy /etc/shorewall/
sudo vi /etc/shorewall/policy $FW net ACCEPT

net $FW DROP info
net all DROP info
# The FOLLOWING POLICY MUST BE LAST
all all REJECT info

For safety in case it goes down.

sudo cp /usr/share/doc/shorewall/default-config/routestopped /etc/shorewall/
sudo vi /etc/shorewall/routestopped
eth0 0.0.0.0 routeback

For higher security you can use the netmask of your IP range if you’re more concerned.

The next step is to define the main firewall rules. You can find predetermined macro rules for Shorewall in the /usr/share/shorewall directory.

sudo cp /usr/share/doc/shorewall/default-config/rules /etc/shorewall/
sudo vi /etc/shorewall/rules

SSH/ACCEPT net $FW

Open for business?

Once your server is working properly, come back to this step and open up the required protocols to the net.

sudo vi /etc/shorewall/rules

Ping/ACCEPT net $FW
# Permit all ICMP traffic FROM the firewall TO the net zone.
ACCEPT $FW net icmp
# Add these for mail services.
SMTP/ACCEPT net $FW
SMTPS/ACCEPT net $FW
Submission/ACCEPT net $FW
IMAP/ACCEPT net $FW
IMAPS/ACCEPT net $FW
# Access for web clients.
Web/ACCEPT net $FW

Firewall configuring is always a risky business and it is easy to lock yourself out. To test the syntax of your configuration run

sudo shorewall check

Then to switch it on during boot

sudo vi /etc/default/shorewall

startup=1

Restart it with

sudo /etc/init.d/shorewall restart

For more details on IP Tables and Shorewall, reference the author’s website.

MTA – Mail Transfer Agent

Postfix

You should put the name of your server in this file…

sudo vi /etc/mailname

It could be something like smtp.domain.name, where domain.name is replaced with your own domain name.

Now open the main postfix configuration file:

sudo vi /etc/postfix/main.cf

Debian and Ubuntu by default include some sensible values in this file. You may need to comment some of them out if we duplicate with the commands below.

First specify the name of your server.

# This is already done in /etc/mailname
# myhostname= mail.example.com

NOTE:

After running my Postfix installation for a month or two I noticed in the message headers that the originating mail server was not what was defined in /etc/mailname but the local hostname – this was also confirmed by running postconf | grep myhostname. So… In my install I added the following entries in the main.cf file to accomodate.
myhostname = mail.example.com
mydomain = example.com

Next is the origin, which is the domain appended to each email sent from this machine. This can be your full servername, or just the DN.

# myorigin=/etc/mailname
myorigin=example.com

Next, decide what the greeting text will be when Postfix is queried. Provide enough info so it is useful, but not enough to divulge everything to potential hackers!

smtpd_banner = $myhostname ESMTP $mail_name

…or hardcode the server name to avoid a banner/hostname mismatch.

smtpd_banner = smtp.example.com ESMTP $mail_name

Next you need to decide whether to send all outgoing mail via another SMTP server, or send it yourself. I send via my ISP’s server so they have to worry about the queuing etc. If you send it yourself, then you are NOT reliant on a 3rd party’s server, BUT you may risk more exposure, being accidentally blocked by “spam blockers” and providing more work for your own server.

It’s also worth mentioning that many servers block dynamic DNS hosts (DynDNS etc), so you may find your server’s mail gets rejected. Weight up the pros and cons and choose a method that you are most comfortable with.

# leave blank to do it yourself.
relayhost =

# or put it an accessible smtp server.
relayhost = smtp.yourisp.com

Next are network details. We will define accepting connections from anywhere, and that we only trust this machine

inet_interfaces = all
mynetworks_style = host

Next you can masquerade some outgoing addresses. Say your machine’s name is mail.example.com. You may not want outgoing mail to come from username@mail.example.com, as you’d prefer the standard username@example.com. You can state which domain(s) not to masquerade (If you use a dynamic DNS service, then your server address will be a sub domain) and also specify which users not to masquerade.

# masquerade_domains = mail.example.com www.example.com !sub.dyndomain.com
# masquerade_exceptions = root

As this HOWTO describes using virtual domains, these need to be empty.

local_recipient_maps =
mydestination =

I’ve had problems with web based IMAP clients sending messages so I add the local IP subnet for the server (192.168.1.0/24) to the “mynetworks” variable.

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.1.0/24

Now we set a few numbers.

# How long if undelivered before sending warning update to sender.
delay_warning_time = 4h
# Will it be a permanent or temporary error.
unknown_local_recipient_reject_code = 450
# How long to keep messages on queue before returning as failed.
maximal_queue_lifetime = 7d
# MAX and MIN time in seconds between retries if connection has failed.
minimal_backoff_time = 1000s
maximal_backoff_time = 8000s
# How long to wait when servers connect before receiving the rest of the data.
smtp_helo_timeout = 60s
# How many addresses can be used in one message - effective stopper to mass spammers or
# accidental copy in of whole address list but may restrict intentional mail shots.
smtpd_recipient_limit = 16
# How many errors before server backs off.
smtpd_soft_error_limit = 3
# How many MAX errors before blocking it.
smtpd_hard_error_limit = 12

Now we can specify some restrictions. Be careful that each of these settings is on one line only.

# Requirements for the HELO statement
smtpd_helo_restrictions = permit_mynetworks, warn_if_reject reject_non_fqdn_hostname, reject_invalid_hostname, permit
# Requirements for the sender details.
smtpd_sender_restrictions = permit_mynetworks, warn_if_reject reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unauth_pipelining, permit
# Requirements for the connecting server.
smtpd_client_restrictions = reject_rbl_client sbl.spamhaus.org, reject_rbl_client blackholes.easynet.nl, reject_rbl_client dnsbl.njabl.org
# Requirement for the recipient address.
smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, permit
smtpd_data_restrictions = reject_unauth_pipelining

Note: No point in including dnsbl.njabl.org (as detailed above), as this was shutdown in 2013…[see here]

Further restrictions that are used to mitigate malicious use.

# Require proper HELO at each connection.
smtpd_helo_required = yes
# Waste spammers time before rejecting them
smtpd_delay_reject = yes
disable_vrfy_command = yes

Next we need to set some maps and lookups for the virtual domains. These will reference files that will contain access information for the MySQL databases that contain the relevant data

# Not sure of the difference of the next two but they are needed for local aliasing
alias_maps = hash:/etc/postfix/aliases
alias_database = hash:/etc/postfix/aliases
# This specifies where the virtual mailbox folders will be located.
virtual_mailbox_base = /var/spool/mail/virtual
# This is for the mailbox location for each user.
virtual_mailbox_maps = mysql:/etc/postfix/mysql_mailbox.cf
# and this is for aliases
virtual_alias_maps = mysql:/etc/postfix/mysql_alias.cf
# and this is for domain lookups
virtual_mailbox_domains = mysql:/etc/postfix/mysql_domains.cf
# This is how to connect to the domains (all virtual, but the option is there)
# ** NOT USED YET **
# transport_maps = mysql:/etc/postfix/mysql_transport.cf

You can use a lookup for the UID and GID of the owner of the mail files but I tend to have one owner (virtual), so instead add this.

virtual_uid_maps = static:5000
virtual_gid_maps = static:5000

You need to set up an alias file. This is only used locally, and not by your own mail domains.

sudo cp /etc/aliases /etc/postfix/aliases

… you may want to view the file to check if OK, especially if the final alias e.g. root goes to a real person…

sudo postalias /etc/postfix/aliases

Next you need to set up the folder where the virtual mail will be stored. This may have already been done by the apt-get. Also, create the user who will own the folders.

Use this to add if there is not a virtual user

sudo mkdir /var/spool/mail/virtual
sudo groupadd --system virtual -g 5000
sudo useradd --system virtual -u 5000 -g 5000
sudo chown -R virtual:virtual /var/spool/mail/virtual

Postfix’s MySQL configuration

We now need to set up the files to access the lookups via the MySQL database. We will only set up the essential files for now, and the rest later if/when needed.

NOTE:

The mailPASSWORD that you choose and define in the files below, will be configured in the next step titled “Database“. It is not the rootPASSWORD you created when installing MySQL.

Create the file that tells Postfix how to find the users mailbox location.

sudo vi /etc/postfix/mysql_mailbox.cf
user=mail
password=mailPASSWORD
dbname=maildb
table=users
select_field=maildir
where_field=id
hosts=127.0.0.1
additional_conditions = and enabled = 1

Create the file that tells Postfix how to find the email alias.

sudo vi /etc/postfix/mysql_alias.cf

user=mail
password=mailPASSWORD
dbname=maildb
table=aliases
select_field=destination
where_field=mail
hosts=127.0.0.1
additional_conditions = and enabled = 1

Create the file that tells Postfix how to find the domains.

sudo vi /etc/postfix/mysql_domains.cf

user=mail
password=mailPASSWORD
dbname=maildb
table=domains
select_field=domain
where_field=domain
hosts=127.0.0.1
additional_conditions = and enabled = 1

If you specify an IP in hosts file (as opposed to ‘localhost’) then it will communicate over TCP and not the MySQL socket (a chroot restriction).

Database

MySQL

The next step now is to create the tables for those lookups we just defined. First you need to create a user to use in MySQL for mail ONLY. Then you need to create the database. Make sure you take note of your chosen mail username and password. You will need the password you specified for root during the MySQL package installation.

If you haven’t already done this in package installation…

mysqladmin -u root password newPassword

mysql -u root -p

Then enter password for the root account when prompted. Now we create the mail database…

CREATE DATABASE maildb;

Then we create a new user called “mail”

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON maildb.* TO 'mail'@'localhost' IDENTIFIED BY 'mailPASSWORD';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON maildb.* TO 'mail'@'%' IDENTIFIED BY 'mailPASSWORD';
exit;

Obviously replace mailPASSWORD with your chosen password, and the one defined in the Postfix section above!

We’ve talked about them, defined them… now it’s time to actually create them. ‘Create what?’ you ask? The following tables of course:

aliases
domains
users

We will create more later on for further extensions, but for now only these are relevant.

Log into MySQL as the new mail user, connecting to the maildb database.

mysql -u mail -p maildb

Then run the following commands to create the specified tables. Quick side note here: It makes a difference if you use ` vs. ‘ so take note of which one is used in the commands below.

CREATE TABLE `aliases` ( `pkid` smallint(3) NOT NULL auto_increment, `mail` varchar(120) NOT NULL default '', `destination` varchar(120) NOT NULL default '', `enabled` tinyint(1) NOT NULL default '1', PRIMARY KEY (`pkid`), UNIQUE KEY `mail` (`mail`));

CREATE TABLE `domains` ( `pkid` smallint(6) NOT NULL auto_increment, `domain` varchar(120) NOT NULL default '', `transport` varchar(120) NOT NULL default 'virtual:', `enabled` tinyint(1) NOT NULL default '1', PRIMARY KEY (`pkid`));

CREATE TABLE `users` ( `id` varchar(128) NOT NULL default '', `name` varchar(128) NOT NULL default '', `uid` smallint(5) unsigned NOT NULL default '5000', `gid` smallint(5) unsigned NOT NULL default '5000', `home` varchar(255) NOT NULL default '/var/spool/mail/virtual', `maildir` varchar(255) NOT NULL default 'blah/', `enabled` tinyint(3) unsigned NOT NULL default '1', `change_password` tinyint(3) unsigned NOT NULL default '1', `clear` varchar(128) NOT NULL default 'ChangeMe', `crypt` varchar(128) NOT NULL default 'sdtrusfX0Jj66', `quota` varchar(255) NOT NULL default '', `procmailrc` varchar(128) NOT NULL default '', `spamassassinrc` varchar(128) NOT NULL default '', PRIMARY KEY (`id`), UNIQUE KEY `id` (`id`));

The last few fields in the ‘users’ table are not required, but useful if you decide to extend functionality later on.

To visualise the tables created, you can use the following command(s) when logged into the MySQL database.

DESCRIBE `aliases`;
DESCRIBE `domains`;
DESCRIBE `users`;
exit;

Next you need to edit the MySQL my.cnf file. In Ubuntu/Debian this is created by default. In Mandrake I had to manually create a blank one in /etc.

sudo vi /etc/mysql/my.cnf

In previous versions you needed to comment out this line #skip-networking. However in the latest version of the file the default is to bind the address to localhost, which is fine.

bind-address = 127.0.0.1

It is very useful at the start to log any SQL calls that make it to MySQL. To do this enable these lines.

general_log_file = /var/log/mysql/mysql.log
general_log = 1

Then in a few weeks when your sure everything is working OK, comment it out as it slows MySQL down

Once the changes are complete, restart MySQL to make sure it picks up the new settings.

sudo /etc/init.d/mysql restart

POP/IMAP

Courier IMAP

You need to change Courier to use MySQL and tell it where to find the information in our databases.

sudo vi /etc/courier/authdaemonrc

Change to MySQL mode.

authmodulelist="authmysql"

Further down in the file enable logging.

WARNING:

This level of logging will show passwords in cleartextin the syslog!

DEBUG_LOGIN=2

Now change/update user, password, database, table names, crypt, mail directory and ‘where’ clause…

sudo vi /etc/courier/authmysqlrc

MYSQL_USERNAME mail
MYSQL_PASSWORD mailPASSWORD
MYSQL_DATABASE maildb
MYSQL_USER_TABLE users
MYSQL_CRYPT_PWFIELD crypt
# MYSQL_CLEAR_PWFIELD clear
MYSQL_MAILDIR_FIELD concat(home,'/',maildir)
MYSQL_WHERE_CLAUSE enabled=1

Lastly you can have a look at the imapd file, but no changes are needed so it’s recommended to leave it as is.

vi /etc/courier/imapd

Stage Two: Advanced mail server

Content check interface

Amavisd-new

Amavisd-new is a high-performance interface between mailer (MTA) and content checkers: virus scanners, and/or SpamAssassin. The default values are a good start and the Ubuntu documentation is pretty clear with its recommendations.

Here is a tweaked version of those recommendations. Initially we will not enable spam or virus detection! This is so we can get amavis set up to receive, check and pass on emails before we go on to over-complicate it.

All of amavis’ configuration files are in /etc/amavisd. They are now broken down into several configuration files in conf.d/.

cd /etc/amavis/conf.d

The 01-debian, 25-amavis_helpers and 30-template-localization defaults are fine and it’s worth while having a look at the following files, but don’t change anything in them.

less 05-domain_id
less 05-node_id
less 15-av_scanners
less 20-debian_defaults
less 21-ubuntu_defaults

Edit content check file.

sudo vi 15-content_filter_mode

Comment out both virus and spam scan lines. (They are commented out by default).

#@bypass_virus_checks_maps = (
#   \%bypass_virus_checks, \@bypass_virus_checks_acl,/$bypass_virus_checks_re);
#@bypass_spam_checks_maps = (
#   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

Edit the user file.

sudo vi 50-user

In the middle of the file insert the following:

@local_domains_acl = qw(.);
$log_level = 2;
$syslog_priority = 'debug';
# $sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level
# $sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 8.0; # triggers spam evasive actions
# $sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent
$final_spam_destiny = D_PASS;
# $final_spam_destiny = D_REJECT; # default
# $final_spam_destiny = D_BOUNCE; # debian default
# $final_spam_destiny = D_DISCARD; # ubuntu default, recommended as sender is usually faked

We have now setup amavis to scan and pass along incoming emails. Next we will configure Postfix to talk to Amavis.

sudo vi /etc/postfix/master.cf

Append these lines to the end of the file but make sure they are not already present. (Note the -o lines have spaces in front of them).

amavis      unix    -       -       -       -       2       smtp
        -o smtp_data_done_timeout=1200
        -o smtp_send_xforward_command=yes
        -o disable_dns_lookups=yes
        -o max_use=20
127.0.0.1:10025 inet    n       -       -       -       -       smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_delay_reject=no
        -o smtpd_client_restrictions=permit_mynetworks,reject
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_data_restrictions=reject_unauth_pipelining
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o smtpd_error_sleep_time=0
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000
        -o smtpd_client_connection_count_limit=0
        -o smtpd_client_connection_rate_limit=0
        -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

… then add the following two lines immediately below the “pickup” transport service:

        -o content_filter=
        -o receive_override_options=no_header_body_checks

…and then add the reference to the content_filter in main.cf

sudo vi /etc/postfix/main.cf

content_filter = amavis:[127.0.0.1]:10024

We now enable scanning of amavis’ temporary files by ClamAV.

sudo adduser clamav amavis

This should be it to get Amavis-new working. If emails are picked up by Amavis-new and passed back to Postfix then everything should be okay.

Open for business?

Only when you have finished the next section (SpamAssassin) and testing, do you proceed to uncomment the anti-virus and anti-spam lines in 15-content_filter_mode

sudo vi 15-content_filter_mode

@bypass_virus_checks_maps = (
    \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
@bypass_spam_checks_maps = (
    \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

…and turn down the logging level, and start bouncing/discarding spam.

sudo vi /etc/amavis/conf.d/50-user

@local_domains_acl = qw(.);
$log_level = 1;
$syslog_priority = 'info';
# $sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level
# $sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 8.0; # triggers spam evasive actions
# $sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent
# $final_spam_destiny = D_PASS;
# $final_spam_destiny = D_REJECT; # default
# $final_spam_destiny = D_BOUNCE; # debian default
$final_spam_destiny = D_DISCARD; # ubuntu default, recommended as sender is usually faked

Anti-SPAM

SpamAssassin

The default configuration of SpamAssassin is fine, but you do need to tell SpamAssassin to start “smapd” on boot.

sudo vi /etc/default/spamassassin

ENABLED=1

One configuration option you could tweak is to enable “Bayes” and “Bayes auto-learning“.

sudo vi /etc/spamassassin/local.cf

# Use Bayesian classifier (default: 1)
#
use_bayes 1
# Bayesian classifier auto-learning (default: 1)
#
bayes_auto_learn 1

Postgrey

The default configuration of Postgrey is fine. However you do need to tell Postfix to use it.

sudo vi /etc/postfix/main.cf

…then edit the recipient restrictions setting:

smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, check_policy_service inet:127.0.0.1:10023, permit

You can then tweak whitelisting in /etc/postgrey if you wish. You can also tweak the Postgrey configuration by editing /etc/default/postgrey. For example you could change delay, auto whitelisting, or message rejection.

POSTGREY_OPTS="--inet=10023 --max-age=365"

Anti-VIRUS

ClamAV

ClamAV does not need any setting up. The configuration files are located in /etc/clamav, but they are automatically generated, so do not edit them. By default freshclam, the daemon that updates the virus definition database, is run 24 times a day(!!). That seems a little excessive, so I tend to set that to once a day.

sudo dpkg-reconfigure clamav-freshclam

Running this “reconfigure” will also prompt you whether you want it to be daemon (answer: Yes) and which server is closest to you. If needed, the command below will redefine the entire configuration with a lot of questions/prompting. It’s not needed unless you really need to configure ClamAV to the nth degree.

sudo dpkg-reconfigure clamav-base

Stage Three: Secure your mail server

SSL Certificates… or “How I lost my mind for a week”

Don’t cry little baby…

OK, I confess I had a hell of time getting my head around the certificates for the traffic encryption. I think it was compounded by the fact that I didn’t think I was doing anything out of the ordinary, but I couldn’t seem to find a solid set of steps to follow, and instead had to amalgamate a number of different procedures from different sources to get it working (at least I think it’s working?!). It also doesn’t help when some clients are more forgiving than others about how they handle incorrect certificates.

In the end it was Microsoft’s obtuse Outlook 2007 that helped me the most, refusing to work until I’d got the certificate details sorted. Well, to help save you some pain, hopefully the following steps will assist you (…and your sanity!)

Down to business

Ubuntu will have created it’s own self-signed certificate in /etc/ssl/certs with the key in /etc/ssl/private called ssl-cert-snakeoil.pem and ssl-cert-snakeoil.key respectively. I think these are fine if you’re only going to use the server locally, but once it’s opened up to the internet, clients will complain about the certificate validity (at least mine did!)

The two processes that will use the certificates – in context of this HOWTO – are Postfix and Courier. The instructions are based to some degree on the Ubuntu documentation here. I created a directory called certs in my admin user’s home directory so I could keep everything together.

First we create a file with the encrypted private server key in it and this will form the basis of certificate generation. I’ve used 2048 as the key length based on security recommendations at the time of writing this (Sept 2010). This command will prompt you for a server passphrase – the longer the phrase the better, but the minimum for DES3 is four (4).

cd ~/certs
sudo openssl genrsa -des3 -out server.key 2048

Now create the insecure key, the one without a passphrase, and shuffle the key names.

sudo openssl rsa -in server.key -out server.key.insecure
sudo mv server.key server.key.secure
sudo mv server.key.insecure server.key

The insecure key is now named server.key, and you can use this file to generate the CSR without a passphrase stored in it. To create the CSR, run the following command. You can combine steps within the one command to create the certificate without using a CSR, but if you decide to get an SSL certificate from a CA, having the CSR will come in handy.

sudo openssl req -new -key server.key -out server.csr

This will now prompt you for some certificate information like country, location, company name etc.

IMPORTANT:

Make sure that you use the server’s FQDN when prompted for the common name (CN). In most of the examples we’ve referred to it as smtp.example.com (or similar) and will continue to use that in the following steps.

You can now submit this CSR file to a CA (Verisign, Comodo etc…) for processing. The CA will use this CSR file and issue the certificate but we’re going to use it to create a self-signed certificate. It’s up to you how many days you create make the certificate valid for, the default is 365 days.

sudo openssl x509 -sha256 -req -days 999 -in server.csr -signkey server.key -out server.crt

Fantastic… we have a certificate and a key, but PEM format certificates (a certificate that may include just the public certificate, or may include an entire certificate chain including public key, private key, and root certificates. The name is from Privacy Enhanced Email, a failed method for secure email but the container format it used lives on.) often include the key and the certificate in one file. Combining them is simple…

sudo cat server.key server.crt >> server.pem

Next, we copy some of these files to the relevant locations where they will be used.

sudo cp server.crt /etc/ssl/certs/smtp.example.com.pem
sudo cp server.key /etc/ssl/private/smtp.example.com.key
sudo cp server.pem /etc/courier/imapd.pem

That’s pretty much it for the certificates. The next step is to secure your mail server using the certificate we’ve just created.

Some helpful commands

To view the contents of a PEM (Privacy Enhanced Mail) certificate

sudo openssl x509 -in certname.pem -noout -text

Create a public key, using your private key (using the files created above…)

sudo openssl rsa -in server.pem -pubout -out server.public

Create a certificate file for import in Windows certificate manager (important if the certificate is self-signed).

sudo openssl x509 -in certname.pem -out certname.crt

Create a self-signed certificate with a “auto-generated” key and write it out to a PEM format file.

sudo openssl req -x509 -newkey rsa:2048 -keyout certname.pem -out certname.pem -nodes -days 999

Configuring authentication

SASL

SASL secures the actual authentication (login), by encoding the passwords so that it can not be easily intercepted. The rest of the emails are however in clear plain text.

SASL can be a royal pain to set up, especially as it does not support storing encrypted passwords by default in Ubuntu.

Obviously this is not ideal, so there are ways to combine SASL and storing encrypted passwords. In the future the packages that come with Ubuntu may support the password_format configuration option for SASL, but until then you can configure SASL to ask PAM to compare the passwords.

Install packages if not all installed already…

sudo aptitude install sasl2-bin libpam-mysql libsasl2-modules libsasl2-modules-sql

Enable postfix to access SASL files.

sudo adduser postfix sasl

Create SASL directories which will be chrooted Postfix.

sudo mkdir -p /var/spool/postfix/var/run/saslauthd

Add SASL configurations to Postfix.

sudo vi /etc/postfix/main.cf

# SASL
smtpd_sasl_auth_enable = yes
# If your clients use Outlook Express or other older clients then
# broken_sasl_auth_clients needs to be set to yes
broken_sasl_auth_clients = no
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =

Modify these existing configurations

# Add permit_sasl_authenticated to you existing smtpd_sender_restrictions
smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks,warn_if_reject reject_non_fqdn_sender, reject_unknown_sender_domain,reject_unauth_pipelining, permit
# Add permit_sasl_authenticated to you existing smtpd_recipient_restrictions
smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks,permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unknown_recipient_domain,reject_unauth_destination, check_policy_service inet:127.0.0.1:10023, permit

Change how saslauthd is run.

sudo vi /etc/default/saslauthd

START=yes

OPTIONS="-r -c -m /var/spool/postfix/var/run/saslauthd"

Now tell Postfix how to interact with SASL.

sudo vi /etc/postfix/sasl/smtpd.conf

pwcheck_method: saslauthd
mech_list: plain login cram-md5 digest-md5
log_level: 7
allow_plaintext: true
auxprop_plugin: mysql
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: mail
sql_passw: mailPASSWORD
sql_database: maildb
sql_select: select crypt from users where id='%u@%r' and enabled = 1

Note: When SASL is working, you can remove the log_level line.

Using Ubuntu 12.04.1 or greater?

I had a few problems sending emails after I upgraded my 10.04 system to 12.04 and finally traced the problems to the smtpd.conf file. It would seem that the definition names had changed slightly from the original documentation, or the Ubuntu build was more sensitive to proper naming convention.

Either way, I was unable to send emails until I made the changes described below:
pwcheck_method: saslauthd
mech_list: plain login cram-md5 digest-md5
log_level: 7
allow_plaintext: true
auxprop_plugin: sql
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: mail
sql_passwd: mailPASSWORD
sql_database: maildb
sql_select: select crypt from users where id='%u@%r' and enabled = 1
You can find more details on the configuration of the smtpd.conf file here.

Tell pam how to to authenticate smtp via mysql.

sudo vi /etc/pam.d/smtp

Note: These must be on 2 lines only.

auth required pam_mysql.so user=mail passwd=mailPASSWORD host=127.0.0.1 db=maildb table=users usercolumn=id passwdcolumn=crypt crypt=1
account sufficient pam_mysql.so user=mail passwd=mailPASSWORD host=127.0.0.1 db=maildb table=users usercolumn=id passwdcolumn=crypt crypt=1

In addition to tailing var/log/mail.log and /var/log/mysql/mysql.log it is quite useful to tail the auth.log as well when you’re testing SASL.

sudo tail -f /var/log/auth.log

Restart Postfix and saslauthd in order to enable SASL for sending emails.

sudo /etc/init.d/saslauthd restart
sudo /etc/init.d/postfix restart

Update 8th February 2011

Interesting point I uncovered while checking into saslauthd. Accouring to the Postfix SASL documentation, using “saslauthd” in the “pwcheck_method” (see above) means you cannot use cram-md5 or digest-md5!

To quote:
Important

Do not specify any other mechanisms in mech_list than PLAIN or LOGIN when using saslauthd! It can only handle these two mechanisms, and authentication will fail if clients are allowed to choose other mechanisms.
Then a bit further down it says:
Important

These three plugins [sasldb,sql,ldapdb] support shared-secret mechanisms i.e. CRAM-MD5, DIGEST-MD5 and NTLM. These mechanisms send credentials encrypted but their verification process requires the password to be available in plaintext. Consequently passwords cannot (!) be stored in encrypted form.
It seems fairly conclusive to me that I was never going to get this to work and have failed to do so tinkering with a multitude of settings and configurations (cleartext passwords, MD5 encrypted passwords, encrypted passwords etc..) to date. Now I’ve decided to omit the step below, move on to the section “Configuring encryption” and have altered /etc/postfix/sasl/smtpd.conf so that…
mech_list: plain login cram-md5 digest-md5
now looks like…
mech_list: plain login
The section from Flurdy’s guide is included for consistency and completeness.

IMAP SASL / Courier

I tend not to have SASL for my courier authentication, as I enforce TLS for all my clients. However, if you have a more lenient access policy (which is wise if you have many users), then you may want SASL in Courier as well.

sudo vi /etc/courier/imapd

This is available as a commented out line. Uncomment it or replace the current line by adding AUTH=CRAM-MD5 AUTH=CRAM-SHA1 so it resembles something like this on one line.

IMAP_CAPABILITY="IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA AUTH=CRAM-MD5 AUTH=CRAM-SHA1 IDLE"sudo /etc/init.d/courier-authdaemon restart;

sudo /etc/init.d/courier-imap restart

Configuring encryption

Notes on TLS

Encrypting the traffic stops anyone else from listening in on your email communications, which is very desirable. There are different types of communication that need to be encrypted:

The data traffic between your email applications and the server when you read (or send) emails.
Communication between other email servers and your server.

For the encryption of reading emails, it is Courier you need to configure. For sending and the “between server” communication encryption, it’s Postfix we change.

TLS in Postfix

To encrypt you need a certificate. As I’ve mentioned in the section above, Ubuntu creates some for you for which you can use while setting up the server. However before you go live, it is recommended to create your own with your proper domain name. In the examples below we will refer to the fictitious certificates created above.

sudo vi /etc/postfix/main.cf

There are already some TLS settings in the default Debian/Ubuntu version of this file. I amalgamated them and moved them to the beginning for clarity, but that’s up to you.

# TLS parameters
#smtp_use_tls = no
smtp_tls_security_level = may
#smtpd_use_tls=yes
smtpd_tls_security_level = may
#smtpd_tls_auth_only = no
smtp_tls_note_starttls_offer = yes
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
# The original references to the system cert and key.
# smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
# smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_cert_file=/etc/ssl/certs/smtp.example.com.pem
smtpd_tls_key_file=/etc/ssl/private/smtp.example.com.key
#smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
#smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

Next we edit the master.cf file.

sudo vi /etc/postfix/master.cf

By default, only the normal smtp service is enabled, which is fine. It’s preferable to enable submission (port 587), so that clients can use it, and I can restrict them to TLS only. Also enabled is the smtps service (port 465), for some compatibility with some older clients (Outlook Express etc…)

submission inet n       -       n       -       -       smtpd
  -o smtpd_sasl_auth_enable=yes
# if you do not want to restrict it encryption only, comment out next line
  -o smtpd_tls_auth_only=yes
# -o smtpd_tls_security_level=encrypt
# -o header_checks=
# -o body_checks=
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject_unauth_destination,reject
  -o smtpd_sasl_security_options=noanonymous,noplaintext
  -o smtpd_sasl_tls_security_options=noanonymous
# -o milter_macro_daemon_name=ORIGINATING
smtps     inet  n       -       -       -       -       smtpd
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes 
  -o smtpd_tls_auth_only=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_sasl_security_options=noanonymous,noplaintext
  -o smtpd_sasl_tls_security_options=noanonymous
# -o milter_macro_daemon_name=ORIGINATING

TLS in Courier

As mentioned previously, Ubuntu has created a certificate for you, but if you want to create your own, especially for a properly named server, then follow the steps in the section SSL Certificates above.

sudo vi /etc/courier/imapd-ssl

By default Ubuntu already points to your certificate file…

TLS_CERTFILE=/etc/courier/imapd.pem

…modify this if needed.

Also you if want to restrict IMAP users to only using SSL/TLS, toggle the following setting to “1”.

IMAP_TLS_REQUIRED=1

To maintain maximum compatibility it is not wise or recommended to restrict the traffic between servers to TLS only, as this means some of the valid emails sent by others may not reach your server.

Be aware that the emails are not encrypted on your machine, nor on the server.

There are some situations where SASL and TLS do not play well together. Those situations are normally when you combine the storing of encrypted passwords and using MD5 authentication over encrypted traffic. I (read:flurdy) recommend insisting on TLS traffic with your authenticating clients, which then negates the need for SASL.

Stage Four: Roundcube webmail client

Installing Roundcube

As an alternative to SquirrelMail, Roundcube has a more modern feel to it. It doesn’t have the proven track record of SquirrelMail and is still not quite a version “1.0” release (~~0.4~~ ~~0.5~~ ~~0.6~~ ~~0.7~~ 0.9.5 was the latest release at the time this was written).

However if you prefer a more appealing interface (and design is everything for me!) then follow these easy steps to get it up and running.

If you’ve already installed it and want to remove SquirrelMail, then

sudo aptitude remove squirrelmail squirrelmail-locales

Then install Roundcube (if you didn’t do so already in the “Packages” section).

sudo aptitude install roundcube roundcube-mysql

During the install it will ask you if you want to configure database access automatically. Answer Yes, then select MySQL as the database. You will then be prompted for the root MySQL password. This script will create the relevant database. tables and a Roundcube MySQL user. You will be prompted for a password for this new user.

The install will create a symblink in /etc/apache2/conf.d/ to /etc/roundcube/apache.conf. Edit this file…

sudo vi /etc/roundcube/apache.conf

Depending on your setup you may want to move those Alias commands to the top to your virtual hosts configuration, or for this example enable them here for all hosts.

Alias /roundcube/program/js/tiny_mce/ /usr/share/tinymce/www/
Alias /roundcube /var/lib/roundcube

Roundcube uses PEAR, which is a framework and distribution system for reusable PHP components. Assuming your system is up-to-date, it might pay to make sure PEAR is also.

sudo pear channel-update pear.php.net

…and make sure the MDB2 module is up to date.

sudo pear install mdb2

Now edit the Roundcube configuration file.

sudo vi /etc/roundcube/main.inc.php

Modify the following lines for added security/support of SASL/TLS (see informational note below)…

$rcmail_config['default_host'] = 'ssl://localhost:993';
$rcmail_config['smtp_server'] = 'ssl://localhost';
$rcmail_config['smtp_port'] = 465;

NOTE:

I deviated from “Flurdy’s” guide at this point as I had problems logging in until tweaked the following (based on the debug of messages in /var/log/syslog)
$rcmail_config['smtp_user'] = '%u';
$rcmail_config['smtp_pass'] = '%p';

Update the following to enable some default features. Note: for “smtp_helo_host” you can keep as the default value or change to your mail server name.

$rcmail_config['smtp_helo_host'] = 'mail.example.com';
$rcmail_config['language'] = 'en_GB';
$rcmail_config['product_name'] = '[your organisation name] RoundCube Webmail';
$rcmail_config['default_imap_folders'] = array('INBOX', 'Drafts', 'Read', 'Sent', 'Junk', 'Trash');
$rcmail_config['create_default_folders'] = TRUE;
$rcmail_config['preview_pane'] = TRUE;
$rcmail_config['logout_purge'] = TRUE;
$rcmail_config['logout_expunge'] = TRUE;

There are other tweaks and security features you can enable such as…

$rcmail_config['sendmail_delay'] = 1;

NOTE:

If you’re configuring Roundcube prior to having setup SASL/TLS (detailed under the “Secure” section above) then you’ll need to use the following settings until you do:
$rcmail_config['default_host'] = 'localhost';
$rcmail_config['smtp_server'] = 'localhost';
$rcmail_config['smtp_port'] = 25;
$rcmail_config['smtp_user'] = '';
$rcmail_config['smtp_pass'] = '';

Lastly we’ll add support for the default time of the server…

sudo vi /usr/share/roundcube/.htaccess

…by adding this parameter. Note: Of course you substitude “Pacific/Auckland” with your own time zone.

php_value date.timezone "Pacific/Auckland"

You can find a list of valid timezones here at the PHP website.

A lot of these tweaks are personal preference, but as an advisory it might pay to get the basics working first before playing with the settings too much! When you’re finished save and restart the apache process to enable changes.

sudo /etc/init.d/apache2 reload

Go to any web browser and point it to your Roundcube installation. Normally it’s http://mail.example.com/roundcube by default.

You can obviously modify and tweak further. One thing that may be useful is to have the Roundcube Apache Alias on different virtual hosts and configure username_domain in main.inc.php to append different email addresses, or configure the default_host to a different mail server depending on the virtual host…

More details on the Roundcube wiki.

Configuring HTTPS (If you want it?)

The mod_ssl module adds an important feature to the Apache2 server – the ability to encrypt communications. So, when your browser is communicating using SSL, the https:// prefix is used at the beginning of the Uniform Resource Locator (URL) in the browser navigation bar.

The mod_ssl module is available in the apache2-common package. Execute the following command from a terminal prompt to enable the mod_ssl module.

sudo a2enmod ssl

The default HTTPS configuration file is /etc/apache2/sites-available/default-ssl. In order for Apache2 to provide HTTPS, a certificate and key file are also needed. The default HTTPS configuration will use a certificate and key generated by the ssl-cert package.

They are good for testing, but the auto-generated certificate and key should be replaced by a certificate specific to the site or server. For information on generating a key and obtaining a certificate see the section Secure » SSL Certificates of this HOWTO or refer to Ubuntu’s documentation in the section called “Certificates”

To configure Apache2 for HTTPS, enter the following command.

sudo a2ensite default-ssl
sudo vi /etc/apache2/sites-available/default-ssl

SSLCertificateFile /etc/ssl/certs/<your certificate file name>.pem
SSLCertificateKeyFile /etc/ssl/private/<your key file name>.key

NOTE:

The directories /etc/ssl/certs and /etc/ssl/private are the default locations. If you install the certificate and key in another directory make sure to change SSLCertificateFile and SSLCertificateKeyFile appropriately.

Then restart Apache…

sudo /etc/init.d/apache2 restart

You can access the secure server pages by typing https://your_hostname/url/ in your browser’s address bar.

Appendix One: Adding data to the server

Add users and domains

Alright, so you finally got your mail server installed and you want to test it… BUT there’s no users or domains! Nada!! OK, to kick things off you need to add some default data, some of which is required and most which should make sense.

After that, we need to start adding your users and associated mail domains. Then you should be ready to start testing, or if you’ve done the testing… roll the server into production!

Base domain and user

First the required domain for local mail.

mysql -u mail -p maildb

Insert the domain information:

INSERT INTO domains (domain) VALUES ('localhost'), ('localhost.localdomain');

Then the default aliases. Some people say these are not needed, but I recommend including them.

INSERT INTO aliases (mail,destination) VALUES ('postmaster@localhost','root@localhost'), ('sysadmin@localhost','root@localhost'), ('webmaster@localhost','root@localhost'), ('abuse@localhost','root@localhost'), ('root@localhost','root@localhost'), ('@localhost','root@localhost'), ('@localhost.localdomain','@localhost');

It’s worth noting at this point, that this “set” of aliases is common across each domain i.e. postmaster,abuse and if you like sysadmin or admin.

Now we create a “root” user.

INSERT INTO users (id,name,maildir,crypt) VALUES ('root@localhost','root','root/', encrypt('apassword') );

Domains and users

It’s at this point that we add some proper data. In this example we’ll say you want this machine to handle data for the fictional domains of “blobber.org” and “whopper.nu“.

We will refer to this machine’s name as “mail.blobber.org“.

INSERT INTO domains (domain) VALUES ('blobber.org'), ('mail.blobber.org'), ('whopper.nu');
INSERT INTO aliases (mail,destination) VALUES ('@mail.blobber.org','@blobber.org'), ('postmaster@whopper.nu','postmaster@localhost'), ('abuse@whopper.nu','abuse@localhost'), ('postmaster@blobber.org','postmaster@localhost'), ('abuse@blobber.org','abuse@localhost');

Having created the domains and aliases we now add names for two users called “Fred” and “Wilma“.

INSERT INTO users (id,name,maildir,crypt) VALUES ('fred@blobber.org','Fred Flintsone','fredflinstone/', encrypt('apassword') ), ('wilma@blobber.org','Wilma Fintstone','wilmaflinstone/', encrypt('anotherpassword') );
INSERT INTO aliases (mail,destination) VALUES ('fred@blobber.org','fred@blobber.org'), ('wilma@blobber.org','wilma@blobber.org');

Adding tweaks to the basic users

If you want all the mail for whooper.nu to go to Fred (i.e. *@whooper.nu).

INSERT INTO aliases (mail,destination) VALUES ('@whopper.nu','fred@blobber.org');

Say there is a user called “Pebbles“, but she wants all her mail forwarded to an external account.

INSERT INTO aliases (mail,destination) VALUES ('pebbles@blobber.org','pebbles.flintstone@gmail.com');

So what do each of these lines above actually do? Well the domain entries are pretty straight forward. So too are the lines relating to the users, however it’s worth noting here that each entry requires the following four fields to be included.

id is the email address of the user, and also its username when logging in. This is described later on.
name is optional description or “full name” of the user.
maildir is the name of the folder inside /var/spool/mail/virtual. It MUST end in a /, otherwise it won’t be used as the unix maildir format.
crypt is the encrypted text password to use.

The aliases are the interesting part. Lets track an incoming message to see how it’s delivered.

Say an email arrives addressed to “john@whopper.nu“:

Postfix looks up the domains table and locates whopper.nu as a domain it listens to.
Postfix then looks up aliases and searches for a row where the mail field matches “john@whopper.nu“.
There is no match, so it searches for “@whopper.nu“, which is the “catchall” for that domain.
Postfix then finds one row and its destination is “fred@blobber.org“.
It then searches for “fred@blobber.org” and finds an entry, which has a destination that is the same as the mail and therefore it is the final destination.
It then tries to deliver this mail. The look up says blobber.org is a local mail domain so it looks up users for a matching id and delivers it to its maildir.

Any mail arriving for “pebbles@blobber.org” gets forwarded to the external address of “pebbles.flintstone@gmail.com“. So forwarding is as simple as adding a line into the aliases table.

I also added the required aliases of postmaster and abuse to blobber.org and whopper.nu. Another useful alias to add is root, as often you get admin mail from cron jobs etc within those domains. Other often used aliases are info, sysadmin, support, sales, webmaster, mail, contact and all. The problem with creating all of them is that they are also honeypots for spam, so just include the ones you think you will need.

Adding domains, users and aliases using a template

To add a new domain to the system you use this command in MySQL, replacing the values in italics with data relevant to you:

INSERT INTO domains (domain) VALUES ('domain.tld');
INSERT INTO aliases (mail,destination) VALUES ('@domain.tld','email@address'), ('postmaster@domain.tld','email@address'), ('abuse@domain.tld','email@address');

To add a new user to the system:

INSERT INTO users (id,name,maildir,crypt) VALUES ('email@address','short description','foldername/',encrypt('password'));
INSERT INTO aliases (mail,destination) VALUES ('email@address','email@address');

Common SQL statements

A selection of useful SQL statements to use if you are not using an admin/manager program to maintain your email domains and users.

Find domains without a “catchall”

Remember some might be disabled!!

SELECT dom.domain FROM domains dom LEFT JOIN aliases al ON CONCAT( '@', dom.domain ) = al.mail WHERE al.mail is null OR al.enabled = 0 ORDER BY dom.domain ASC;

Find aliases for an invalid domain

SELECT al.* FROM aliases al LEFT JOIN domains dom ON dom.domain = SUBSTRING(al.mail,LOCATE('@',al.mail)+1) WHERE dom.domain is null OR dom.enabled = 0 ORDER BY al.mail ASC;

Find all non local destination aliases

SELECT al.* FROM aliases al LEFT JOIN domains dom ON dom.domain = SUBSTRING(al.destination,LOCATE('@',al.destination)+1) WHERE dom.domain is null ORDER BY al.enabled, al.destination ASC, al.mail ASC;

Find all aliases for a certain domain

SELECT al.* FROM aliases al WHERE SUBSTRING(al.mail,LOCATE('@',al.mail)+1) = 'domain.tld' ORDER BY al.enabled, al.mail ASC;

Find all aliases for a certain domain, checking if enabled for both domain and alias

SELECT * FROM domains d JOIN aliases a ON a.mail like concat( '%','@',d.domain) AND a.enabled = 1 WHERE d.enabled = 1 AND d.domain like '%foobar%' ORDER BY d.domain,a.mail;

Appendix Two: Testing the server

Common problems

Missed a step?

If you mistakenly or intentionally skipped past sections, you may have missed an important step in your configuration, which this guide presumes you have followed.
Typo!

99% of all problems is spelling errors or typos you entered while following this guide. Sorry, but it just happens. Often it can be trivial, such as a space at the end of the configuration line which was not expected etc. Or not understanding my example where it is a multi line entry.
Typo in he documentation.

Yes, I (read: flurdy) can make mistakes. Nothing wrong in that, but I hope I have corrected most over time after each proof read. However, any new sections to this guide are potentially at risk… ☺
Different application or configuration.

It is obviously entirely up to you how you set up your system. But the more you deviate from this guide, the more likely incompatibilities or confusion will arise.
Distribution/version differences.

If you run a different version or even distribution to this guide, then some things will be different. Small issues, such as default values and significant things such as path differences etc. Some sections in this guide are not always thoroughly tested with every new release of Ubuntu, but these differences get pointed out by people for me.
Walking before crawling.

Don’t try the full blown mail server before the basics are working.
Gamma rays and little goblins!

Because we’ve got to blame it on something… right? 😉

Test strategy

Test early and frequently

It is very helpful to test early in this set-up to establish if the first sections are working as expected. So when you only have your very basic Postfix and MySQL up and running – TEST IT! That way you know that certain step worked and you can rule it out of any future problems. Don’t wait until you complicated the install by adding additional features like AMAVIS, SASL, Postgrey etc…

By constantly testing if you can send and receive you can tick off and black box each section as working, and immediately spot issues.

Isolate the problem!

Testing how things work is often about isolating the problem first. So by using the steps of testing early above, you can see which step caused the problem. Also if you can’t log into your webmail it is often nothing to do with the webmail section that is causing the problem. Often postfix itself is broken…

Test in order

As part of the isolating the problem step it’s a good idea to test steps in order, thus isolating the problem. This would then quickly indicate where the problem might lie. e.g. In the example relating to webmail access, the order of testing might go:

Access: Can I get (SSH) access to the box, and is there a firewall issue?
Database: Is the database up, can my applications talk to it?
Postfix: Can I send an email by command line (telnet)? Can I receive emails via telnet?
Content checks: Is the content of my message causing a problem?
Courier: Can I read emails?
Webmail: Does the web integration work?

Simplify the system

Assisting in isolating the problem, you often have to disable options and applications. An example might be to turn off Postgrey or the content checks to make sure emails to get delivered.

Get out the toolbox… MXToolbox that is

One website I found invaluable for testing my server from the “outside” was MXToolbox. You can use it to lookup/check your MX records, run SMTP tests to the server, check for open ports… and (hopefully never) check if your server has been blacklisted!

It’s probably really widely known by those in the “mail” support field, but I stumbled across it trying to troubleshot why a colleague’s emails were getting bounced when he tried to send them to us. The support guy on the phone told me to use MXToolbox to check if the server had been blacklisted and follow it up from there as there was nothing they could do. Lo-and-behold, his server was blacklisted on one of the many “blacklist reference servers” used by our ISP (and others). I can’t stop using it now, it’s a great site…

Tail, tail and tail again

It’s essential to monitor what actually happens, and tailing (specifically the mail and mysql log) is key in tracking down problems. The following are most commonly referred to when troubleshooting:

/var/log/syslog
/var/log/mail.log
/var/log/mysql.log
/var/log/apache2/access.log

You could have multiple windows open “tailing” the log files and changing the configuration in a third.

tail -f /var/log/mail.log
tail -f /var/log/mysql.log

Switch debugging on

Shorewall

You can also switch on more messages for when the firewall is rejecting connections. Add info to all REJECT, BOUNCE and DROP policies.

sudo vi /etc/shorewall/policy

Net        all        DROP       info

MySQL

There is no point in tailing the mysql log if query debugging is not turned one. By default it is not. However in this guide I do switch it on. In case that was missed switch it on now:

sudo vi /etc/mysql/my.cnf

and make sure this line is not commented out.

log = /var/log/mysql/mysql.log

Courier

As mentioned in the setup , switching on debugging for Courier is easy:

sudo vi /etc/courier/authdaemonrc
DEBUG_LOGIN=2

WARNING:

With this setting passwords logged in syslog will be in clear text!

AMAVIS

You can also debug AMAVIS:

sudo vi /etc/amavis/conf.d/50-user

And perhaps bump up the log level if you are already debugging:

$log_level = 2;

Telnet is your friend!

When testing a mail server, telnet is alpha & omega. You use it to simulate another mail server in order to test responses by your mail server.

First you test it on the server to rule out firewall and network issues.
Then you test it from another machine to simulate another mail server trying to communicate.
Once these are working you can use a proper email client, however in 99% of cases I just use MUTT locally when I need to test if a server is working.

Can Postfix receive?

Lets assume:

You have followed the guide up to “Basic” configuration at the very least.
You have entered data into the database (see previous section).
The services MySQL and Postfix are running.
If testing a fuller stack, then amavis, postgrey, clamav-daemon, spamassassin etc must also be running.

Try this locally on the server first, then try from another machine once you confirm it works locally.

Lets try and send a message to fred@example.org (replace with your own user in this setup, or use postmaster@localhost) from you@example.com (again, replace with a real email address that is not associated with this server).

telnet localhost 25
 # Open and hand shake with EHLO and the server name you are connecting from. Change mail.example.com to something valid. e.g. your servername
 EHLO mail.example.com
 # The mail server will then dump out some details about its capabilities, e.g.
 > 250-mail.flurdy.net
 > 250-PIPELINING
 > ...
 > ...
 # then say who is the sender of this email
 MAIL FROM: your@example.com
 > 250 Ok
 # then say who the mail is for...
 RCPT TO: fred@example.org
 > 250 Ok
 # then enter the keyword data
 data
 > 354 End data with <CR><LF>.<CR><LF>
 # enter message body and end with a line with only a full stop then enter.
 blah blah blah
 more blah
 .
 > 250 Ok; queued as QWKJDKASAS
 # end the connection with quit
 quit
 > 221 BYE

If (while you were doing this) you were tailing /var/log/mail.log you would (read: should) have seen some activity and whether any errors occurred. (You might get some complaints about missing headers as we skipped most of them…)

If (while you were doing this) you were tailing /var/log/mysql.log as well, you would definitely have seen some activity – if not you have a problem!

If you see any errors (or worse no activity) in these log files, then this highlights what you need to fix! However, if no errors popped up, and the folder /var/mail/virtual/fred (or similar) now exists then your server can receive emails!

Can Postfix send?

You need to first make sure that receive emails (as detailed above) and the services MySQL and Postfix are running.

Try this locally on the server first, then try from another machine once you confirm it works locally.

telnet localhost 25
 # Open and hand shake with EHLO and the server name you are connecting from. Change mail.example.com to something valid. e.g. your servername
 EHLO mail.example.com
 # The mail server will then dump out some details about its capabilities, e.g.
 > 250-mail.flurdy.net
 > 250-PIPELINING
 > ...
 > ...
 # then say who is the sender of this email
 MAIL FROM: fred@example.org
 > 250 Ok
 # then say who the mail is for...
 RCPT TO: your@example.com
 > 250 Ok
 # then enter the keyword data
 data
 > 354 End data with <CR><LF>.<CR><LF>
 # enter message body and end with a line with only a full stop then enter.
 blah blah blah
 more blah
 .
 > 250 Ok; queued as QWKJDKASAS
 # end the connection with quit
 quit
 > 221 BYE

We have to assume (I know what they say about assuming) that receiving works so there is no ned to tail the MySQL log. However, if any rejection messages occurred in the mail.log then you need to troubleshoot it to resolution.

If no errors occurred and you see something in the mail.log that looks like this:

Dec 17 10:25:45 servername postfix/smtp[12345]: 12345678: to=<you@example.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=15, delays=15/0.01/0.02/0.11, dsn=2.0.0, status=sent (250 2.0.0 Ok, id=12345-09, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 1234567)

then the sending of emails is working perfectly!

Can Courier read emails?

You need to first make sure that you can receive emails (as detailed above).
You need to make sure that you can send emails (as detailed above).
You need to make sure you have received an email by checking that the folder /var/mail/virtual/username exists.
The MySQL, courier-authdaemon and courier-imap services are all running.

There is not too much you can test via telnet for courier. But you can check if it is up and that you can connect to it.

telnet 127.0.0.1 143
 Trying 127.0.0.1...
 Connected to 127.0.0.1. Escape character is '^]'.
 * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS LOGINDISABLED] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc. See COPYING for distribution information

For more in depth testing, you would have to use a proper IMAP email client.

Can AMAVIS check and pass on emails?

You need to first make sure that you can receive emails (as detailed above).
You need to make sure that you can send emails (as detailed above).
You need to make sure you have received an email by checking that the folder /var/mail/virtual/username exists.

All you can do here is check to make sure that the service is responding.

telnet 127.0.0.1 10024
 Trying 127.0.0.1...
 Connected to 127.0.0.1.
 Escape character is '^]'.
 220 [127.0.0.1] ESMTP amavisd-new service ready

then tail /var/log/mail.log and see if there are any problems.

Introduction.

Background.

Base OS Distribution

Software components

OS: Ubuntu Linux

MTA: Postfix

POP/IMAP: Courier IMAP

Database: MySQL

Content Check: Amavisd-new

Anti-SPAM: SpamAssassin & Postgrey

Anti-Virus: ClamAV

Authentication: Cyrus SASL

Encryption: TLS

Webmail: Roundcube

Software configuration

Repositories

Packages

MySQL

Postfix

openSSH Server

SASL

ClamAV

Amavis, SpamAssassin, postgrey

Roundcube

phpMyAdmin

ShoreWall

Courier

Extras

Package status

Stage One: A simple mail server

Firewall

Shorewall

SSH only

Open for business?

MTA – Mail Transfer Agent

Postfix

Postfix’s MySQL configuration

Database

MySQL

POP/IMAP

Courier IMAP

Stage Two: Advanced mail server

Content check interface

Amavisd-new

Open for business?

Anti-SPAM

SpamAssassin

Postgrey

Anti-VIRUS

ClamAV

Stage Three: Secure your mail server

SSL Certificates… or “How I lost my mind for a week”

Don’t cry little baby…

Down to business

Some helpful commands

Configuring authentication

SASL

IMAP SASL / Courier

Configuring encryption

Notes on TLS

TLS in Postfix

TLS in Courier

Stage Four: Roundcube webmail client

Installing Roundcube

Configuring HTTPS (If you want it?)

Appendix One: Adding data to the server

Add users and domains

Base domain and user

Domains and users

Adding tweaks to the basic users

Adding domains, users and aliases using a template

Common SQL statements

Find domains without a “catchall”

Find aliases for an invalid domain

Find all non local destination aliases

Find all aliases for a certain domain

Find all aliases for a certain domain, checking if enabled for both domain and alias

Appendix Two: Testing the server

Common problems

Missed a step?