Migrate from amavisd-new to Maia Mailguard on Ubuntu 10.4 or Debian lenny


Absolutely no warranty, use entirely at your own risk. See the disclaimer and a newer version of this doc at http://verchick.com/mecham/public_html/spam/

This document assumes you already have Postfix, amavisd-new, ClamAV and SpamAssassin installed, configured and working properly and Postfix is using amaivsd-new as a content_filter on port 10024. If you have not already done this, it is up to you to find documentation to accomplish this. This may help: http://verchick.com/mecham/public_html/spam/spamfilter.html. I will assume you are using the package maintainer's version of amavisd-new. The normal Debian/Ubuntu version of amavisd should not use amavisd.conf. Test to see if you have amavisd.conf:
ls -l /etc/amavis/amavisd.conf

If you have an amavisd.conf file, in other words: if you built your system using the document at http://verchick.com/mecham/public_html/spam/spamfilter20090215.html or other document that has you create an amavisd.conf file then this is not the right document to follow; instead use http://verchick.com/mecham/public_html/spam/debian-maia-lenny.html. This document also assumes you have at least 1GB RAM installed. You will notice later that I'm logged in as root. I simply prefer it. If you are not familiar with turning the root login on and off on Ubuntu, I'm sure you can find documentation on that elsewhere.

Save this document to your computer. If using Internet Explorer, save as "Web Page, HTML only (*.htm,*,html)" then use a plain text editor - like WordPad for example, and replace the text amavis_password with a password of your choice. You also need to replace the text sfa with the hostname of your server, and the text example.com with the domain name of your server. Lastly, replace the text Widgits Inc. with the name for your company. Once completed, open this page from the copy you saved. Do not replace these items more than once.

This HOWTO is a quick and dirty guide on migrating from amavisd-new to Maia Mailguard installed on a Ubuntu 10.4 or Debian lenny email server that was originally built using Postfix, amavisd-new, ClamAV and SpamAssassin. This guide is designed to get someone past the initial brain damage of getting the right programs and files in the right place.

You must read https://www.maiamailguard.com/maia/wiki/Install so you will understand what we are trying to accomplish. Keep in mind that Maia will replace amavisd-new. Maia and amavisd-new are two different programs and only one or the other can be used - not both. This guide does not cover important configuration settings that need to be made once the software is in place; it just gets you to the point where you can log into Maia. This guide does not explain how to use Maia Mailguard, it merely gets it up and running. It uses many of the default settings (which may not filter mail). I don't use Maia myself, so I cannot be helpful regarding the configuration or use of the program. I also don't use Apache, and I am not skilled at MySQL. There is no support for this document. For help with Maia, you should join the Maia users mailing list http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users.

There is a 10MB .PDF available from http://www.novell.com/coolsolutions/feature/16093.html that has some good tips for configuring and using Maia. This is a SuSE document, but the Maia stuff should give you a number of good ideas.

The first thing to do is backup our current amavisd-new config file (if there is one):
cp -i /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-ubuntu

We need to make a symbolic link (if we don't already have one) because Maia will want to use /etc/amavisd.conf, not /etc/amavis/amavisd.conf:
ln -s /etc/amavis/amavisd.conf /etc/amavisd.conf

We want to turn off automatic upgrades of amavisd-new, so we place the package on hold:
echo "amavisd-new hold" | dpkg --set-selections

On a Ubuntu machine, in order for ClamAV to work correctly, remove AppArmor:
/etc/init.d/apparmor stop
update-rc.d -f apparmor remove
aptitude remove apparmor apparmor-utils


Download revision 1518 Maia 1.0.3. I also install vim as that is my preferred editor. You can browse https://www.maiamailguard.com/svn/branches/1.0 and make a note of the revision of the trunk you may choose to install today. Also see https://www.maiamailguard.com/maia/browser/branches/1.0. You can pick another revision, but be aware that this document is based on revision 1518. The newest stable build I can recommend is 1518. There are a lot of changes after that so a newer version will not work with this document. OK, let's get started:
apt-get update
apt-get install subversion vim

mkdir /usr/local/src/maia
cd /usr/local/src/maia


If you choose a revision other than 1518, keep the number handy:
svn -r 1518 checkout https://www.maiamailguard.com/svn/branches/1.0

Answer (p) if it asks...
If you ever use this method to download Maia again, you should first move the old downloaded files to another directory to avoid overwriting them. Read this thread. Now, install MySQL 5.1 (if you do not have MySQL installed):
apt-get install mysql-server

Supply a password for root when prompted - and remember it!
If you need to manually set the password: Change the roots_password entry to a password of your choice and make sure the host name is correct:
mysql

SET PASSWORD FOR 'root'@'localhost' = PASSWORD('roots_password');
SET PASSWORD FOR 'root'@'sfa' = PASSWORD('roots_password');
FLUSH PRIVILEGES;
QUIT


Out of the box, MySQL is poorly tuned for use with InnoDB databases like Maia uses. I will assume you have at least 1GB of RAM. In the [mysqld] section of /etc/mysql/my.cnf we need to add some entries. By default, innodb_buffer_pool_size is only 8MB. This makes data access with InnoDB very slow. You can set this at somewhere around 25% of physical RAM. Then, innodb_log_file_size should be set to 25% of innodb_buffer_pool_size. Here I'm going to use 128MB for innodb_buffer_pool_size and 32MB for innodb_log_file_size. The change to innodb_log_file_size will not have any effect however until we have MySQL create new logs. Therefore, we have to shut MySQL down, rename the logs and then let MySQL create new logs. Also, we need to set max_allowed_packet to the maximum size of a message that will be allowed into our system. Here I'm setting it at a reasonably large 20MB. Before deleting the log files, to avoid loss of data, log into MySQL and verify that innodb_fast_shutdown is set to 1:
mysql -p

SHOW VARIABLES LIKE 'innodb_fast_shutdown';
QUIT


If it is not set to 1, you would have to edit /etc/mysql/my.cnf and find where this has been changed from the default, then restart MySQL. Edit /etc/mysql/my.cnf and place these entries (after adjusting if needed) in the location shown:
vi /etc/mysql/my.cnf

# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
innodb_data_home_dir = /var/lib/mysql
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /var/lib/mysql
innodb_buffer_pool_size = 256M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 64M
innodb_log_buffer_size = 8M
max_allowed_packet = 20M
# * Security Features
Save and exit the file, then restart MySQL:
/etc/init.d/mysql restart

Now shut down MySQL and rename and rebuild the log files (assuming MySQL restarted without errors):
/etc/init.d/mysql stop
mv /var/lib/mysql/ib_logfile0 /var/lib/mysql/iblogfile0-old
mv /var/lib/mysql/ib_logfile1 /var/lib/mysql/iblogfile1-old
/etc/init.d/mysql start
ls -l /var/lib/mysql/ib_logfile*


Install more needed programs:
apt-get install patch libossp-uuid-perl libtemplate-perl libtext-csv-perl libwww-perl cabextract libmail-dkim-perl

Install PHP5, Pear and some additional Pear modules (watch for errors - WARNINGS are not fatal):
apt-get install php5 php-pear php5-common php5-mysql php5-gd php5-sqlite smarty

pear channel-update pear.php.net

pear install Mail_Mime-1.8.1
pear install Mail_mimeDecode-1.5.5
pear install Log-1.12.4
pear install Pager-2.4.8

pear install Image_Color-1.0.4
pear install Image_Canvas-0.3.1
pear install Image_Graph-0.8.0
pear install Numbers_Roman-1.0.2
pear install Numbers_Words-0.16.1

pear install Auth_SASL-1.0.4
pear install Net_Socket-1.0.10
pear install Net_IMAP-1.1.0
pear install Net_POP3-1.3.8
pear install DB-1.7.13
pear install Net_SMTP-1.4.4

pear channel-discover htmlpurifier.org
pear install hp/HTMLPurifier

When the time comes you want to view Pie charts, enable them with Settings (the gear)->Miscellaneous Settings->Display graphic charts? AFTER you "Enable graphical charts?" in the Charts section in Admin (the key)->System configuration.

Install Apache2
apt-get install apache2 libapache2-mod-php5

You should now be able to open a browser and browse to the IP address of the server. I recommend adding the IP address and FQDN hostname of the server to your hosts file, so you can browse the server using the hostname instead of the IP address. Of course, eventually you will need to set up the host in DNS if you have not already done so. If this is a production server, obviously you already have.

We need to create some directories and copy some files per the INSTALL instructions. We use 'sed' to change /var/amavisd to /var/lib/amavis in a few files:
mkdir /var/lib/amavis/maia
chmod -R 750 /var/lib/amavis/maia
mkdir /var/lib/amavis/maia/scripts
chmod -R 750 /var/lib/amavis/maia/scripts
cd /var/lib/amavis/maia/scripts
cp /usr/local/src/maia/1.0/scripts/* .
sed -i 's|var/amavisd|var/lib/amavis|g' load-sa-rules.pl
sed -i 's|var/amavisd|var/lib/amavis|g' process-quarantine.pl
sed -i 's|var/amavisd|var/lib/amavis|g' send-quarantine-digests.pl
sed -i 's|var/amavisd|var/lib/amavis|g' expire-quarantine-cache.pl
sed -i 's|var/amavisd|var/lib/amavis|g' send-quarantine-reminders.pl
sed -i 's|var/amavisd|var/lib/amavis|g' stats-snapshot.pl
test -e /etc/maia.conf && mv /etc/maia.conf /etc/amavis/maia.conf-backup
cp /usr/local/src/maia/1.0/maia.conf.dist /etc/amavis/maia.conf
ln -s /etc/amavis/maia.conf /etc/maia.conf
sed -i 's|var/amavisd|var/lib/amavis|g' /etc/amavis/maia.conf
sed -i 's|passwd|amavis_password|' /etc/amavis/maia.conf
chmod 0640 /etc/amavis/maia.conf
chgrp amavis /etc/amavis/maia.conf
cp /usr/local/src/maia/1.0/*.cf /etc/spamassassin
cp -r /usr/local/src/maia/1.0/templates /var/lib/amavis/maia/templates
chown -R amavis:amavis /var/lib/amavis
chmod 640 /var/lib/amavis/maia/templates/*.tpl
cp /usr/local/src/maia/1.0/amavisd-maia /usr/sbin

Now we need to edit maia.conf to reflect what our URL will be. We are going to place our files in a /mail directory under the www root:
vi /etc/amavis/maia.conf

$base_url = "http://sfa.example.com/mail";


Save and exit the file. We need to log into MySQL and create the maia database:
mysql -p

CREATE DATABASE maia;
USE maia;


Be careful that the next command points to the correct location of "maia-mysql.sql":
SOURCE /usr/local/src/maia/1.0/maia-mysql.sql;

You should have seen a number of lines similar to "Query OK, 1 row affected (0.01 sec)" scroll by.
Now we create the amavis user and then quit (there are 3 commands here):
GRANT CREATE, DROP, ALTER, SELECT, INSERT, UPDATE, DELETE ON maia.* TO amavis@localhost IDENTIFIED BY 'amavis_password';
FLUSH PRIVILEGES;
QUIT

Let's see if our configuration looks OK so far:
cd /var/lib/amavis/maia/scripts/
./configtest.pl

This should result in something like:
Application/Module      Version   Status
========================================================================
Perl                 :   5.10.1 : OK
file(1)              :     5.03 : OK
Archive::Tar         :     1.52 : OK
Archive::Zip         :     1.30 : OK
BerkeleyDB           :     0.39 : OK
Compress::Zlib       :     2.02 : OK
Convert::TNEF        :     0.17 : OK
Convert::UUlib       :     1.12 : OK
Crypt::Blowfish      :      N/A : NOT INSTALLED ... 
Crypt::CBC           :      N/A : NOT INSTALLED ... 
Crypt::OpenSSL::RSA  :     0.25 : OK
Data::UUID           :   1.0602 : OK
DB_File              :     1.82 : OK
DBD::mysql           :    4.012 : OK
DBD::Pg              :      N/A : NOT INSTALLED ...
DBI                  :    1.609 : OK
Digest::MD5          :     2.39 : OK
Digest::SHA1         :     2.12 : OK
File::Spec           :      3.3 : OK
HTML::Parser         :     3.64 : OK
HTTP::Date           :    5.831 : OK
IO::Stringy          :    2.110 : OK
IO::Zlib             :     1.09 : OK
IP::Country          :      N/A : NOT INSTALLED ... 
LWP::UserAgent       :    5.834 : OK
Mail::Address        :     2.05 : OK
Mail::DKIM           :     0.38 : OK
Mail::Internet       :     2.05 : OK
Mail::SpamAssassin   :    3.3.1 : OK
Mail::SPF::Query     :      N/A : NOT INSTALLED ...
MIME::Base64         :     3.08 : OK
MIME::Parser         :    5.427 : OK
MIME::QuotedPrint    :     3.08 : OK
Net::CIDR::Lite      :      N/A : NOT INSTALLED .. 
Net::Cmd             :     2.29 : OK
Net::DNS             :     0.65 : OK
Net::Server          :     0.97 : OK
Net::SMTP            :     2.31 : OK
Pod::Usage           :     1.36 : OK
Template             :     2.20 : OK
Time::HiRes          :   1.9719 : OK
Unix::Syslog         :      1.1 : OK
URI                  :     1.52 : OK
Text::CSV            :     1.16 : OK

Database DSN test    : PASSED
We do not need the six items shown as NOT INSTALLED. We don't use PostgreSQL, so we don't need DBD::Pg

Now we copy SpamAssassin's rules to the MySQL database (make sure 'spamassassin --lint' does not return any errors; if it does, repair the errors before you continue):
su amavis -c 'spamassassin --lint'

Since we are using SpamAssassin 3.1.1 or greater we can use the new sa-update feature:
sa-update

cd /var/lib/amavis/maia/scripts
./load-sa-rules.pl


You should have seen something like "2010-12-11 12:20:26 Maia: [load-sa-rules] 979 new rules added (979 rules total), all scores updated." If not, then check that this ./load-sa-rules.pl script points to the correct directories (you edited it earlier). Now we will copy the web interface files to the web site. We also need to make 'themes' writable by the www-data user (and amavis):
mkdir /var/www/mail
cp -r /usr/local/src/maia/1.0/php/* /var/www/mail
ln -s /usr/share/php/smarty/libs /usr/share/php/Smarty
cp /var/www/mail/config.php.dist /var/www/mail/config.php
sed -i "s/passwd/amavis_password/g" /var/www/mail/config.php
chmod 0640 /var/www/mail/config.php
chown root:www-data /var/www/mail/config.php
chgrp amavis /var/www/mail/themes/*/compiled
chmod 775 /var/www/mail/themes/*/compiled
gpasswd -a www-data amavis

mkdir /var/www/mail/purifier_cache
chown www-data:www-data /var/www/mail/purifier_cache
sed -i 's|purifier_cache = null;|purifier_cache = "/var/www/mail/purifier_cache";|' /var/www/mail/config.php

/etc/init.d/apache2 restart
/etc/init.d/postfix restart


Now you browse to configtest.php to test the installation, for example:
http://192.168.1.222/mail/admin/configtest.php
(or)
http://sfa.example.com/mail/admin/configtest.php

The result should look something like this. Note: if you need the IMAP, LDAP or MCrypt libraries you are free to
apt-get install php5-imap php5-ldap php5-mcrypt (but only as needed). Note that encrypting mail in the database is NOT recommended. By default Maia will use built in authentication, but you can configure it later to use IMAP or LDAP if required. I do not cover that in this document. Also, remember to stop and start apache2 if you make changes.

You should ensure the database schema is up to date:
http://sfa.example.com/mail/admin/upgrade.php


Assuming you are using the original init script supplied by the amavisd-new package, make a change to the amavis init script so it will not complain about having an amavisd.conf file laying around:
sed -i "s/  \tcheck_noncompatible_upgrade/#  \tcheck_noncompatible_upgrade/" /etc/init.d/amavis

You should vi /etc/init.d/amavis and ensure that the two lines shown below are commented out and that the comment mark is at the beginning of each line:
        fixdirs
#       check_noncompatible_upgrade
        if start-stop-daemon ${START} -- ${PARAMS} start >/dev/null ; then
		.....
        fixdirs
#       check_noncompatible_upgrade
        exec ${DAEMON} ${PARAMS} "${mode}"
		....
We will start off with a configuration file specific to Maia. I have a patch that modifies the file for use with this Ubuntu setup. Make sure the patch applies cleanly:
cd /etc/amavis
cp /usr/local/src/maia/1.0/amavisd.conf.dist .
cp amavisd.conf.dist amavisd.conf
wget http://verchick.com/mecham/public_html/spam/amavisd.conf.1500.patch.txt

patch amavisd.conf < amavisd.conf.1500.patch.txt
sed -i "s/'password'/'amavis_password'/" amavisd.conf
sed -i "s/yourdomain.tld/example.com/" amavisd.conf
sed -i "s/host.domain.tld/sfa.example.com/" amavisd.conf
chmod 0640 amavisd.conf


We need to make a few changes to amavisd.conf:
vi /etc/amavis/amavisd.conf

I suggest you place the SVN release number of Maia in a comment in this file. Make sure your $max_servers setting matches the maxproc setting for the smtp-amavis transport in /etc/postfix/master.cf. If you have not configured this amavisd.conf in the past, at the very least you need check $mydomain. I also suggest you check $myhostname. Note that you may also wish to temporarily set $log_level to 5 when we send a few test messages through, examining /var/log/mail.log for signs of trouble. Also, please read section "12. Configure amavisd-maia" of the Maia installation instructions for other suggested amavisd-maia settings. Blowfish encryption is NOT recommended. Save and exit the file, and then make a backup copy:
cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-maia-backup

Now, when you want to try Maia as your program, you first stop amavisd-new:
/etc/init.d/amavis stop

Then you edit the init script to start amavisd-maia instead of amavisd-new:
sed -i "s/amavisd-new/amavisd-maia/" /etc/init.d/amavis

Then you start up amavisd-maia (actually, the first time you start it up it would be a good idea to do so with 'amavisd-maia debug'):
/etc/init.d/amavis start

You can keep an eye out for errors by using:
tail -f /var/log/mail.log

Send a few test messages through and look for errors. Since the system has not been configured, the default is to pass everything (including viruses and executables).

If you need to revert to amavisd-new, first you would stop amavisd-maia, edit the init script and start up amavisd-new, like this:
/etc/init.d/amavis stop
sed -i "s/amavisd-maia/amavisd-new/" /etc/init.d/amavis
/etc/init.d/amavis start

Then to switch back to Maia:
/etc/init.d/amavis stop
sed -i "s/amavisd-new/amavisd-maia/" /etc/init.d/amavis
/etc/init.d/amavis start

At this point, nothing is set up in Maia, so If I am correct, all mail will simply pass. I believe the default behavior is to bypass all checks for recipients (or domains) not in the database so you will quickly want to add and configure your users and/or domains. I suggest configuring domains first in order to set up what you believe are acceptable domain wide defaults.

Continue on by going to https://www.maiamailguard.com/maia/wiki/Install and start reading from "14. Login and become the super-administrator". You want to use your email address as the "E-mail address your login credentials should be mailed to:" (so you get the initial super-administrator password sent to you). Hopefully you have configured this system as a working SMTP server so mail sent to that address from this server will end up in a mailbox you have access to. My internal-init.php page looked similar to this example. Since you are starting out using the "Internal" authentication method, run the
http://sfa.example.com/mail/internal-init.php   script before running the
http://sfa.example.com/mail/login.php?super=register  script.   If all goes well, you will get a "250 Ok" response on the screen and your password in a message in your inbox. Place your email address in the "E-mail address your login credentials should be mailed to" box. Run the super-register script shown above next. Don't forget you need to set up DNS (if you have not done so) so your users can use your server's FQDN to access Maia.



Your site is at http://sfa.example.com/mail

Now, please realize you need to read the Maia documentation, and you MUST set up some cron jobs to run the maintenance scripts: https://www.maiamailguard.com/maia/wiki/MaintenanceScripts. I know that for example, you will need to set the full path to reminder.tpl (/var/lib/amavis/maia/templates/reminder.tpl) in the System Configuration or you will get an error when running send-quarantine-reminders.pl. You will need to run process-quarantine.pl hourly and expire-quarantine-cache.pl daily. Other scripts may need to be configured to run also. You will want to remove the admin scripts from the website (you will have a backup in /usr/local/src/maia/1.0/php/admin/):
rm -r /var/www/mail/admin/

Once logged in as the super user, click on the Key icon, go into System Configuration and provide the full path to "E-mail reminder template file:", which is /var/lib/amavis/maia/templates/reminder.tpl. Your crontab entries for the Maia maintenance scripts may look something like this:
# Minute   Hour   Day of Month       Month          Day of Week        Command    
# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)
30 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/process-quarantine.pl --learn --report --quiet'
10 1 * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/expire-quarantine-cache.pl --quiet'
10 2 * * 3 /bin/su amavis -c '/var/lib/amavis/maia/scripts/send-quarantine-reminders.pl'
*/5 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/send-quarantine-digests.pl'
0 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/stats-snapshot.pl' >/null
The SQL based Bayes and AWL tables have also been created for you. You can enable them by placing this in /etc/spamassassin/local.cf (note that I have disabled AWL as SpamAssassin 3.3.x no longer loads the AWL module):
So, vi /etc/spamassassin/local.cf
bayes_store_module              Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn                   DBI:mysql:maia:localhost
bayes_sql_username              amavis
bayes_sql_password              amavis_password

bayes_sql_override_username         amavis

#auto_whitelist_factory          Mail::SpamAssassin::SQLBasedAddrList
#user_awl_dsn                    DBI:mysql:maia:localhost
#user_awl_sql_username           amavis
#user_awl_sql_password           amavis_password
If you have loadplugin Mail::SpamAssassin::Plugin::AWL enabled in /etc/spamassassin/v310.pre then you should uncomment the AWL entries shown above. Save and exit the file, then hide the password:
chmod 0640 /etc/spamassassin/local.cf
chown root:amavis /etc/spamassassin/local.cf


Then test with:
cd
wget http://verchick.com/mecham/public_html/spam/sample-spam.txt
sa-learn --spam sample-spam.txt
sa-learn --dump magic
spamassassin --lint

It should show our nspam (number of spam) count is 1, and --lint should be clean.
Auto-whitelist and bayes_seen grow forever, so we will make some changes to the Bayes database structure so we have a means of cleaning up old data. and We makes some other changes so the database is compatible with SA 3.3.0:
mysql -p

USE maia;
ALTER TABLE `awl` CHANGE `email` `email` VARCHAR( 255 ) NOT NULL;
ALTER TABLE `awl` CHANGE `ip` `ip` VARCHAR( 40 ) NOT NULL;
ALTER TABLE `awl` ADD `lastupdate` timestamp default CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;
UPDATE `awl` SET `lastupdate` = NOW( ) WHERE `lastupdate` < 1;
DELETE FROM `bayes_seen`;
ALTER TABLE `bayes_seen` ADD `lastupdate` timestamp default CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;
OPTIMIZE TABLE `bayes_seen`;
ALTER TABLE `awl` ADD `signedby` VARCHAR( 255 ) NOT NULL;
ALTER TABLE `awl` DROP PRIMARY KEY, ADD PRIMARY KEY ( `username` ,`email` , `signedby` , `ip` );
QUIT


We continue by adding an AWL and bayes_seen maintenance script (watch for errors):
cd /etc
wget http://verchick.com/mecham/public_html/spam/maia-trim-awl.sql.txt
mv maia-trim-awl.sql.txt maia-trim-awl.sql
cd /usr/sbin
wget http://verchick.com/mecham/public_html/spam/maia-trim-awl.txt
mv maia-trim-awl.txt maia-trim-awl
sed -i 's/paSSw0rd/amavis_password/' maia-trim-awl
chmod 0750 maia-trim-awl
cd /etc/cron.weekly/
wget http://verchick.com/mecham/public_html/spam/maia-trim-sql-awl-weekly.txt
mv maia-trim-sql-awl-weekly.txt maia-trim-sql-awl-weekly
chmod +x maia-trim-sql-awl-weekly
./maia-trim-sql-awl-weekly


OPTIONAL: Create and install SSL Certificate and use https:

This assumes you do not already other have web sites and/or certificates set up on this machine. If you do, I would hate to see you mess that up.
apt-get install openssl

Every client that connects to this server will need to be able to resolve the hostname of the server. Hopefully you have already create an A record and an MX record for the server. We are going to be our own Certificate Authority and sign our own certificates. These commands are dependent on /etc/ssl/openssl.cnf as supplied by Debian. We start by making a small change to /etc/ssl/openssl.cnf. We make it so by default our certificates are good for 10 years instead of 1:
sed -i 's/= 365\t/= 3653\t/' /etc/ssl/openssl.cnf
grep 365 /etc/ssl/openssl.cnf


We will set up a common place to put our certificates:
cd /root
mkdir CA
cd CA
mkdir demoCA
cd demoCA
mkdir newcerts
mkdir private
echo '01' > serial
touch index.txt
cd ..


Create a Root Certificate:
openssl req -new -x509 -extensions v3_ca -keyout demoCA/private/cakey.pem -out cacert.pem -days 3653

Enter a passphrase when prompted. You will need this passphrase in the future. What I mean is: make it unique and don't ever loose it. You will be asked questions. Sample answers follow. Be sure to use the full name for your state or province name and the Common Name should be something that describes you as an authority (like Widgits Inc. RootCA):
Country Name US
State or Province Name Utah
Locality Name Salt Lake City
Organisation Name Widgits Inc.
Organisational Unit Name WebMail Server
Common Name (eg, YOUR name) Widgits Inc. RootCA
Email Address postmaster@example.com


This process produces two files as output: a private key in demoCA/private/cakey.pem and a root CA certificate in cacert.pem. Any and all key files we produce will need to be protected from unauthorized persons reading them, and must not be lost for the next 10 years. Also realize that the CA you created can sign any number of certificates (until it expires 10 years from now) so you only need to (or want to) create the CA once. We will copy our cert and our key to something more descriptive:
cp -i demoCA/private/cakey.pem demoCA/private/cakey.example.com.pem
chmod 600 demoCA/private/*
cp -i cacert.pem cacert.example.com.pem
cp -i cacert.pem cacert.example.com.crt


We will copy the root CA certificate to the web server so we can install it on clients by having them browse to it:
cp -i cacert.example.com.crt /var/www/
chmod 0644 /var/www/cacert.example.com.crt


We copy the root CA certificate to /usr/share/ca-certificates/self:
mkdir /usr/share/ca-certificates/self
cp -i cacert.example.com.crt /usr/share/ca-certificates/self/


Now, run dpkg-reconfigure ca-certificates, answer yes to "Trust new certificates from certificate authorities?" and then scroll down to self/cacert.example.com.crt, use the spacebar to select it, Tab to Ok, and press Enter to finish the job. This will create a sym-link to our CA certificate in /etc/ssl/certs:
dpkg-reconfigure ca-certificates

The cacert.example.com.pem and cacert.example.com.crt are copies of our certificate and are the files that can be distributed and installed on the client's machines. Windows clients would use the .crt file. On my Windows 2000 system, double clicking this file would install it in Internet Explorer (which is exactly what want). Simply browsing to our server will give us the opportunity to install the web server certificate we will create (this will be the Common Name sfa.example.com) but this is not the same as installing the CA certificate in the Trusted Root Certification Authorities store (seen as the Common Name you entered above). Just in case you are not familiar, in IE6 it's Tools->Internet Options->Content->Certificates->Trusted Root Certification Authorities. Outlook and Outlook Express use the same certificate store as Internet Explorer. In Mozilla Thunderbird it's Tools->Options->Privacy->Security->View Certificates->Authorities. In Firefox it's Tools->Options->Advanced->Encryption->View Certificates->Authorities->Import. If you go through this process more than once while testing, don't install duplicate certificates. Delete any old 'test' certificate you previously installed before adding your new one that replaces it. In my old version of The Bat! I add a new contact in the "Trusted Root CA" section of the address book and import the certificate from there. I suggest using WinSCP to transfer the cacert.example.com.crt certificate to your machine. I think the worst part of getting this server set up is getting the CA certificates installed on the clients. Sometimes it's worth it to buy a certificate from a well known commercial CA that is already in the Trusted Root Certification Authorities store.

OK, we are a certificate authority. We have the ability to sign our own certificates. We are now going to create a request for a certificate from the CA (which is us - but could be a commercial CA if you like). Everyone that connects to us will connect to the hostname of this machine. The Secure Web server, Secure IMAP server, Secure POP server and Postfix Secure SMTP server will all be sfa.example.com, so the Common Name MUST BE our FQDN hostname when we create the request. The Organization name needs to be the same as the one in the CA cert. Do not enter your email address, challenge password or an optional company name when generating the CSR:
openssl req -new -nodes -out req.pem

Country Name US
State or Province Name Utah
Locality Name Salt Lake City
Organisation Name Widgits Inc.
Organisational Unit Name WebMail Server
Common Name (eg, YOUR name) sfa.example.com
Email Address


This process produces two files as output, a private key in privkey.pem and a certificate signing request in req.pem. These files should be kept. The private key is of course necessary for SSL encryption. We will make backup copies of these files with more descriptive names:
cp -i privkey.pem privkey.sfa.example.com.pem

chmod 600 privkey.*
cp -i req.pem req.sfa.example.com.pem


Sign the Certificate (you will be asked for the pass phrase):
openssl ca -out cert.pem -cert cacert.pem -infiles req.pem

This process updates the CA database and produces two files as output, a certificate in cert.pem and a copy of the certificate in demoCA/newcerts/ named xx.pem, where xx is the serial number. We will copy the cert to a more descriptive name. The certificate has both the encoded version and a human-readable version in the same file. We want to strip off the human-readable portion as follows:
mv -i cert.pem temp.cert.sfa.example.com.pem

openssl x509 -in temp.cert.sfa.example.com.pem -out cert.pem
cp -i cert.pem cert.sfa.example.com.pem


Postfix will want the cert and the key in two separate files, apache2 will want the two combined (but can use two separate files if configured to do so).
cat privkey.sfa.example.com.pem cert.sfa.example.com.pem >key-cert.pem
cp -i key-cert.pem key-cert.sfa.example.com.pem

chmod -R 600 /root/CA


After those steps, you have three installable components (and some more descriptive backup copies):
A private key in privkey.pem (with a copy in privkey.sfa.example.com.pem)
A certificate in cert.pem (with a copy in cert.sfa.example.com.pem)
A combined private key and certificate in key-cert.pem (with a copy in key-cert.sfa.example.com.pem)

Now give a copy of the combined certificate to Apache2. We will configure sfa.example.com in Apache2 to use /etc/apache2/key-cert.sfa.example.com.pem:
/etc/init.d/apache2 restart

cd /root/CA/
cp key-cert.sfa.example.com.pem /etc/apache2/
chmod 600 /etc/apache2/key-cert.sfa.example.com.pem


Now we will configure Apache2. First enable the SSL module, and we will also enable the rewrite module so we can optionally redirect port 80 requests to port 443:
a2enmod ssl
a2enmod rewrite


We will make a copy of the default site. This copy will be used for configuration of the SSL site:
cp /etc/apache2/sites-available/default /etc/apache2/sites-available/ssl

Now edit /etc/apache2/sites-available/default:
vi /etc/apache2/sites-available/default

And change:
<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/
To:
<VirtualHost *:80>
        ServerAdmin webmaster@example.com
        ServerName sfa.example.com
        DocumentRoot /var/www/
Now edit /etc/apache2/sites-available/ssl:
vi /etc/apache2/sites-available/ssl

And change:
<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/
To:
<VirtualHost *:443>
        ServerAdmin webmaster@example.com
        ServerName sfa.example.com
        SSLEngine on
        SSLCertificateFile /etc/apache2/key-cert.sfa.example.com.pem
        DocumentRoot /var/www/
Once the files have been edited, enable the new site we called ssl, and restart Apache2:
a2ensite ssl
/etc/init.d/apache2 restart


If you did it like I did it, you should have no errors when it shuts down or starts up. If it does not start up, you can disable the ssl site with 'a2dissite ssl'. See /var/log/apache2/error.log for clues to the problem. Now you should be able to browse to both: http://sfa.example.com/mail and https://sfa.example.com/mail.

If you have not already installed the Root CA certificate on your computer, see if you can install it by downloading it via your browser at http://sfa.example.com/cacert.example.com.crt

Note that if you created a root certificate and installed it on your computer, and then started this process over and created a new one with the same name, you would have to remove the duplicate "test" certificate and install the new certificate. I have enabled port 80 only as a convenience. We don't want people to connect to our site without using SSL, so I am going to set up redirection. This is optional, but highly recommended. Edit /etc/apache2/sites-available/default once again:
vi /etc/apache2/sites-available/default

And insert these additional items in the location shown:
<VirtualHost *:80>
        RewriteEngine   on
        RewriteCond     %{SERVER_PORT} ^80$
        RewriteRule     ^(.*)$ https://%{SERVER_NAME}$1 [L,R]
        RewriteLog      "/var/log/apache2/rewrite.log"
        RewriteLogLevel 2
Then restart Apache2:
/etc/init.d/apache2 restart

Now, if you close down the browser window for http://sfa.example.com and then browse to it again, you should be redirected to the SSL site. This is a bit wasteful however as twice as many processes are required to maintain both port 80 and port 443. If you like, you can optionally prevent Apache2 from starting up processes on port 80 by editing /etc/apache2/ports.conf and commenting out 'Listen 80' then restarting Apache2. Of course, then your users will always have to remember to use the https:// URL. BTW, personally I use lsof -P | grep LISTEN to show what ports are in use. Note that I am an Apache noob. If you want to host multiple http sites with multiple certificates, I leave it up to you to figure out how to do it. I can tell you that you can create and sign as many certificates as you want, just make sure you do not create a new certificate authority. The one and only certificate authority you created can sign all your certificates. And from what I understand, if more than one site uses SSL, each site will need a different IP address. Google is your friend. To be clear, you can host as many email domains as you need, but in my setup, I access all of them through one and only one http interface. Let's continue on...

Tell Postfix where to find the certificates (and set a couple other TLS parameters). We also make a backup of main.cf before we modify it for the first time:
cd /root/CA/
cp -i cert.sfa.example.com.pem /etc/ssl/certs

cp -i privkey.sfa.example.com.pem /etc/ssl/private

cp -i /etc/postfix/main.cf /etc/postfix/main.cf-11dec2010

postconf -e "smtpd_tls_cert_file = /etc/ssl/certs/cert.sfa.example.com.pem"
postconf -e "smtpd_tls_key_file = /etc/ssl/private/privkey.sfa.example.com.pem"
postconf -e "smtpd_tls_CAfile = /etc/ssl/certs/cacert.example.com.pem"
postconf -e "smtpd_tls_received_header = yes"
postfix reload
tail -f /var/log/mail.log


And make sure Postfix is still working OK. Running some of these commands again will result in overwriting keys and certificates. That may not be good. Some files will necessarily be overwritten if additional certificates are requested, signed and created. That is expected, and is the reason we make host-specific copies of everything as we go along. Just be careful not to overwrite any host-specific files we have created. And remember, only one Root Certificate Authority needs creation. Make a backup of the session, both on and off the system (transfer the directory via WinSCP).
cp -r /root/CA /root/CA-12dec2010




PS:
Here is a hint if you would like to change the password used by amavis to access the MySQL database. The default MySQL password for the amavis user is 'passwd'.

Stop Apache2:
/etc/init.d/apache2 stop

Stop Maia (amavisd-new):
/etc/init.d/amavis stop

Locate and change the $maia_sql_dsn password in config.php:
vi /var/www/mail/config.php

Locate and change the @lookup_sql_dsn password in amavisd.conf:
vi /etc/amavis/amavisd.conf

Locate and change the password in /etc/maia.conf:
vi /etc/amavis/maia.conf

Locate and change the password in /etc/spamassassin/local.cf:
vi /etc/spamassassin/local.cf

Locate and change the password in /usr/sbin/maia-trim-awl
vi /usr/sbin/maia-trim-awl

Log in to mysql and change the password there, obviously, replace NEW-passwd with the new password:
mysql -p
SET PASSWORD FOR 'amavis'@'localhost'= PASSWORD('NEW-passwd');
FLUSH PRIVILEGES;
quit

Run configtest.pl to make sure the "Database DSN test" passes:
/var/lib/amavis/maia/scripts/configtest.pl

Start Maia (amavisd-new):
/etc/init.d/amavis start

Start Apache2:
/etc/init.d/apache2 start

It would be a good idea to run configtest.php:
http://sfa.example.com/mail/admin/configtest.php


Useful links:
You used 'subversion' (svn) to 'check out' the current version of Maia Mailguard.
To see what 'subversion' is capable of, see: http://svnbook.red-bean.com/

mr88talent at yahoo dot com
12 DEC 2010