Installing Maia Mailguard 1.0.3 on a Debian lenny mail server


Absolutely no warranty, use entirely at your own risk. See the disclaimer at http://verchick.com/mecham/public_html/spam/

Since this is a copy and paste document, you need to replace the SQL password amavis will use to access the maia database. Save this document to your computer. If using Internet Explorer, save as "Web Page, HTML only (*.htm,*,html)" then use a plain text editor - like Wordpad for example, and replace the text amavis_password with a password of your choice. You also need to replace the text sfa with the hostname of your server, and the text example.com with the domain name of your server. lastly, replace the text "Widgits Inc." with the name for your company. Once completed, open this page from the copy you saved. Do not replace these items more than once.

This document assumes you built a spamfilter server using my document: http://verchick.com/mecham/public_html/spam/spamfilter20090215.html. That document uses a custom installation of amavisd-new that makes use of amavisd.conf for settings. The normal Debian/Ubuntu packaged version of amavisd-new would not use amavisd.conf but would instead use a number of files in the /etc/amavis/conf.d directory. Test to see if you have amavisd.conf:
ls -l /etc/amavis/amavisd.conf

If you do not have an amavisd.conf file, you should preferably follow the document at http://verchick.com/mecham/public_html/spam/ubuntu104-maia.html.


This HOWTO is a quick and dirty guide to getting Maia Mailguard version 1.0.3 installed on a Debian lenny gateway email server that was originally built using http://verchick.com/mecham/public_html/spam/spamfilter20090215.html as a guide. This document is based on Debian lenny stable. This guide is designed to get someone past the initial brain damage of getting the right programs and files in the right place. You need to start with a fully functional amavisd-new installation per those (or similar) instructions.

You must read https://www.maiamailguard.com/maia/wiki/Install so you will understand what we are trying to accomplish. Keep in mind that Maia will replace amavisd-new. Maia and amavisd-new are two different programs and only one or the other can be used - not both. This guide does not cover important configuration settings that need to be made once the software is in place; it just gets you to the point where you can log into Maia. This guide does not explain how to use Maia Mailguard, it merely gets it up and running. It uses many of the default settings (which may not filter mail). I don't use Maia myself, so I cannot be helpful regarding the configuration or use of the program. I also don't use Apache, and I am not skilled at MySQL. There is no support for this document, but you can contact me at 'mr88talent at yahoo dot com' if you have a correction or comment. There is also the forum at http://www.freespamfilter.org/forum/ but for help with Maia, you should join the Maia users mailing list http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users.

There is a 10MB .PDF available from http://www.novell.com/coolsolutions/feature/16093.html that has some good tips for configuring and using Maia. This is a SuSE document, but the Maia stuff should give you a number of good ideas.

The first thing to do is backup our current amavisd-new config file and program file:
cp -i /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-debian
cp /usr/sbin/amavisd-new /usr/sbin/amavisd-new-debian


We want to turn off automatic upgrades of amavisd-new, so we place the package on hold:
echo "amavisd-new hold" | dpkg --set-selections

We need to make a symbolic link (if we don't already have one) because Maia will want to use /etc/amavisd.conf, not /etc/amavis/amavisd.conf. If you followed my spamfilter document, this will result in an error "File exists" wich is nothing to be concerned with:
ln -s /etc/amavis/amavisd.conf /etc/amavisd.conf

Download revision 1518 Maia 1.0.2a. You can browse https://www.maiamailguard.com/svn/branches/1.0 and make a note of the revision of the trunk you may choose to install today. Also see https://www.maiamailguard.com/maia/browser/branches/1.0. You can pick another revision, but be aware that this document is based on revision 1518. The newest stable build I can recommend is 1518. There are a lot of changes after that so a newer version will not work with this document. OK, let's get started:
apt-get update
apt-get install subversion

mkdir /usr/local/src/maia
cd /usr/local/src/maia


If you choose a revision other than 1518, keep the number handy:
svn -r 1518 checkout https://www.maiamailguard.com/svn/branches/1.0

Answer (p) if it asks...
If you ever use this method to download Maia again, you should first move the old downloaded files to another directory to avoid overwriting them. Read this thread. Now, install MySQL 5.0 (if you do not have MySQL installed):
apt-get install mysql-server

Supply a password for root when prompted - and remember it!
If you need to manually set the password: Change the roots_password entry to a password of your choice and make sure the host name is correct:
mysql

SET PASSWORD FOR 'root'@'localhost' = PASSWORD('roots_password');
SET PASSWORD FOR 'root'@'sfa' = PASSWORD('roots_password');
FLUSH PRIVILEGES;
QUIT


Out of the box, MySQL is poorly tuned for use with InnoDB databases like Maia uses. I will assume you have at least 1GB of RAM. In the [mysqld] section of /etc/mysql/my.cnf we need to add some entries. By default, innodb_buffer_pool_size is only 8MB. This makes data access with InnoDB very slow. You can set this at somewhere around 25% of physical RAM. Then, innodb_log_file_size should be set to 25% of innodb_buffer_pool_size. Here I'm going to use 128MB for innodb_buffer_pool_size and 32MB for innodb_log_file_size. The change to innodb_log_file_size will not have any affect however until we have MySQL create new logs. Therefore, we have to shut MySQL down, rename the logs and then let MySQL create new logs. Also, we need to set max_allowed_packet to the maximum size of a message that will be allowed into our system. Here I'm setting it at a reasonably large 20MB. Before deleting the log files, to avoid loss of data, log into MySQL and verify that innodb_fast_shutdown is set to 1:
mysql -p

SHOW VARIABLES LIKE 'innodb_fast_shutdown';
QUIT


If it is not set to 1, you would have to edit /etc/mysql/my.cnf and find where this has been changed from the default, then restart MySQL. Now shut dowm MySQL and rename the log files (provided there were no errors when it shut down):
/etc/init.d/mysql stop
mv /var/lib/mysql/ib_logfile0 /var/lib/mysql/iblogfile0-old
mv /var/lib/mysql/ib_logfile1 /var/lib/mysql/iblogfile1-old


Then, edit /etc/mysql/my.cnf and place these entries (after adjusting if needed) in the location shown:
vi /etc/mysql/my.cnf

# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
# You might want to disable InnoDB to shrink the mysqld process by circa 100MB.
#skip-innodb
#
innodb_buffer_pool_size = 128M
innodb_additional_mem_pool_size = 4M
innodb_log_file_size = 32M
innodb_log_buffer_size = 8M
max_allowed_packet = 20M

# * Security Features
Save and exit the file, then start MySQL:
/etc/init.d/mysql start
ls -l /var/lib/mysql/ib_logfile*


Install more needed programs:
apt-get install libcrypt-blowfish-perl libcrypt-cbc-perl libossp-uuid-perl libtemplate-perl libwww-perl libtext-csv-perl

All of these programs should already be installed, but you can run this just in case:
apt-get install libdigest-sha1-perl libhtml-parser-perl libdbd-mysql-perl libdbi-perl libunix-syslog-perl libio-stringy-perl libnet-server-perl libmailtools-perl libmime-perl libconvert-uulib-perl libconvert-tnef-perl libarchive-zip-perl libarchive-tar-perl curl rsync

We install some others:
apt-get install cabextract libberkeleydb-perl libdigest-sha1-perl libdigest-hmac-perl libnet-dns-perl pax

Install PHP5, Pear and some additional Pear modules. This may also install/upgrade x11-common.
Be forewarned you may get this warning. This should not be an issue if you are not using a GUI.
apt-get install php5 php-pear php5-common php5-mysql php5-gd php5-sqlite smarty

pear channel-update pear.php.net

pear install Mail_Mime-1.6.0
pear install Mail_mimeDecode-1.5.1
pear install Log-1.12.0
pear install Pager-2.4.8

pear install Image_Color-1.0.3
pear install Image_Canvas-0.3.1
pear install Image_Graph-0.7.2
pear install Numbers_Roman-1.0.2
pear install Numbers_Words-0.16.1

pear install Auth_SASL-1.0.4
pear install Net_Socket-1.0.9
pear install Net_IMAP-1.1.0
pear install Net_POP3-1.3.7
pear install DB-1.7.13
pear install Net_SMTP-1.4.1

pear channel-discover htmlpurifier.org
pear install hp/HTMLPurifier

There is a bug in Pie.php provided with Image_Graph-0.7.2. If you have installed this version of Image_Graph, grab a file from me to fix the bug:
cd /usr/share/php/Image/Graph/Plot/
cp Pie.php Pie.php.original
wget http://verchick.com/mecham/public_html/spam/Pie.php.patch.txt
patch -p0<Pie.php.patch.txt


Other files in Image_Graph need to be patched for PHP 5.3.x:
cd /usr/share/php/Image/
wget http://verchick.com/mecham/public_html/spam/Graph.php.patch.txt
patch Graph.php < Graph.php.patch.txt

cd /usr/share/php/Image/Graph/
wget http://verchick.com/mecham/public_html/spam/Layout.php.patch.txt
patch Layout.php < Layout.php.patch.txt

wget http://verchick.com/mecham/public_html/spam/Axis.php.patch.txt
patch Axis.php < Axis.php.patch.txt

When the time comes you want to view Pie charts, enable them with Settings (the gear)->Miscellaneous Settings->Display graphic charts? AFTER you "Enable graphical charts?" in the Charts section in Admin (the key)->Systemconfiguration.

Install Apache2
apt-get install apache2 libapache2-mod-php5

Change the firewall rules to allow access to ports 80 and 443. Edit this to reflect your network (and any other custom modifications you made to the original version), or this will lock you out! Please see http://verchick.com/mecham/public_html/spam/debian-smtp-firewall.html if this does not look familiar to you. It is quite likely you have your firewall set up using some other means, but the idea remains that you need to open up tcp port 80:

iptables -F
iptables -N FIREWALL
iptables -F FIREWALL
iptables -A INPUT -j FIREWALL
iptables -A FORWARD -j FIREWALL
iptables -A FIREWALL -p tcp -m tcp --dport 25 --syn -j ACCEPT
iptables -A FIREWALL -p tcp -m tcp -s 222.222.222.222/24 --dport 22 --syn -j ACCEPT
iptables -A FIREWALL -i lo -j ACCEPT
iptables -A FIREWALL -p udp -m udp --sport 53 -j ACCEPT
iptables -A FIREWALL -p tcp -m tcp --sport 53 -j ACCEPT
iptables -A FIREWALL -p tcp -m tcp --dport 80 --syn -j ACCEPT
iptables -A FIREWALL -p udp -m udp --dport 123 -j ACCEPT
iptables -A FIREWALL -p udp -m udp --sport 123 -j ACCEPT
iptables -A FIREWALL -p tcp -m tcp --dport 443 --syn -j ACCEPT
iptables -A FIREWALL -p udp -m udp --sport 6277 -j ACCEPT
iptables -A FIREWALL -p udp -m udp --sport 24441 -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type destination-unreachable -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type source-quench -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type time-exceeded -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type parameter-problem -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type echo-reply -j ACCEPT
iptables -A FIREWALL -p icmp --icmp-type echo-request -j ACCEPT
iptables -A FIREWALL -p tcp -m tcp --syn -j REJECT
iptables -A FIREWALL -p udp -m udp -j REJECT
iptables -A FIREWALL -p icmp -j DROP
iptables-save > /etc/firewall-rules
iptables-restore < /etc/firewall-rules

You should now be able to open a browser and browse to the IP address of the spamfilter. I recommend adding the IP address and FQDN hostname of the spamfilter to your hosts file, so you can browse the server using the hostname instead of the IP address. Of course, eventually you will need to set up the host in DNS if you have not already done so. If this is a production server, obviously you already have.

We need to create some directories and copy some files per the INSTALL instructions. We use 'sed' to change /var/amavisd to /var/lib/amavis in a few files:
mkdir /var/lib/amavis/maia
chmod -R 750 /var/lib/amavis/maia
mkdir /var/lib/amavis/maia/scripts
chmod -R 750 /var/lib/amavis/maia/scripts
cd /var/lib/amavis/maia/scripts
cp /usr/local/src/maia/1.0/scripts/* .
sed -i 's|var/amavisd|var/lib/amavis|g' load-sa-rules.pl
sed -i 's|var/amavisd|var/lib/amavis|g' process-quarantine.pl
sed -i 's|var/amavisd|var/lib/amavis|g' send-quarantine-digests.pl
sed -i 's|var/amavisd|var/lib/amavis|g' expire-quarantine-cache.pl
sed -i 's|var/amavisd|var/lib/amavis|g' send-quarantine-reminders.pl
sed -i 's|var/amavisd|var/lib/amavis|g' stats-snapshot.pl
test -e /etc/maia.conf && mv /etc/maia.conf /etc/amavis/maia.conf-backup
cp /usr/local/src/maia/1.0/maia.conf.dist /etc/amavis/maia.conf
ln -s /etc/amavis/maia.conf /etc/maia.conf
sed -i 's|var/amavisd|var/lib/amavis|g' /etc/amavis/maia.conf
sed -i 's|passwd|amavis_password|' /etc/amavis/maia.conf
chmod 0640 /etc/amavis/maia.conf
chgrp amavis /etc/amavis/maia.conf
cp /usr/local/src/maia/1.0/*.cf /etc/spamassassin
cp -r /usr/local/src/maia/1.0/templates /var/lib/amavis/maia/templates
chown -R amavis:amavis /var/lib/amavis
chmod 640 /var/lib/amavis/maia/templates/*.tpl
cp /usr/local/src/maia/1.0/amavisd-maia /usr/sbin

Now we need to edit maia.conf to reflect what our URL will be. We are going to place our files in a /mail/ directory. The use of the vi editor is only a suggestion, of course you can use whatever editor you please:
vi /etc/amavis/maia.conf

$base_url = "http://sfa.example.com/mail/";


Save and exit the file. We need to log into MySQL and create the maia database:
mysql -p

CREATE DATABASE maia;
USE maia;


Be careful that the next command points to the correct location of "maia-mysql.sql":
SOURCE /usr/local/src/maia/1.0/maia-mysql.sql;

You should have seen a number of lines similar to "Query OK, 1 row affected (0.01 sec)" scroll by.
Now we create the amavis user and then quit (there are 3 commands here):
GRANT CREATE, DROP, ALTER, SELECT, INSERT, UPDATE, DELETE ON maia.* TO amavis@localhost IDENTIFIED BY 'amavis_password';
FLUSH PRIVILEGES;
QUIT

Let's see if our configuration looks OK so far:
cd /var/lib/amavis/maia/scripts/
./configtest.pl

This should result in something like:
Application/Module      Version   Status
========================================================================
Perl                 :   5.10.1 : OK
file(1)              :     5.03 : OK
Archive::Tar         :     1.52 : OK
Archive::Zip         :     1.30 : OK
BerkeleyDB           :     0.39 : OK
Compress::Zlib       :     2.02 : OK
Convert::TNEF        :     0.17 : OK
Convert::UUlib       :     1.12 : OK
Crypt::Blowfish      :      N/A : NOT INSTALLED ... 
Crypt::CBC           :      N/A : NOT INSTALLED ... 
Crypt::OpenSSL::RSA  :     0.25 : OK
Data::UUID           :   1.0602 : OK
DB_File              :     1.82 : OK
DBD::mysql           :    4.012 : OK
DBD::Pg              :      N/A : NOT INSTALLED ...
DBI                  :    1.609 : OK
Digest::MD5          :     2.39 : OK
Digest::SHA1         :     2.12 : OK
File::Spec           :      3.3 : OK
HTML::Parser         :     3.64 : OK
HTTP::Date           :    5.831 : OK
IO::Stringy          :    2.110 : OK
IO::Zlib             :     1.09 : OK
IP::Country          :      N/A : NOT INSTALLED ... 
LWP::UserAgent       :    5.834 : OK
Mail::Address        :     2.05 : OK
Mail::DKIM           :     0.38 : OK
Mail::Internet       :     2.05 : OK
Mail::SpamAssassin   :    3.3.1 : OK
Mail::SPF::Query     :      N/A : NOT INSTALLED ...
MIME::Base64         :     3.08 : OK
MIME::Parser         :    5.427 : OK
MIME::QuotedPrint    :     3.08 : OK
Net::CIDR::Lite      :      N/A : NOT INSTALLED .. 
Net::Cmd             :     2.29 : OK
Net::DNS             :     0.65 : OK
Net::Server          :     0.97 : OK
Net::SMTP            :     2.31 : OK
Pod::Usage           :     1.36 : OK
Template             :     2.20 : OK
Time::HiRes          :   1.9719 : OK
Unix::Syslog         :      1.1 : OK
URI                  :     1.52 : OK
Text::CSV            :     1.16 : OK

Database DSN test    : PASSED
We do not need the six items shown as NOT INSTALLED. We don't use PostgreSQL, so we don't need DBD::Pg

Now we copy SpamAssassin's rules to the MySQL database (make sure 'spamassassin --lint' does not return any errors; if it does, repair the errors before you continue):
su amavis -c 'spamassassin --lint'

Since we are using SpamAssassin 3.1.1 or greater we can use the new sa-update feature:
sa-update

cd /var/lib/amavis/maia/scripts
./load-sa-rules.pl


You should have seen a bunch of rules get loaded; if not, then check that this script points to the correct directories (you edited it earlier). Now we will copy the web interface files to the web site. We also need to make 'themes' writable by the www-data user (and amavis):
mkdir /var/www/mail
cp -r /usr/local/src/maia/1.0/php/* /var/www/mail
ln -s /usr/share/php/smarty/libs /usr/share/php/Smarty
cp /var/www/mail/config.php.dist /var/www/mail/config.php
sed -i "s/passwd/amavis_password/g" /var/www/mail/config.php
chmod 0640 /var/www/mail/config.php
chown root:www-data /var/www/mail/config.php
chgrp amavis /var/www/mail/themes/*/compiled
chmod 775 /var/www/mail/themes/*/compiled
gpasswd -a www-data amavis

mkdir /var/www/mail/purifier_cache
chown www-data:www-data /var/www/mail/purifier_cache
sed -i 's|purifier_cache = null;|purifier_cache = "/var/www/mail/purifier_cache";|' /var/www/mail/config.php

/etc/init.d/apache2 restart
/etc/init.d/postfix restart


Now you browse to configtest.php to test the installation, for example:
http://192.168.1.222/mail/admin/configtest.php
(or)
http://sfa.example.com/mail/admin/configtest.php

The result should look something like this. Note: if you need the IMAP, LDAP or MCrypt libraries you are free to
apt-get install php5-imap php5-ldap php5-mcrypt (but only as needed). Also note: we have already applied the patche to Image_Graph. Also, remember to stop and start apache2 if you make changes.

You should ensure the database schema is up to date:
http://sfa.example.com/mail/admin/upgrade.php


Make sure we have the proper init script:
cd /etc/init.d
wget http://verchick.com/mecham/public_html/debian/amavis.init.lenny.txt
mv amavis.init.lenny.txt amavis
chmod +x amavis

We will start off with a configuration file specific to Maia. I have a patch that modifies the file for use with Debian. Make sure the patch applies cleanly:
cd /etc/amavis
cp /usr/local/src/maia/1.0/amavisd.conf.dist .
cp amavisd.conf.dist amavisd.conf-maia
wget http://verchick.com/mecham/public_html/spam/amavisd.conf.1500.patch.txt
patch amavisd.conf-maia < amavisd.conf.1500.patch.txt
sed -i "s/'password'/'amavis_password'/" amavisd.conf-maia
sed -i "s/yourdomain.tld/example.com/" amavisd.conf-maia
sed -i "s/host.domain.tld/sfa.example.com/" amavisd.conf-maia
chmod 0640 amavisd.conf-maia


We need to make a few changes to amavisd.conf-maia:
vi /etc/amavis/amavisd.conf-maia

I suggest you place the SVN release number of Maia in a comment in this file. Make sure your $max_servers setting matches the maxproc setting for the smtp-amavis transport in /etc/postfix/master.cf. If you have not configured this amavisd.conf-maia in the past, at the very least you need check $mydomain. I also suggest you check $myhostname. Note that you may also wish to temporarily set $log_level to 5 when we send a few test messages through, examining /var/log/mail.log for signs of trouble. Also, please read section "12. Configure amavisd-maia" of the Maia installation instructions for other suggested amavisd-maia settings. Blowfish encryption is NOT recommended. Save and exit the file, and then make a backup copy:
cp /etc/amavis/amavisd.conf-maia /etc/amavis/amavisd.conf-maia-backup

Now, when you want to try Maia as your program, you first stop amavisd-new:
/etc/init.d/amavis stop

Then you copy the Maia configuration file on top of the amavisd-new configuration and change the init scrip to start up amavisd-maia:
cp -p /etc/amavis/amavisd.conf-maia /etc/amavis/amavisd.conf
sed -i "s/amavisd-new/amavisd-maia/" /etc/init.d/amavis


Then you start up amavisd-new (Maia) (actually, the first time you start it up it would be a good idea to do so with 'amavisd-new debug'):
/etc/init.d/amavis start

You can keep an eye out for errors by using:
tail -f /var/log/mail.log

Send a few test messages through and look for errors. Since the system has not been configured, the default is to pass everthing (including viruses and executables).

If you had amavisd-new installed and should ever need to revert to it, first you would stop amavisd-maia, copy the Debian amavisd.conf on top of the Maia amavisd.conf, and start up amavisd-new, like this:
/etc/init.d/amavis stop
cp /etc/amavis/amavisd.conf-debian /etc/amavis/amavisd.conf
sed -i "s/amavisd-maia/amavisd-new/" /etc/init.d/amavis
/etc/init.d/amavis start

Then to switch back to Maia:
/etc/init.d/amavis stop
cp /etc/amavis/amavisd.conf-maia /etc/amavis/amavisd.conf
sed -i "s/amavisd-new/amavisd-maia/" /etc/init.d/amavis
/etc/init.d/amavis start

To avoid losing configuration changes, you should always make configuration changes to amavisd.conf-debian or amavisd.conf-maia and then copy the one you are currently using to amavisd.conf.

At this point, nothing is set up in Maia, so If I am correct, all mail will simply pass. I believe the default behavior is to bypass all checks for recipients not in the database so you will quickly want to add and configure your users and/or domains.

Continue on by going to https://www.maiamailguard.com/maia/wiki/Install and start reading from "14. Login and become the super-administrator". You want to use your email address as the "E-mail address your login credentials should be mailed to:" (so you get the initial super-administrator password sent to you). My internal-init.php page looked similar to this example. Since you are starting out using the "Internal" authentication method, run the
http://sfa.example.com/mail/internal-init.php   script before running the
http://sfa.example.com/mail/login.php?super=register  script.   If all goes well, you will get a "250 Ok" response on the screen and your password in a message in your inbox. Place your email address in the "E-mail address your login credentials should be mailed to" box. Run the super-register script shown above next. Don't forget you need to set up DNS (if you have not done so) so your users can use your server's FQDN to access Maia.



Now, please realize you need to read the Maia documentation, and you must set up some cron jobs to run the maintenance scripts: https://www.maiamailguard.com/maia/wiki/MaintenanceScripts. I know that for example, you will need to set the full path to reminder.tpl (/var/lib/amavis/maia/templates/reminder.tpl) in the System Configuration or you will get an error when running send-quarantine-reminders.pl. You will need to run process-quarantine.pl hourly and expire-quarantine-cache.pl daily. Other scripts may need to be configured to run also. You will want to remove the admin scripts from the website (you will have a backup in /usr/local/src/maia/1.0/php/admin/):
rm -r /var/www/mail/admin/

Once logged in as the super user, click on the Key icon, go into System Configuration and provide the full path to "E-mail reminder template file:", which is /var/lib/amavis/maia/templates/reminder.tpl. Your crontab entries for the Maia maintenance scripts may look something like this:
# Minute   Hour   Day of Month       Month          Day of Week        Command    
# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)
30 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/process-quarantine.pl --learn --report --quiet'
10 1 * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/expire-quarantine-cache.pl --quiet'
10 2 * * 3 /bin/su amavis -c '/var/lib/amavis/maia/scripts/send-quarantine-reminders.pl'
*/5 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/send-quarantine-digests.pl'
0 * * * * /bin/su amavis -c '/var/lib/amavis/maia/scripts/stats-snapshot.pl' >/null
The SQL based Bayes and AWL tables have also been created for you. You can enable them by placing this in /etc/spamassassin/local.cf:
So, vi /etc/spamassassin/local.cf
bayes_store_module              Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn                   DBI:mysql:maia:localhost
bayes_sql_username              amavis
bayes_sql_password              amavis_password

bayes_sql_override_username         amavis

#auto_whitelist_factory          Mail::SpamAssassin::SQLBasedAddrList
#user_awl_dsn                    DBI:mysql:maia:localhost
#user_awl_sql_username           amavis
#user_awl_sql_password           amavis_password
If you have loadplugin Mail::SpamAssassin::Plugin::AWL enabled in /etc/spamassassin/v310.pre then you should uncomment the AWL entries shown above. Save and exit the file, then hide the password:
chmod 0640 /etc/spamassassin/local.cf
chown root:amavis /etc/spamassassin/local.cf


Then test with:
cd
wget http://verchick.com/mecham/public_html/spam/sample-spam.txt
sa-learn --spam sample-spam.txt
sa-learn --dump magic
spamassassin --lint

It should show our nspam (number of spam) count is 1, and --lint should be clean.
Auto-whitelist and bayes_seen grow forever, so we will make some changes to the Bayes database structure so we have a means of cleaning up old data, and We makes some other changes so the database is compatible with SA 3.3.0:
mysql -p

USE maia;
ALTER TABLE `awl` CHANGE `email` `email` VARCHAR( 255 ) NOT NULL;
ALTER TABLE `awl` CHANGE `ip` `ip` VARCHAR( 40 ) NOT NULL;
ALTER TABLE `awl` ADD `lastupdate` timestamp(14) NOT NULL;
UPDATE `awl` SET `lastupdate` = NOW( ) WHERE `lastupdate` < 1;
DELETE FROM `bayes_seen`;
ALTER TABLE `bayes_seen` ADD `lastupdate` timestamp(14) NOT NULL;
OPTIMIZE TABLE `bayes_seen`;
ALTER TABLE `awl` ADD `signedby` VARCHAR( 255 ) NOT NULL;
ALTER TABLE `awl` DROP PRIMARY KEY, ADD PRIMARY KEY ( `username` ,`email` , `signedby` , `ip` );
QUIT


We continue by adding an AWL and bayes_seen maintenance script (watch for errors):
cd /etc
wget http://verchick.com/mecham/public_html/spam/maia-trim-awl.sql.txt
mv maia-trim-awl.sql.txt maia-trim-awl.sql
cd /usr/sbin
wget http://verchick.com/mecham/public_html/spam/maia-trim-awl.txt
mv maia-trim-awl.txt maia-trim-awl
sed -i 's/paSSw0rd/amavis_password/' maia-trim-awl
chmod 0750 maia-trim-awl
cd /etc/cron.weekly/
wget http://verchick.com/mecham/public_html/spam/maia-trim-sql-awl-weekly.txt
mv maia-trim-sql-awl-weekly.txt maia-trim-sql-awl-weekly
chmod +x maia-trim-sql-awl-weekly
./maia-trim-sql-awl-weekly


OPTIONAL: Create and install SSL Certificate and use https:

This assumes you do not already other have web sites and/or certificates set up on this machine. If you do, I would hate to see you mess that up.
apt-get install openssl

Every client that connects to this server will need to be able to resolve the hostname of the server. Hopefully you have already create an A record and an MX record for the server. We are going to be our own Certificate Authority and sign our own certificates. These commands are dependent on /etc/ssl/openssl.cnf as supplied by Debian. We start by making a small change to /etc/ssl/openssl.cnf. We make it so by default our certificates are good for 10 years instead of 1:
sed -i 's/= 365\t/= 3653\t/' /etc/ssl/openssl.cnf
grep 365 /etc/ssl/openssl.cnf


We will set up a common place to put our certificates:
cd /root
mkdir CA
cd CA
mkdir demoCA
cd demoCA
mkdir newcerts
mkdir private
echo '01' > serial
touch index.txt
cd ..


Create a Root Certificate:
openssl req -new -x509 -extensions v3_ca -keyout demoCA/private/cakey.pem -out cacert.pem -days 3653

Enter a passphrase when prompted. You will need this passphrase in the future. What I mean is: make it unique and don't ever loose it. You will be asked questions. Sample answers follow. Be sure to use the full name for your state or province name and the Common Name should be something that describes you as an authority (like Widgits Inc. RootCA):
Country Name US
State or Province Name Utah
Locality Name Salt Lake City
Organisation Name Widgits Inc.
Organisational Unit Name WebMail Server
Common Name (eg, YOUR name) Widgits Inc. RootCA
Email Address postmaster@example.com


This process produces two files as output: a private key in demoCA/private/cakey.pem and a root CA certificate in cacert.pem. Any and all key files we produce will need to be protected from unauthorized persons reading them, and must not be lost for the next 10 years. Also realize that the CA you created can sign any number of certificates (until it expires 10 years from now) so you only need to (or want to) create the CA once. We will copy our cert and our key to something more descriptive:
cp -i demoCA/private/cakey.pem demoCA/private/cakey.example.com.pem
chmod 600 demoCA/private/*
cp -i cacert.pem cacert.example.com.pem
cp -i cacert.pem cacert.example.com.crt


We will copy the root CA certificate to the web server so we can install it on clients by having them browse to it:
cp -i cacert.example.com.crt /var/www/
chmod 0644 /var/www/cacert.example.com.crt


We copy the root CA certificate to /usr/share/ca-certificates/self:
mkdir /usr/share/ca-certificates/self
cp -i cacert.example.com.crt /usr/share/ca-certificates/self/


Now, run dpkg-reconfigure ca-certificates, answer yes to "Trust new certificates from certificate authorities?" and then scroll down to self/cacert.example.com.crt, use the spacebar to select it, Tab to Ok, and press Enter to finish the job. This will create a sym-link to our CA certificate in /etc/ssl/certs:
dpkg-reconfigure ca-certificates

The cacert.example.com.pem and cacert.example.com.crt are copies of our certificate and are the files that can be distributed and installed on the client's machines. Windows clients would use the .crt file. On my Windows 2000 system, double clicking this file would install it in Internet Explorer (which is exactly what want). Simply browsing to our server will give us the opportunity to install the web server certificate we will create (this will be the Common Name sfa.example.com) but this is not the same as installing the CA certificate in the Trusted Root Certification Authorities store (seen as the Common Name you entered above). Just in case you are not familiar, in IE6 it's Tools->Internet Options->Content->Certificates->Trusted Root Certification Authorities. Outlook and Outlook Express use the same certificate store as Internet Explorer. In Mozilla Thunderbird it's Tools->Options->Privacy->Security->View Certificates->Authorities. In Firefox it's Tools->Options->Advanced->Encryption->View Certificates->Authorities->Import. If you go through this process more than once while testing, don't install duplicate certificates. Delete any old 'test' certificate you previously installed before adding your new one that replaces it. In my old version of The Bat! I add a new contact in the "Trusted Root CA" section of the address book and import the certificate from there. I suggest using WinSCP to transfer the cacert.example.com.crt certificate to your machine. I think the worst part of getting this server set up is getting the CA certificates installed on the clients. Sometimes it's worth it to buy a certificate from a well known commercial CA that is already in the Trusted Root Certification Authorities store.

OK, we are a certificate authority. We have the ability to sign our own certificates. We are now going to create a request for a certificate from the CA (which is us - but could be a commercial CA if you like). Everyone that connects to us will connect to the hostname of this machine. The Secure Web server, Secure IMAP server, Secure POP server and Postfix Secure SMTP server will all be sfa.example.com, so the Common Name MUST BE our FQDN hostname when we create the request. The Organization name needs to be the same as the one in the CA cert. Do not enter your email address, challenge password or an optional company name when generating the CSR:
openssl req -new -nodes -out req.pem

Country Name US
State or Province Name Utah
Locality Name Salt Lake City
Organisation Name Widgits Inc.
Organisational Unit Name WebMail Server
Common Name (eg, YOUR name) sfa.example.com
Email Address


This process produces two files as output, a private key in privkey.pem and a certificate signing request in req.pem. These files should be kept. The private key is of course necessary for SSL encryption. We will make backup copies of these files with more descriptive names:
cp -i privkey.pem privkey.sfa.example.com.pem

chmod 600 privkey.*
cp -i req.pem req.sfa.example.com.pem


Sign the Certificate (you will be asked for the pass phrase):
openssl ca -out cert.pem -cert cacert.pem -infiles req.pem

This process updates the CA database and produces two files as output, a certificate in cert.pem and a copy of the certificate in demoCA/newcerts/ named xx.pem, where xx is the serial number. We will copy the cert to a more descriptive name. The certificate has both the encoded version and a human-readable version in the same file. We want to strip off the human-readable portion as follows:
mv -i cert.pem temp.cert.sfa.example.com.pem

openssl x509 -in temp.cert.sfa.example.com.pem -out cert.pem
cp -i cert.pem cert.sfa.example.com.pem


Postfix will want the cert and the key in two separate files, apache2 will want the two combined (but can use two separate files if configured to do so).
cat privkey.sfa.example.com.pem cert.sfa.example.com.pem >key-cert.pem
cp -i key-cert.pem key-cert.sfa.example.com.pem

chmod -R 600 /root/CA


After those steps, you have three installable components (and some more descriptive backup copies):
A private key in privkey.pem (with a copy in privkey.sfa.example.com.pem)
A certificate in cert.pem (with a copy in cert.sfa.example.com.pem)
A combined private key and certificate in key-cert.pem (with a copy in key-cert.sfa.example.com.pem)

Now give a copy of the combined certificate to Apache2. We will configure sfa.example.com in Apache2 to use /etc/apache2/key-cert.sfa.example.com.pem:
/etc/init.d/apache2 restart

cd /root/CA/
cp key-cert.sfa.example.com.pem /etc/apache2/
chmod 600 /etc/apache2/key-cert.sfa.example.com.pem


Now we will configure Apache2. First enable the SSL module, and we will also enable the rewrite module so we can optionally redirect port 80 requests to port 443:
a2enmod ssl
a2enmod rewrite


We will make a copy of the default site. This copy will be used for configuration of the SSL site:
cp /etc/apache2/sites-available/default /etc/apache2/sites-available/ssl

Now edit /etc/apache2/sites-available/default:
vi /etc/apache2/sites-available/default

And change:
<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/
To:
<VirtualHost *:80>
        ServerAdmin webmaster@example.com
        ServerName sfa.example.com
        DocumentRoot /var/www/
Now edit /etc/apache2/sites-available/ssl:
vi /etc/apache2/sites-available/ssl

And change:
<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/
To:
<VirtualHost *:443>
        ServerAdmin webmaster@example.com
        ServerName sfa.example.com
        SSLEngine on
        SSLCertificateFile /etc/apache2/key-cert.sfa.example.com.pem
        DocumentRoot /var/www/
Once the files have been edited, enable the new site we called ssl, and restart Apache2:
a2ensite ssl
/etc/init.d/apache2 restart


If you did it like I did it, you should have no errors when it shuts down or starts up. If it does not start up, you can disable the ssl site with 'a2dissite ssl'. See /var/log/apache2/error.log for clues to the problem. Now you should be able to browse to both: http://sfa.example.com/mail and https://sfa.example.com/mail.

If you have not already installed the Root CA certificate on your computer, see if you can install it by downloading it via your browser at http://sfa.example.com/cacert.example.com.crt

Note that if you created a root certificate and installed it on your computer, and then started this process over and created a new one with the same name, you would have to remove the duplicate "test" certificate and install the new certificate. I have enabled port 80 only as a convenience. We don't want people to connect to our site without using SSL, so I am going to set up redirection. This is optional, but highly recommended. Edit /etc/apache2/sites-available/default once again:
vi /etc/apache2/sites-available/default

And insert these additional items in the location shown:
<VirtualHost *:80>
        RewriteEngine   on
        RewriteCond     %{SERVER_PORT} ^80$
        RewriteRule     ^(.*)$ https://%{SERVER_NAME}$1 [L,R]
        RewriteLog      "/var/log/apache2/rewrite.log"
        RewriteLogLevel 2
Then restart Apache2:
/etc/init.d/apache2 restart

Now, if you close down the browser window for http://sfa.example.com and then browse to it again, you should be redirected to the SSL site. This is a bit wasteful however as twice as many processes are required to maintain both port 80 and port 443. If you like, you can optionally prevent Apache2 from starting up processes on port 80 by editing /etc/apache2/ports.conf and commenting out 'Listen 80' then restarting Apache2. Of course, then your users will always have to remember to use the https:// URL. BTW, personally I use lsof -P | grep LISTEN to show what ports are in use. Note that I am an Apache noob. If you want to host multiple http sites with multiple certificates, I leave it up to you to figure out how to do it. I can tell you that you can create and sign as many certificates as you want, just make sure you do not create a new certificate authority. The one and only certificate authority you created can sign all your certificates. And from what I understand, if more than one site uses SSL, each site will need a different IP address. Google is your friend. To be clear, you can host as many email domains as you need, but in my setup, I access all of them through one and only one http interface. Let's continue on...

Tell Postfix where to find the certificates (and set a couple other TLS parameters). We also make a backup of main.cf before we modify it for the first time:
cd /root/CA/
cp -i cert.sfa.example.com.pem /etc/ssl/certs

cp -i privkey.sfa.example.com.pem /etc/ssl/private

cp -i /etc/postfix/main.cf /etc/postfix/main.cf-11dec2010

postconf -e "smtpd_tls_cert_file = /etc/ssl/certs/cert.sfa.example.com.pem"
postconf -e "smtpd_tls_key_file = /etc/ssl/private/privkey.sfa.example.com.pem"
postconf -e "smtpd_tls_CAfile = /etc/ssl/certs/cacert.example.com.pem"
postconf -e "smtpd_tls_received_header = yes"
postfix reload
tail -f /var/log/mail.log


And make sure Postfix is still working OK. Running some of these commands again will result in overwriting keys and certificates. That may not be good. Some files will necessarily be overwritten if additional certificates are requested, signed and created. That is expected, and is the reason we make host-specific copies of everything as we go along. Just be careful not to overwrite any host-specific files we have created. And remember, only one Root Certificate Authority needs creation. Make a backup of the session, both on and off the system (transfer the directory via WinSCP).
cp -r /root/CA /root/CA-11dec2010




PS:
Here is a hint if you would like to change the password used by amavis to access the MySQL database. The default MySQL password for the amavis user is 'passwd'.

Stop Apache2:
/etc/init.d/apache2 stop

Stop Maia (amavisd-new):
/etc/init.d/amavis stop

Locate and change the $maia_sql_dsn password in config.php:
vi /var/www/mail/config.php

Locate and change the @lookup_sql_dsn password in amavisd.conf:
vi /etc/amavis/amavisd.conf

Locate and change the @lookup_sql_dsn password in amavisd.conf-maia:
vi /etc/amavis/amavisd.conf-maia

Locate and change the password in /etc/maia.conf:
vi /etc/amavis/maia.conf

Locate and change the password in /etc/spamassassin/local.cf:
vi /etc/spamassassin/local.cf

Locate and change the password in /usr/sbin/maia-trim-awl
vi /usr/sbin/maia-trim-awl

Log in to mysql and change the password there, obviously, replace NEW-passwd with the new password:
mysql -p
SET PASSWORD FOR 'amavis'@'localhost'= PASSWORD('NEW-passwd');
FLUSH PRIVILEGES;
quit

Run configtest.pl to make sure the "Database DSN test" passes:
/var/lib/amavis/maia/scripts/configtest.pl

Start Maia (amavisd-new):
/etc/init.d/amavis start

Start Apache2:
/etc/init.d/apache2 start

It would be a good idea to run configtest.php:
http://sfa.example.com/mail/admin/configtest.php


Useful links:
You used 'subversion' (svn) to 'check out' the current version of Maia Mailguard.
To see what 'subversion' is capable of, see: http://svnbook.red-bean.com/

mr88talent at yahoo dot com
12 DEC 2010