Additional Anti-UCE settings

Additional anti-UCE settings for our Debian spamfilter.

This document is obsolete. There is a newer version at http://verchick.com/mecham/public_html/spam/additional_settings_v2.html

This HOWTO is a guide to substantially enhancing the spam fighting abilities of our Debian spamfilter (built using one of the documents found at http://verchick.com/mecham/public_html/spam/). The disclaimer at that address also applies to this document. Let me start by saying that the reason these instructions are not included in the original documents, and instead are included in this supplement, is for two reasons. One reason is these techniques will only work on a machine that is on the public Internet, and the other reason is there is additional risk involved. These techniques cannot be used on a spamfilter that sits behind another gateway server because they depend directly on the conversation with the mail server attempting to deliver mail to your network (the client conversation). There is increased risk because mail may get rejected by what I call "a jury of one", or some legitimate mail servers may possibly be unable to send mail through your server. That said, you can use these settings to substantially reduce the amount of spammy mail that is accepted by your server, and therefore substantially reduce the load on your server (and also reduce network traffic).

I am going describe setting up two different "policy servers" in Postfix similar to what is described in the SMTPD_POLICY_README. I am also going to illustrate the use of the reject_rbl_client restriction as described in Postfix_Configuration_-_UCE_Controls. The two policy servers I use are 'policyd-weight' and 'postgrey', and I use two RBLs (real-time block/blackhole lists): zen.spamhaus.org and list.dsbl.org. Of the hundreds of RBLs, these two are often regarded as being effective, and are considered to be effective with few false positives.

Of these three types of settings, I have found that policyd-weight (by design) is the least likely to reject legitimate mail and therefore I use it on my primary MX Debian spamfilter. If you only have one Debian gateway server, I suggest you use policyd-weight alone as opposed to the two other methods. If you have problems with policyd-weight, or simply prefer not to use it, then my second choice would be to use a single RBL: zen.spamhaus.org. Now, if you have built a second Debian spamfilter and it is currently in use on the Internet as your secondary MX (not round robin), then you can get more aggressive and take more risks with the mail that is sent to that machine. For that machine, I suggest using both postgrey and the two RBLs. I would not use postgrey on my primary MX server because there are legitimate servers on the Internet that do not have the ability to "try back later" when postgrey tells them that the server is temporarily unavailable. The simple idea behind a Greylist server is that when a new client connects for the first time, the postgrey service tells them the server is temporarily unavailable, then it waits a predetermined amount of time to see if the same client tries back later, and if it does, it lets that client send mail. Most spammer's servers will not try back later, so it is quite effective (for now). There are some Trend Micro servers (among others) that will not try back later, so they will not be able to send you mail unless you whitelist them in the Greylist server's whitelist table or a check_client_access table that we will create. To summarize: all your domains will use the server running policyd-weight as their primary MX server (priority 10 for example) and all your domains will use the server running postgrey and the two RBLs as their secondary server (priority 20 for example).

Using any of these settings requires that you consider what happens when a legitimate message is not able to get past these settings. We will need to add a mechanism that will bypass these checks for mail addressed to postmaster@ and abuse@, and we may need to allow other particular clients to bypass these checks, so we will set up a new check_recipient_access hash table, and a new check_client_access hash table.

A cautionary note on policyd-weight. Policyd-weight is beta software. Once you have the program set up and it is working without issues, don't be overly hasty to install a newer version. A little patience may reveal others who have problems with a newer version. Take the "if it's not broke, don't fix it approach" and keep your eye on the mailing list. Newer versions often have new features that make the upgrade very worthwhile, but as I said, have a little patience.

To see how effective these techniques are going to be for you, I suggest saving yesterday's pflogsumm report (or recreating it) so you can compare reports created over the next couple days to it. I think you will be amazed at how much more mail is rejected.

Let's download and install the current policyd-weight. You will want to begin by visiting http://policyd-weight.org/ and write down the version number we are getting today. The last time I updated this, the version was "0.1.12-beta-4", so we are going to make a directory that reflects the version we download, and place the files there. The program changes often, and sometimes there are issues with a new version, so we always want to keep old versions in case we need to use them. If you are upgrading policyd-weight from a version older than 0.1.12 beta, it is now required that a user and group 'polw' is created; policyd-weight will no longer run as user 'nobody' and policyd-weight.conf and master.cf must be updated to reflect this change. Modify this as needed to reflect the current version:

mkdir /usr/local/src/policyd-weight-0.1.12-beta-4

cd /usr/local/src/policyd-weight-0.1.12-beta-4

wget http://policyd-weight.org/policyd-weight

wget http://policyd-weight.org/policyd-weight.conf.sample

I am now going to configure the program to suit my preferences by editing the sample .conf file, then move it into place. I suggest using these settings until you get more familiar with the program. Note that I'm using the vi editor here, but of course you can use a different editor if you desire:

vi policyd-weight.conf.sample

Find the line:
$REJECTLEVEL = 1;
and change it to a more conservative (this number is from my personal experience, yours may differ):
$REJECTLEVEL = 4.24;

Find the line:
$ADD_X_HEADER = 1;
and change it to (a personal choice of):
$ADD_X_HEADER = 0;

Find the line:
$syslog_ident = "postfix/policyd-weight";
and change it to :
$syslog_ident = "postfix/policydweight";

Save and exit the file. We changed from policyd-weight to policydweight as a telltale that policyd-weight is reading our .conf file, and to make it easier to configure logcheck to ignore policyd-weight's log entries. Now we can move our program into place, and set permissions:

cp policyd-weight /usr/lib/postfix

chmod 0755 /usr/lib/postfix/policyd-weight

cp policyd-weight.conf.sample /etc/policyd-weight.conf

If this is the first time installing policyd-weight, create a user and group 'polw':

groupadd polw

useradd -d /var/polw -s /bin/false -g polw polw

Then make our chrooted Postfix aware of the new user:
cp /etc/passwd /var/spool/postfix/etc/passwd

We need to add a few settings to main.cf:

cd

postconf -e "smtpd_policy_service_timeout = 240s"

postconf -e "policy_time_limit = 3730"

postconf -e "smtpd_policy_service_max_idle = 900s"

I personally also like to cut in half the number of hard errors allowed before we cut a client off, this cuts off dictionary attackers a little quicker.


postconf -e "smtpd_hard_error_limit = 10"

postconf -e "smtpd_soft_error_limit = 8"

Edit master.cf and add these two lines (just above the smtp-amavis line would be a good place):
vi /etc/postfix/master.cf

policy	unix	-	n	n	-	-	spawn
                user=polw argv=/usr/bin/perl /usr/lib/postfix/policyd-weight

Save and exit the file. Now we need to create a couple new hash files for Postfix to use:

vi /etc/postfix/roleaccount_exceptions

And insert the following (insure you have a carriage return at the end of the last line):


# mail addressed to these recipients are allowed to bypass RBL checks

postmaster@ OK

abuse@ OK

Save and exit the file, postmap it, and create and postmap the next file:

postmap hash:/etc/postfix/roleaccount_exceptions

vi /etc/postfix/rbl_client_exceptions

And insert the following:

# these client IP addresses are allowed to bypass RBL checks

66.35.250.225 OK

(that IP address is a 'picked at random' sourceforge server) Then postmap that file:

postmap hash:/etc/postfix/rbl_client_exceptions

Now we need to vi /etc/postfix/main.cf and add a few things to the smtpd_recipient_restrictions restriction stage. Let's make an assumption your smtpd_recipient_restrictions looks similar to this at this time:

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination

It is VERY important these lines are added AFTER "reject_unauth_destination". We want to add three lines below (after) reject_unauth_destination, so the result looks something like this:

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination,
    reject_unauth_pipelining,
    check_recipient_access hash:/etc/postfix/roleaccount_exceptions,
    check_client_access hash:/etc/postfix/rbl_client_exceptions,
    check_policy_service unix:private/policy

The white space in front of the settings above is required. This is true whenever you extend a Postfix setting to more than one line. Note that the commas are actually optional, Postfix can use either a comma, or a space to separate settings. Policyd-weight requires the use of reject_invalid_hostname. You can place this in smtpd_recipient_restrictions prior to check_policy_service or as is shown in this example in smtpd_helo_restrictions:

smtpd_helo_restrictions = reject_invalid_hostname

OK, policyd-weight is installed, and all that remains is to reload Postfix, AND WATCH OUR LOG FOR ERRORS:

postfix stop

postfix start

tail -f /var/log/mail.log

Note that because we have "permit_mynetworks" prior to "check_policy_service", you cannot test policyd-weight by sending messages from a computer listed in $mynetworks, and you cannot test with messages sent to postmaster@ or abuse@. Keep an eye on the log as mail goes by. Watch it for at least 10 messages or so. Use [Ctrl]+c to return to the shell prompt. If there are errors that relate to policyd-weight, remove (do NOT comment out) the "check_policy_service unix:private/policy" line in main.cf, and reload Postfix to get your mail going again. I don't believe you can comment out a line in a multi-line statement in Postfix (unless it is the last line) because Postfix may think the line(s) below the commented out line(s) are separate statements. It is normal to get a "could not connect to cache" message the first time Postfix loads policyd-weight, but you should not get this error repeatedly. Hopefully all went well. Note that Postfix controls the startup and shutdown of policyd-weight, we do not start the daemon outside of Postfix. Now, assuming you have logcheck installed, ( apt-get install logcheck logcheck-database if you don't) every single line in our log that is created by policyd-weight will trigger a line in a logcheck report so we need to tell logcheck to ignore those lines:
vi /etc/logcheck/ignore.d.server/postfix

and at the top of the file, insert on a line by itself, the text:
policydweight

I suggest you subscribe to the (low volume) policyd-weight mailing list. I also suggest you read more about the program to gain an understanding of how it works. I think of it as a 'miniature SpamAssassin'. Here is a useful command:


 egrep -o rate:.* /var/log/mail.log

###########################################################################################################

Now we move on to installing postgrey. Remember that I do not recommend Greylisting on the primary MX, and I also personally do not recommend combining it with policyd-weight on the same server. I believe it is quite appropriate for a secondary MX however. Since most mail sent to a secondary server is spam, it is also quite effective there. We start by installing postgrey:

apt-get update

apt-get install postgrey

By default postgrey will Greylist for 300 seconds. I noticed that Yahoo retries every 60 seconds, so I think it's a good idea to change the delay from 300 to 50 seconds so a valid server like that does not retry so many times that it gives up. Shortening the time does not seem to make the program any less effective.


vi /etc/default/postgrey

and change:
POSTGREY_OPTS="--inet=127.0.0.1:60000"
to:
POSTGREY_OPTS="--inet=127.0.0.1:60000 --delay=50"

Since we made this change, we need to stop and start (not reload) postgrey:

/etc/init.d/postgrey stop

/etc/init.d/postgrey start

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination

It is VERY important these lines are added AFTER "reject_unauth_destination". We want to add four lines below (after) reject_unauth_destination, so the result looks something like this:

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination,
    reject_unauth_pipelining,
    check_recipient_access hash:/etc/postfix/roleaccount_exceptions,
    check_client_access hash:/etc/postfix/rbl_client_exceptions,
    check_policy_service inet:127.0.0.1:60000

Postgrey is the server that listens on port 60000.
By adding postgrey, the /etc/passwd file has changed, so we will give Postfix an updated copy of it:
cp /etc/passwd /var/spool/postfix/etc/passwd

As we did on our other server:


postconf -e "smtpd_hard_error_limit = 10"

postconf -e "smtpd_soft_error_limit = 8"

OK, postgrey is installed, and all that remains is to reload Postfix, AND WATCH OUR LOG FOR ERRORS:

postfix stop

postfix start

tail -f /var/log/mail.log

Keep an eye on the log as mail goes by. Watch it for at least 10 messages or so. If there are errors that relate to postgrey, remove (do NOT comment out) the "check_policy_service unix:private/policy" line in main.cf, and reload Postfix to get your mail going again. If someone complains they are unable to send you mail, you can whitelist their server in /etc/postgrey/whitelist_clients or our rbl_client_exceptions map. There is also a /etc/postgrey/whitelist_recipients file that does the same thing as our roleaccount_exceptions map.

Now, adding the two RBL client checks is simply a matter of placing them in front of postgrey. They of course can be used by themselves if you prefer.

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination,
    reject_unauth_pipelining,
    check_recipient_access hash:/etc/postfix/roleaccount_exceptions,
    check_client_access hash:/etc/postfix/rbl_client_exceptions, 
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client list.dsbl.org,
    check_policy_service inet:127.0.0.1:60000

OK, the RBL checks are installed, and all that remains is to reload Postfix, AND WATCH OUR LOG FOR ERRORS:

postfix stop

postfix start

tail -f /var/log/mail.log

http://policyd-weight.org/
http://isg.ee.ethz.ch/tools/postgrey/
http://www.spamhaus.org/xbl/index.lasso
http://dsbl.org/main

mr88talent at yahoo dot com
12/24/2005